Hippoformer: Integrating Hippocampus-inspired Spatial Memory with Transformers
Authors: Tiantian Li
Xingxing Cao
Yifei Wang
Xiaojiao Yang
Xiaolong Zou
Bo Hong
Venue: ICLR 2026 Poster (2026)
Keywords: Hippocampus
grid cell
spatial reasoning
relational memory
transformer
Primary area: applications to neuroscience & cognitive science
TL;DR: We propose Hippoformer, a hybrid architecture that integrates hippocampal-inspired structured spatial memory with transformers, enabling scalable spatial reasoning and outperforming existing models in 2D and 3D grid tasks.

Abstract: Transformers form the foundation of modern generative AI, yet their key–value memory lacks inherent spatial priors, constraining their capacity for spatial reasoning. In contrast, neuroscience points to the hippocampal–entorhinal system, where the medial entorhinal cortex provides structural codes and the hippocampus binds them with sensory codes to enable flexible spatial inference. However, existing hippocampus models such as the Tolman-Eichenbaum Machine (TEM) suffer from inefficiencies due to outer-product operations or context-length bottlenecks in self-attention, limiting their scalability and integration into modern deep learning frameworks. To bridge this gap, we propose mm-TEM, an efficient and scalable structural spatial memory model that leverages meta-MLP relational memory to improve training efficiency, form grid-like representations, and reveal an intriguing link between prediction horizon and grid scales. Extensive evaluation shows its good generalization on long sequences, large-scale environments, and multi-step prediction, with analyses confirming that its advantages stem from explicit understanding of spatial structures. Building on this, we introduce Hippoformer, which integrates mm-TEM with Transformer to combine structural spatial memory with precise working memory, achieving superior generalization in both 2D and 3D prediction tasks and highlighting the potential of hippocampal-inspired architectures for complex domains. Overall, Hippoformer represents a initial step toward seamlessly embedding structured spatial memory into foundation architectures, offering a potential scalable path to endow deep learning models with spatial intelligence.

Published as a conference paper at ICLR 2026
HIPPOFORMER: INTEGRATING HIPPOCAMPUS- INSPIRED SPATIAL MEMORY WITH TRANSFORMERS
Tiantian Li
Xingxing Cao
Yifei Wang²
Xiaojiao Yang!
Xiaolong Zou Bo Hong!
Department of Biomedical Engineering, School Medicine, Tsinghua University
2 Qiyuan Lab
Equal contribution, X Corresponding author
ABSTRACT
Transformers form the foundation of modern generative AI, yet their key- value memory lacks inherent spatial priors, constraining their capacity for spatial reasoning. In contrast, neuroscience points the hippocampal- entorhinal system, where the medial entorhinal cortex provides structural codes and the hippocampus binds them with sensory codes to enable flex- ible spatial inference. However, existing hippocampus models such the Machine (TEM) suffer from inefficiencies due to outer- product operations context-length bottlenecks in self-attention, limiting their scalability and integration into modern deep lcarning frameworks. To bridge this gap, we propose mm-TEM, an efficient and scalable struc- tural spatial memory model that leverages meta-MLP relational memory improve training efficiency, form grid-like representations, and reveal an intriguing link between prediction horizon and grid scales. Extensive evaluation shows its good generalization on long sequences, large-scale en- vironments, and multi-step prediction, with analyses confirming that its advantages stem from explicit understanding of spatial structures. Build- ing this, we introduce Hippoformer, which integrates mm-TEM with Transformer to combine structural spatial memory with precise working memory, achieving superior generalization in both 2D and 3D prediction architectures tasks and highlighting the potential of hippocampal-insste for complex domains. Overall, Hippoformer represents initial step toward seamlessly embedding structured spatial memory into foundation architec- tures, offering a potential scalable path to endow deep learning models with spatial intelligence.
1 INTRODUCTION
The Transformer architecture has driven the recent advances in generative AI, with systems such as ChatGPT as prominent examples. This success has made the search for new archi- tectural designs a central direction in machine learning. Transformer can be viewed as asso- cialive memories implemeuled (hrough key-value caches and sell-attention retrieval relrieval[Vaswarl [Vaswani al.. 2017: Geva al., 2021: Ramsauer et al,, 2020). However, they face inherent lim- itations, most notably quadratic computational cost and redundant memory, which limit their scalability (Zhuang et al., 2023). To address these issucs, many alternatives have been proposed by reconsidering memory design. For instance, Titans leverage fast MLP weights BBrbeyy et al., 2024).. Although these approaches improve long-sequence modeling, their memory structures remain largely flat and lack a critical element: an in- herent spatial memory. Such a memory, however, is vital for organizing the "what-where" of experiences and for building internal models Yang et al (2025). Therefore, developing architectures with structured spatial memory and integrating them into modern frameworks remains an key open challenge toward efficient spatial reasoning.
for large-capacity [Behrouz

---

Published as a conference paper at ICLR 2026
The hippocampal-entorrie (acprhiiiaHHHECCCC system is central to spatial and cod_eppisiii mem- orydBuzsákk Moser, 2013; Eichenbaum et al, 2007: Eichenbaum 2017: Whittington et al, 2022). Experiments highlight two key computational principles: the finctional dissociation between MFC and LEC (aatwwaayaaarrreeeee et al, 2005; Knierim et al 2014). and the hippocampus's role in binding structural and sensory anfornation(Eichenbaum (Fichenbaum et al 2007).
These insights have motivated rich computational models with factorized struchural and sensory perreenttaionss(lasseinn
et all, 2002; Franzius et all 2007; Bush et all, 2015). structural codes and the hippocampus binds them with LEC-dcrived sensory inputs. Building on this foundation, several learnable HC- EC-inspired models have been proposed, including CSCG.Geerye el al. 2021; Raju et al, 2024), Tolman Eichenbaum Machine (TEM) (Whittington
where MEC provides
al, 2020). and Vector- HaSH et al,, 2025). As shown in Fig. [AB, TEM oflers unilied computational framework that learns abstract structure
an unsupervised manner and generalizes this structure to novel environments, making it a promising basis for hippocampus- like mem-- ory models However, existing HC_FC models. focus mainly
faiusy on simplified synthetic sct- al,, 2020; Raju ct al, 2024; Zou et all, 2024; Chandra et al,, 2025). How to integrate these properties with modern deep learning architectures and scale them to richer, real-world tasks remains an open challenge.
tings(Whtttintoog
For example, the original TEM uses tensor-product Hebbian weights for relational mem- ory, which biologically plausible but capacity-limited TEM-t replaces these with key- value memory and gelf-attention-basee retrieval, improving efficiency but still incurring high computational cost(WWhitiintton al, 2021). Moreover, inherits the constraints of transformer-basee architectures, such as limited context windows. Furthermore, both models demand careful memory management and paramcter tuning to realize novelty- based storage and retrieval. Torether, these limitations hinder the practical integration of hippocampallinnsiiee spatial memory into modern deep learning, despite their intriguing conceptual motivation.
To address these challenges, we introduce mm-TEM, a more scalable and efficient hippocampus-inspieee structural memory, and Hippoformer, hybrid model that integrates mm-TEM with transformers. mm- MEm introduces a mcta-MLP memory system, meta- trained for associative binding. Building on this, Hippoformer combincs this mm-TEM with transformer, yiclding complementary strengths. Despite their simplicity, both models achieve good performance on long-houd horizon prediction tasks in 2D and 3D environments. Our main contribution are:
1. mm-TEM: We propose an efficient and scalable TEM variant with newly de- signed MeL based relational memory. mm-TEM substantially improves train- ing efficiency over TEM, generates grid-like patterns through self-supervised learn- ing, and uncovers an intriguing link between prediction horizon and grid scales, offering new insights into how different spatial grid scales are formed at the imple- mentation level.
2. Systematic evaluation: mm-TEM is extensively tested on long sequences, large- scale environments, and multi-stop prediction. It generalizes significantly better than baselines such as transformers and Titans. Ablation studies illustrate the importance of the auxiliary relational loss, and further analyses show that its gen- eralization stems from explicit understanding of spatial structures and rules, demon- strating mm- TEMT as an effective structural spatial memory system.
3. Hippoformer: We propose Hippoformer, which integrates mm-TEM with trans- former to combinc the structural spatial memory of mm-TEM with the precise working memory capability of Transformer. This synergy enhances generalization in both 2D and 3D prediction lasks, demonstrating the potential of hippocampal- inspired architectures in tackling complex domains.
In summary, mm-TEM provides an efficient and scalable structural spatial memory system. And when combined with Transformer, Hippoformer has an potential to serve as a building block for enhancing spatial reasoning in deep learning.
2

---

Published as a conference paper at ICLR 2026
2 METHOD
In this section, we present the mm-TEM and Hippoformer architecture in detail. We use the 2D grid-world prediction task as an example, where an agent moves with discrete ac- tions (up, down, left, right) (Whittington et al,, 2020). The input sequence denoted as do, 5o, d1, ...., ar, St, St, where hot action. The model is trained to prodict the next sensory obsevation 8t-1 given 4+12 thereby mimicking hippocampal predictive coding during spatial exploration Whittington et. al. (2022). The overall model structure is ratustrated in Fig.
I' is the sensory observation at time and one-
A
B
Environment 1
Environment
Relational Meroory
HPC
MEC
LEC
Structural code
Sensory code
C
D
Reconstruction
6
Morsry PC:
Relational Memory
Recanwnoe
Wecko
Cyyt
Cuta
Figure 1: Factorization of structure and content in the hippocampus and model. (A) The hippocampal entorhinal -entrammallentoohinnll information, LEC encodes sensory content, and HPC integrates both via conjunctive coding. (B) Structural codes in MEC can he reused across engironments, enabling compositional generalization, adapted from Whittington
system functions as a memory system: MEC encodes structural
al (2020). (C) The model comprises two components: a path integration network and relational memory network, implemented as an meta-MLP memory. (D) The relational memory module is trained reconstruct
sensory codes from structural codes, structural codes from sensory codes, and both from joint inpuls.
2.1 MODEL ARCHITECTURE AND TRAINING
previionng-peevion previons modeling studies (Hasselmo et al 2002 Franzius et al. 2007: Bush et al 2015; Stachenfeld atl 2017: Waniek, 2019; Rajan et al, 2016) and TEM computational framework (Whittington et al., 2020), mm-TEM consists of two key modules: path inte- gration network and relational memory network (Fig. C).. The path integration network receives action inputs a, and predicts the corresponding structural code St, while the re- lational memory network binds Se with the sensory code Ht, extracted from observations through feature encoder. This design enables flexible bidirectional retrieval between structural and sensory domains.
Path Integration Network. Inspired by grid system in MEC( et al. 2017). the network enforces hasic spatial consistency rules (e.g., North + East + West South 0). Following Gao et al (2021). we implement it a two-layer MLP fg with ReLU activations to map the action a, Id to transformation matrix W E IR xda;
W = (a.).
(1)
The structural code then updated as
9t
g. Re
=
|
(2)
3

---

Published as a conference paper at ICLR 2026
where ey-normalization ensures that 9: remains unit vector.
Relational Memory Network. Mimicking hippocampal relational memory (Eichenbaum
et all 2007), the network binds structural and sensory codes into joint representation my It. zel, enabling bidirectional retrieval between 9i and It. To replace the computa- tionally expensive Hcbbian weights in TEM, introduce a meta-MLP with hierarchical fast weights c. inspired by Titans, to store relational knowledge. This design enables dy- namic memorization, forgetting, and querying at test time, alleviating the complex memory management and parameter tuming required in TEM and TEM-1.
Concretely, the relational network first projects my into three latent vectors:
h = Wgms, 0t = m, gt = =W
(3)
where ke, Ut, qr denote the key, value, and query representations, respectively. Rather than storing my directly, the mcta-MLP learns to associate key hy to value in online by minimizing the reconstruction loss:
[kk, '1: θ2) = ||MMLP 02) -ellzz
(4)
where fM O2) denotes the mcta-MLPP The fast weights Or are updated by incorporating prodiction-aroorddrivee adaptation and forgetting:
θ = (1 01) He.
(5)
= H - 3r Volli, vy; H,-1)-
(6)
Here, a₁ E [0, a data-dependent gating variable that controls forgetting, paralleling hippocampal mechanisms that decay less relevant memories to preserve capacity for novel ones(Beeoott & Anderson 2012 Liu et al, 2016). VeL(.) quantifies the gradient of prediction error the degree of mismatch between the model's predictions and the input data so that only unexpected inputs drive updates, akin to how hhw_hhiphhoee hippoccapppp detects and prioritizes novel stimuli for long-term storage(Sinclair et al,, 2021; Schomaker & Meeter, 2015). The term f acts as a momentum factor__averee averaging prediction error over a tunable timescale to stabilize (Bitter (Bittner et al,, 2017), while Bg is the learning rate. All paramcters are derived from the imput concatenation: ar = (,W,,)), Tie = oWW,,,) 3r = (W),)), where is the sigmoid function.
The query vector q retrieves from memory via fMLP(g: Ot), and the retrieved representation is explained a joint reconstruction mh 0t..
To explicitly enforce relational binding, we introduce three auxiliary relational losses (Fig. ID):
(1) Structure from content: retrieve de given only de (m, = 21)) minimized by Luzg
ll0: - -1.1...
(2) Structure from structure: retrieve given only St (me [gt, 0)), minimized by Lyz0
ll 9e.12.
(3) Content from the structure: retrieve & given only ge (m, Ioc. 0)), minimized by
fy2z = 1| - 11.
The total relational loss Lrel = Lazg + Lggg + Lggs- Since Lgze will be accounted for in the main predictive-learmnt
objective, it can be absorbed into that term. Thus, the relational loss can be simplified to Liel Lx2g + Lg29-
Finally, mm-TEM incorporates feedback loop from relational predictions to the path inte gration network, providing error correction during navigation (Fig. IC; see Appendix. A.2 for details).
mm-TEM Training. The objective mm-TEM is to predict the next observation given past sensory inputs and actions. The model is trained in self-supervised manner. During training, in the relational memory network, the projection matrices Wg. W., Wg are meta- trained in the outer optimization loop, while the connection weights of the meta-MLP are optimized in the inner optimization loop. During testing, the connection weights of the meta- MLP updated online using gradient-based update rules. We introduce hyperparameter
4

---

Published as a conference paper at ICLR 2026
mb to control the memory update frequency in the relational memory network. Specifically, the connection weights in the meta-MLP are updated every mb steps. A larger mb results in sparser updates, which improves training efficiency but requires the model to rely older information when predicting the next observation. Before downstream task training, warm up the relational memory network through meta-training with random {ae, 8. samples, utilizing only the relational loss. This procedure helps stabilize training, All networks are optimized using the Adam optimizer. Additional training details and the objective loss function are provided in Appendix. A.5. For all primary results, models are trained on sequence lengths of 128 or 256 using full backpropagation through time (BPTT) without Iruncation(See Fig. A.5 for ablation).
Hippformer Architecture and Its Training. Building on mm-TEM, we propose Hip- poformer, which integrates mm-TEM and one-layer Transformer in parallel to leverage the complementary strengths of both modules. In Hippoformer, the mm-TEM component is also warm-started following the same protocol described above. Rather than receiving the path integration network's 's output Is: the Transformer architecture here takes an ac- tion embedding generated by single-layer MIP (Kitaev et al, 2020). This embedding is concatenated with a and 8, to form the final imput. The model is trained using the Adam optimizer. Additional architectural and training details provided in Appendix. A.5
Control Models. We used Transformer and Titans as control models. They receive identical input tokens to those used by the the transformer module in Hippoformer. Unless otherwise stated, all models are trained and evaluated under the same task setting for fair comparison.
3 RESULTS
3.1 EFFICIENT TRAINING AND EMERGE OF GRID-LIKE REPRESENTATIONS MM-TEM
A
B
Training convergence
Grid examples (lop5)
1.0
21.3
(8:2177
11
0.8
mb=1
Accuracy
0.6
0.4
mbm4
0.2
mm-TEM
0.0
Adtooss
mb=8
Obs
5000 10000
15000 20000
Context
Imagination
Gradient
Figure 2: Task schematic and network performance. (A) A 2D Grid prediction task (example 6x6). The network predicts the next observation based on the current action at each step in sequence. (B) Multi-step test accuracy over gradient steps for mm-TEM vs TEM (same batch/seque sequuecee training length), averaged over seeds. To assess generalization, the networks predict till 256 steps from initial 64-step context, with resultss averaged four trials. More training details and parameters, see Appendix. A.1. A.2 and A.5 (C) In the path integration network, emerged grid scale varies with hyperparameter m/h Fiye top-gridness neurons shown per condition (more examples Figs. A.12 A.13 and A.14).
We first ask whether mm- TEM can efficiently solve spatial reasoning task and acquire MEC-like represeniation To test this, we evaluate the model on 2D grid prediction tasks (Fig. 2 (W)htiitnng et al. 2020), where the agent must predict the next obsorvation based on the current action at each time step within a 256-step sequence. For each trial, the environment is sized between 9x9 and 11x11 with randomized observations, requiring the network to infer the underlying spatial structure and rules to generalize effectively. Since observations at unseen locations within this discrete environment are unpredictable, both training and evaluation are confined to positions previously encountered within each sequence. More evaluation details, see Appendix. A.4.
5

---

Published as a conference paper at ICLR 2026
In terms of spatial reasoning performance, mm-TEM reaches nearly 90% test accuracy within only 5,000 gradient steps, while TEM converges very slowly, achieving only about 60% accuracy even after 20,000 steps (Fig. 2B), highlighting the superior training efficiency of mm--EMM.
In terms of internal representations, analysis of the path-integration network of mm- mm-TEM further reveals periodic grid-like representations (Fig. 2C). Notably, the grid scale is directly modulated by the update-frequency hyperparameter mb: larger mb yields coarser grids, whereas smaller mb produces finer scales. Since mb sets the effective pereetiivveeeeeettiiinnn prediction horizon, This suggests a novel mechanism for grid-scale diversity in MEC (Fyhn et al, 2004) at the implementation level, and further demonstrates that diverse grid scales can naturally arise from mulji-timessale predictions the brain Waniek (2019); Stachenfeld et al. (2017); Dordek et al. (2016). Overall, mm-TEM not only trains efficiently but also reproduces grid- like patterns the HC-MEC system, offering new insights into the computational basis of grid-scale diversity.
3.2
GENERALIZATION OF MM- MM-TEM IN 2D GRID PREDICTION TASKS
A
1-stap Accuracy
B
Multistop Accuracy
Ablation
1.0
I-tipp acuracy
Muti-olep Accurecy
2.0)
0.8
0.8
0.6
0.6
We
6.4
0.4
0.2
2.4
Tandhmmrn
2
27
2
231
2>
20
2
Contect Langth (log staln)
Imagtwiion Length (ing scale)
Contect wrigh (bgg scale)
Imagnation L Longt (og sovle)
D
E
F 1.0
3-000
6427
0.537
0.6
Aczuurery
Conmest Input
0.4
0.4
0.2
0.2
0.002
round
Imagiratin
CCW
0.0
7k7
L1x11
3.3x133
15:15
Walk Dirction
Emvronment Size
Figure 3: The Generalization and Ablation of mm-TEM. We compare mm-TEM with Titans and Transformer baselines on 2D grid prediction tasks (training length = 128). Titans uses a 3- 3-layer MAC architecture. Transformer128 and Transformer64 denote 3-layer transformers with window sizes of 128 and 64 steps, respectively. (A) 1-step prediction accuracy vs context length. Networks receive action observation context sequence and predict the next observation. (B) Multi-step imagination accuracy vs imagination length. Networks observe fixed 64-step context, then generate future observations conditioned on varying action lengths. (C) Ablation of auxiliary relational loss. "w/o g2g" removes Lggg, "w/o s2g" removes Lu2g: "w/o rel" removes all auxiliary relational memory losses. (D) Circular- grid test setup. Networks explore an 11x11 circular environment clockwise for context, then imagine trajectories in clockwise or counterclockwise directions. (E) Clockwise vs counterclock wise performance across different architectures. (F) Effect of environment sive. ranging from 7x7 to 15x All results are averaged over 3 seeds (see Appendix. A.3 A.5
Fig.
Fig. A.4 and Fig. A.11 for more details).
Mimicking the HC EC system, mm-TEM acts as a structured memory that organizes knowl- edge for generalization. We ask: how well does such system generalize compared to mod- ern architectures like Transformers and Titans? To answer this, we systematically evaluate mm- mm--TEM against these baselines in diverse settings. In the one-step imagination setting, models explore environments with varying context lengths and predict the next observation. In the multi-step imagination setting, models receive a fixed 64-step context and predict fu-
6

---

Published as a conference paper at ICLR 2026
ture observations conditioned action sequences of varying lengths. These tasks probe mm- mm-TEM's ability to generalize beyond its training horizon.
In the one-step prediction task (Fig. [A), all models perform well within the 128-step training horizon. However, Transformer and Titans rapidly degrades once the context length extends beyond this range. In contrast, mm-TEM maintains more robust performance even with sequences up to 4096 steps, retaining 40% accuracy where basclines collapse, highlighting its long-term generalization ability.
In the multi-step imagination lask (Fig. BB), the Transformer model with a 128-step win- dow performs almost perfectly within its training range, but quickly drops off outside it, suggesting reliance on sequence memorization. Titans show similar behavior. In contrast, mm-TEM maintains relatively robust long term-termmnn performance, suggesting that it has grasped the underlying spatial structure for gencralization.
To determine the role of auxiliary relational loss (Fig. CC) on this ability, we conduct ablations. Removing either Lggg or Lazg significantly reduces generalization ability, and eliminating all relational terms leads lo severe performance degradation, confirming their importance.
we Moreover, we further probe generalization under distribution shifts. In the circular-grid test (Fig. (D), mm-TEM achieves over 90% accuracy in the challenging counterclockwise condition, while Titans and Transformer suffer accuracy drops by up to 30,, underscoring mm- T superior spatial reasoning ability. When scaling environment size from 7 x to 15 x 15 (Fig. (F) without additional training, all models decline, but mm-TEM deteri- orates much more slowly and consistently outperforms the baselines. These results further show that mm-TFR generalizes beyond its training horizon, and captures spatial struc- ture and rules more faithfully than control models, which appear to primarily rely on rote memorization.
A
Grid-accuracy correlation
B
58 S41
62 S41
43
59 L9I
52 ID 641
1.0
High-High
0.9
r=0
Accuraty
p=0.0002
32 (3,00)
27 CRI
0.8
Low-High
grid High Low aird-High ace
0.7
42 (2.00
1.10 (00448
S1 (2.47)
62 (D:25)
LoweLow
0.6
0.6
0.8
1.0
1.2
1.4
Grid Score
Topdl raurons
Other typical neumnn
Figure 4: Long-horizon generalization and grid representations. (A) Multi-step prediction accuracy (imagination length 512) positively correlates with grid score in mm-TEM path 0.0002), indicating that stronger grid-like regularity
integration networks (r = 0.647, supports better generalization. (B) Representative models with high-high, low-high, and low-low grid-acc aicd-accuracy combinations show distinct autocorrelation patterns. For each model, the three neurons with the highest grid scores ("Top3 neurons") and other typical neurons displayed, highlighting differences in grid-cell regularity acmss models. More details are diffeerreeiffeeeeeeee
provided in Appendix. A.5. Fig. A.8. Fig. A.15 A.16 and A.17
To uncover why mm-TEM exhibits good generalization in long-horizon inference, we exam- ine the relationship between model's grid score and its multi-step imagination accuracy. As shown in Fig. 4A, multi-step generalization performance in mm- Min-TEMii is closely tied to the quality of its grid-like representations. Models with higher grid scores in the path-integration network cot.rssstttsssttttyycco achiove higher prediction accuracy (further analysis provided in Ap- pendix. A.S), indicating a positive correlation between grid score and prediction accuracy.
Interestingly, we also observe cases where models with relatively low grid scores still achieve relatively high accuracy. Visualization (Fig. (B) reveals that these models develop alterna-
7

---

Published as a conference paper at ICLR 2026
tive but still regular- neural representations, in contrast with the unitary, unstructured patterns found in models with both low grid scores and low accuracy.
Taken together, these results suggest that the presence of strongly grid-like cells may reflect effective structure learning, thereby facilitating long-horizon generalization.
3.3 HIPPOFORMER BENEFITS FROM SHORT- AND LONG-TERM MEMORY INTEGRATION
From memory perspective, Transformer with limited window size functions as precise short-term memory through accurate key-value caching, while mm-TEM provides a struc- tured but. less precise long-term memory. To leverage their complementary strengths, we propose Hippoformer, a unified architecture that combines a one-layer Transformer with mm-TEM. Both modules process the input embeddings independently, and their outputs are concatenated and integrated by an MLP (Fig. AA). We evaluate all models on the 2D grid prediction task.
B
1-stap Acouracy
Accuracy
1.0
0.8
0.6
Aosurcs
0.6
0.4
HopoFooeer man-TEM
Erbadsing
Imagiattion
Figure 5: Hippoformer architecture and generalization in 2D grid prediction. Both Hippo- former and mm-TEM trained using mb = 8 and 256-step sequences. (A) Hippoformer combines a one-layer Transformer and mm-TEM, both receiving action and sensory em- boddings; their outputs are concatenated and integrated by MLP. (B) One-step predic- tion accuracy of Hippoformer and mm-TEM across different context lengths. (C) Multi- step imagination accuracy across different imagination lengths, comparing Hippoformer and mm-TEM. Additional tests for effects of the memory update frequency mb are provided in Appendix. A.6 and Fig. A.2
As shown in Fig. B, mm-TEM with mbb = 8 can be trained elficiently, but its one-step pre- diction drops at short context. lengths due to limited access lo recent information, requiring longer contexts to reach good performance. When combined with Transformer, however, Hippoformer generalizes across both short and long context lengths. In the multi-step imag- ination task (Fig. EC), where performance depends primarily on the mm-TEM component, both models achieve similar accuracy with no significant difference.
Overall, Hippoformer successfully integrates the strengths of both memory systems. The Transformer provides short-term memory for accurate short-range prediction, while mm- TEM supports structured long-horizon forecasting. This hybrid design is appealing for applications, as reducing MT.P memory update frequency in mm-TEM greatly improves training efficiency and minimizes redundant memory storage, though at the cost of short- term accuracy (see Appendix. A.6 and Fig. A. Consequently, Hippoformer achieves both efficient training and good gencralization across diverse temporal horizons.
3.4 HIPPOFORMER LEVERAGES THE SYNERGY BETWEEN ABSTRACTION AND MEMORIZATION
The hippocampus supports not only memorization but also abstraction, whereas traditional TEM and TEM-t models primarily emphasize memory storage and memory-based infer- ence. In contrast, mm-TEM moves beyond structured memorization by enabling abstrac- tion through parametric relational memory. Hippoformer merges Transformers' short-term memory with mm- mm-TEEM's long-horizon abstraction to boost generalization. this capacity, we design 3D empty environment task (as shown in Fig. BAB) (Pasukonis et al. 2022). In this new setting, layout textures and egocentric trajectories are randomly sampled. Observation features extracted through encoder, concatenated with action
8

---

Published as a conference paper at ICLR 2026
A
C
GroundTruth
HippoFoomer
Example obrenvtion inputs
Transformer
Titans
Example action Inputs
x A
S1
Step
Stop Stop 41
Step
Step
Step
Figure 6: Hippoformer generalization in 3D environment prediction tasks. (A) Example
3D environment with randomly sampled layouts and navigation trajoctorics. (B) Example trajectory showing sequences of egocentric observations and actions. (C) Visualization of imagined trajectories from different models, with snapshots shown every 5 steps. More
details Appendix. A.9 and Fig. A.18 and A.19
inputs, and fed into the models. Each model is trained predict the next egocentric frame over 64-step sequences. They are then evaluated on both one-step and multi-step predic- tion, in a manner similar the 2D grid experiments. Performance mcasured using the reconstruction error of predictions in pixel space (see Appendix. A.4 for details).
Table Performance comparison of different models on prediction error in 3D environments. The results are shown for both one-step and multi-step conditions, with errors reported in units of - The results are averaged over seeds.
Models
1-step Prediction
(1e-3)
m-step
Imagination Error (1e-3)
Full
Visible
Not Visible
Full
Visible
Not Visible
Traustormer
12200.00
0.6710.00
2.15:10:00
36 1315.0
11.49-3.3
38.07.133
Titans
1.32+0.00
0.69++0.00
2.20++0.05
33.42++4.6
10.60-2.4
35.21+13
mm- TEM
5.10=0.05
4.2320.05
6.5320.05
14.30=0.36
13.2343.00 0505
14.40=0
Hippoforuer
10.00000
0.6710.00
2.09.0..05
9.71
2.72-0.01
10.27.1..
We systematically evaluate Hippoformer using three complementary metrics: error across the entire sequence ("full"), error on visible frames ("visible"), and error on non-visible frames (see Appendix. A.4 for details). Visible frames probe the model's ability to exploit structured memory, whereas non-visible frames require abstraction from historical context. As shown in Tab. [. Hippoformer outperforms both transformer and Titans models slightly but markodly in multi-step imagination. Consistent with these find- ings, Fig. C shows that Hippoformer maintains cohcrent predictions over long horizons in multi-step settings, whereas transformer and Titans models exhibit oscillatory errors around 36 56 steps, appearing to stack over time. Notably, for consistency with the Hippo- former, the standalone mm-TEM is also trained with mb on the 3D task. A large mb inherently biases mm-TEM. toward long-how horizon prediction sentiction-enaanciingg multi-step imagina- tion but reducing single-step accuracy (Tab. [], limitation that is compensated by the Transformer component in Hippoformer. Additional results are provided in Appendix A.g. Overall, these results demonstrate that Hippoformer effectively leverage abstraction and memorization, with its two modules cooperating to achieve robust long-term prediction.
in mno-step-rrddictie
4 DISCUSSIONS
In this work, we introduce mm-TEM and Hippoformer, two hippocampps--nnspredd models for prediction and spatial reasoning. mm-TEM trains more efficiently than standard TEM and spontaneously develops grid-like codes, whose grid scale modulated by the prediction horizon, offering
perspective grid diversity at the implementation level. Addi- tionally, we propose Hippoformer, which integrates Transformers and mm-TEM. A natural division of labor emerges: Transformers primarily capture short-term dependencies, while

---

Published as a conference paper at ICLR 2026
mm-TEM supports long-horizon forecasting through robust grid codes in 2D environments. In 3D environments, Transformers contribute to short-term memorization, whereas mm- TEM focuses long-term abstraction. Together, these complementary roles yield both improved training efficiency and better generalization.
Related works. Our work extends the computational framework of the HC-EC system. Existing models such as CSCG (George et al 20210, Vector-HaShie HaShhhCanraarra et al, 2025). et al. 2021), and TEM-t WEMMt((Witting (Whittington et al,, 2021), are conceptually elegant but face limitations in scaling to modern deep learning architectures. For exam- ple, TEM relies on computationally expensive tensor-product Hebbian memory; TEM-t is constrained by transformer window size and requires complex memory updates; and Vector- HaSh non-differentiable. These limitations hinder their application to complex tasks. In contrast, we propose mm-TFM, which employs hierarchical MLP as a relational memory system. Augmented with auxiliary relational losses, mm-TEM offers a powerful, flexible memory mechanism that integrates seamlessly with modern transformers, enabling its use in more complex environments.
Long-sequence modeling is central challenge in machine learning. Recent architectures such as Mamba((Gu Dao, 2023), Titans (Behrouz [BehroB et al,, 2024), and Gated Delta Net- works((Yang et a 2024) represent important advances through struetural initialization, hierarchical MLP memory, and novelty-based Hebbian rules. However, real-world infor- mation is inherently spatiotemporal, and simply enlarging memory capacity while ignoring its underlying structure is an inefficient strategy. To address this, we introduce Hippo- former, a novel hybrid memory system that combines the precise short-term memory of transformers with the structured long-term memory of mm-TEM. This design enables more efficient organization of memory sequences, making Hippoformer a promising architecture for long-sequence modeling.
It has long been proposed that diverse grid scales may be natural consequence of multi- timescale predictions in the brain. For example, Waniek analytically derived diverse grid et al. codes (Stachenfeld et al,, 2017; Dordek et al,, 2016). In contrast to these works, mm- TEM produces different grid scales emergently through self-supervised learning, achieved simply by adjusting the memory-update frequency, without imposing any prior multiscale place structure. This end-to-end emergence is important because it allows prior theoretical ideas about multiscale grid representations to be tested and extended in more complex and realistic task settings.
scales from multi-scale predictions (Wanick, 2019). Similarly, Stachenfeld et al. and Dordek can arise as basis functions of multiscale place
that diverse fiivers,grid.
Additionally, mm-TEM offers a new perspective on the biophysical mochanisms underlying diverse grid scales in the brain. Unlike.prior approaches that-eerm to reconstruct predefined multiscale place representations (Stachenfeld et al. 2017; Dordek et al., 2016), mm-TEM suggests that medory-update frequency itself may play a critical role by implicitly imple- menting multiscale prediction horizon. Biologically, variation in memory-update frequency may be able to related to known gradients along the ventral ventral-dorsal hippocampal axis suchhs oscillation- oscillttio-frequency gradients (Goyal et al. 2020) receptor-expressin gradients (Strange et al., 2014).
Limitations and Future work. While mm-TEM provides an efficient structured memory system, our current Hippoformer design only illustrates a straightforward parallelization of transformer and mm-TEM. Moreover, the present Hippoformer is limited to single-layer design, without leveraging the model and computation scaling that has been shown to be crucial in large language models([aplan et al. 2020).
Future work should investigate more efficient integration schernes and multi-layer scaling, positioning mm-TEM as scalable fundamental building block for large systems and spatial reasoning tasks. More broadly, mm-TEM's simplicity may enable hicrarchical models of the hippocampus, offering computational handle on how biological dorsal ventral representa- Lional gradients give rise to functional specialization (Strata (Strange et al. 2014 2021).
Maurer & Nadel,
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China (62336007, B. Hong).
ETHICS STATEMENT
This work does not involve sensitive datasots, human subjects, or potentially harmful ap- plications. Therefore, we have not identified any obvious ethical concerns.
REPRODUCIBILITY STATEMENT
The paper and appendix provide detailed descriptions of model architcctures, hyperparam- eters, training procedures, data preprocessing steps, and computational resource configu- rations. We also include ablation studies and additional results to ensure that the main conclusions robust.
THE USE OF LARGE LANGUAGE MODELS (LLMs)
We used large language models (LLMs) for minor text polishing, but not for generating scientific content, experiments, or analysis. In addition, we employed LLMs to refine visual- ization code for clarity and readability; these edits did not affect any experimental results.
REFERENCES
Ali Behrouz, Peilin Zhong, and Vahab Mirrokni. Titans: Learning to memorize at test time. arXie preprint arXi::2501.0066 2024.
Roland G Benoit and Michael C Anderson. Opposing mechanisms support the voluntary forgetting of unwanted memories. Newron, 76(2):450 460, 2012.
Katie C Bittner, Aaron D Milstein, Christine Grienberger, Sandro Romani, and Jeffrey C Magee, Behavioral time scale synaptic plasticity underlies cal place fields. Science, 357 (6355):1033 1036, 2017.
Daniel Bush, Caswell Barry, Daniel Manson, Neil Burgess, Using grid cells for naviga- tion. Neuron, 87:507 520, 2015. URL petps://apt: semanticscholar, wwtp:/ahhhhaaaaoooo/oooooo/o Irg/CorpusID: 7275119
György Buzsáki and Edvard I Moser. Memory, navigation and theta rhythm in the hippocampal-entorhinal system. Nature neuroscience, 16(2):130-138, 2013.
Sarthak Chandra, Sugandha Sharma, Rishidev Chaudhuri, and la Ficte. Episodic and associative memory from spatial scaffolds in the hippocampus. Nature, 638(8051):739 751, 2025.
Geoffrey W Dichl, Olivia J. Hon, Stefan Leutgeb, and Jill Loutgeb. Stability of medial entorhinal cortex representations over time. Hippocampys, 29:284 302, 2018. URL
http:://api. semanticscholarr httpaarrrrrrrrrrrrrr::s212233330
Yedidyah Dordek, Daniel Soudry, Ron Meir, and Dori Derdikman. Extracting grid cell characteristics from place cell inputs using non-negative principal component analysis. Elife, 5:c10094, 2016.
Howard Eichenbaum. Prefrontal hippocampal interactions episodic memory. Nature Reviews Neuroscience, 1899)5547-558, 2017.
Howard Eichenbaum, Andrew P. Yonelinas, and Charan Ranganath. The medial temporal lobe and recognition memory. Annuol review of neuroscience, 30:123-52. 2007. URL
htttps://api semanticscholar.... wtt.ehho.......rooooopppDD111111
11

---

Published as a conference paper at ICLR 2026
Mathias Franzius, Henning Sprekeler, and Laurenz Wiskott. Slowness and sparseness lead to place_ head-direction. and spatial-riew cells. PLoS Comoutational Biology, 3, 2007. URL httpss///aap.smmmaticccccceeeee
Marianne Fyhn, Sturla Molden, Menno P Witter, Edvard Moser, and May- Britt Moser.
Spatial representation in the entorhinal cortex. Science, 305(5688):1255 1228--1264, 2004.
Ruiqi Gao, Jianwen Xie, Xue-Xin Wei, Song-Chun Zhu, and Ying Nian Wu. On path integration of grid cells: group representation and isotropic scaling. Advances in Newral Information Processing Systems, 34:28623 28635, 2021.
Dileep George, Rajeev V Rikhye, Nishad Gothoskar, Swaroop Guntupalli, Antoine Dedieu, and Miguel Gredilla. Clone-structured graph representations enable flexible learn- ing and vicarious evaluation of cognitive maps. Nature communications, 12(1)) 2021.
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-for forward layers are key-value memories. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, pp. 5484-5495, 2021.
Abhinav Goyal, Jonathan Miller, Salman E Qasim, Andrew J Watrous, Honghui Zhang, Joel M Stein, Cory S Inman, Robert E Gross, Jon T Willie, Bradley Lega, et al. Func- tionally distinet high and low theta oscillations in the human hippocampus. Nature communications, 11 (1):2944,, 2020.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint 2rX 00752, 2023.
Eric L. Hargreaves, Geeta Rao, Inah Lee, and James J. Knierim. Major dissociation between medial and lateral entorhinal input to dorsal hippocampus.. Science, 308:1792 1794, 2005. URL https://ppp.emmmmiccccclll//
org/ (CoppusID:24399777,
Michael E. Hasselmo, Clara Bodelón, and B. Wyble. A proposed function for hippocampal theta rhythm: Separate phases of encoding and retrieval enhance reversal of prior learn- ing. Neural Computation, 14:793-817, 2002. URL https: sttps::/app.semmntttte semanticscholar .oras//rlrrrr//////o/
CorpusID:9128504.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint #xXx::000.08866 2020.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. arXiv preprint arXiv:2001.04440 2020.
James J. Knicrim, Joshua P. Neumucbel, and Sachin S. Deshmukh. Functional correlates objects, path integration and local-global reference frames. Philosophical Transactions of the Royal Society B: Biological Sciences, 369, 2014. URL https: / /api..pp... httpp::/apttsemaaatttttt// semanticachollar org/CorpusIDD.....oooooo ID: 1031828.
of the lateral and medial entorhinal cortex:
Yunlong Liu, Shuwen Du, Li Lv, Bo Lei, Wei Shi, Yikai Tang, Lianzhang Wang, and Yi Zhong, Hippocampal activation of racl regulates the forgetting of object recognition memory. Current Biology, 26(17):2351 12611222357--1 2016.
Andrew Maurer and Lynn Nadel. The continuity of context: role for the hippocampus. Trends in cognilive sciences, 25(3):187-19) 199, 2021.
Edvard Moser, May-Britt Moser, and Bruce L McNaughton. Spatial representation in the hippocampal formation: history. Nature neuroscience, 20(11): 1464, 2017.
Marcello Mulas, Nicolai Wanick, and Jörg Conradt. Hebbian plasticity realigns grid cell
activity with external sensory cues in continuous attractor models, Frontiers in Compu- fational Neuroscience, 10, 2016. URL https:/ //api. http:///aappssssaitseeeeeeee semanticscholar. htw.r/iiiicccca..ooooooorr/ppipp 15499125.
12

---

Published as a conference paper at ICLR 2026
Jurgis Pasukonis, Timothy Lillicrap, and Danijar Hafner. Evaluating long-term memory in 3d mazes. arXiv preprint KKt::2210013988. 2022.
Kanaka Rajan, Christopher D. Harvey, and David W. Tank. Recurrent network models of sequence generation and memory. Newron, 90:128 142, 2016. URL attps://api. semanticscholar.
13781578.
Rajkumar Vasudeva Raju, Swaroop Guntupalli, Guangyao Zhou, Carter Wendelken, Miguel Siizaro-Gredile and Dileep George. Space is a latent sequence: A theory of the hippocampus. Science Advances, 10(31):cadm8470) 2024.
Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, et al. Hopfield networks is all you need. arXiv preprint ar Xrxi:2008..222 02211, 2020.
Judith Schomaker and Martijn Meeter. Short-and long-lasting consequences of novelty, deviance and surprise brain and cognition. Neuroscience & Biobehavioral Reviews, 55: 268 279, 2015.
Alyssa Sinclair, Grace M Manalili, Iva K Brunec, R Alison Adcock, and Morgan D Barense. Prediction errors disrupt hippocampal representations and update episodic mem- ories. Proceedings of the National Academy of Sciences,
7625118, 2021.
Kimberly L. Stachenfeld, Matthow M. Botvinick, and Samuel J. Gershman. The hip- pocampus predictive map. Nature Neuroscience, 20:1643-1653, 2017. URL https: //api. semanticscholar org//CorpusD:::176865777
Bryan A Strange, Menno P Witter, Fd Lein, and Edvard Moser. Functional organization of the hippocampal longitudinal axis. Nature reviews neuroscience, 15(10):655-66) 669, 2014.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasa Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Nicolai Wanick. Transition scale-spacos; A computational theory for the discretized en- torhinal cortex. bioRriv, 2019. URL attps://apt semanticscholar Axg/CorpusID:
92223488.
James CR Whittington, Timothy H Muller, Shirley Mark, Guifen Chen, Caswell Barry, Neil Burgess, and Timothy EJ Behrens. The tolman-cichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation. Cell, 183(:):
1249 1263, 2020.
James CR Whittington, Joseph Warren, and Timothy EJ Behrens. Relating transform- ers to models and neural representations of the hippocampal formation. orXiv preprint 2021.
orxii:2212.00441
James CR Whittington, David McCaffary, Jacob JW Bakermans, and Timothy EJ Behrens. How build a cognitive map. Nature neuroscience, 25 (51)) 1272, 2022.
Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10632-10643, 2025.
Songlin Yang, Jan Kautz, and Ali Hatamizadeh. Gated delta networks: Improving mamba2 with delta rule. arXiv preprint arXiv:2412.06464 2024.
Bohan Zhuang, Jing Liu, Zizheng Pan, Haoyu He, Yuetian Weng, and Chunhua Shen. A survey on efficient training of transformers. arXiv preprint ar Xrx 2023.
Xiaolong Zou, Xingxing Cao, Xiaojiao Yang, and Bo Hong. Leveraging attractor dynamics in spatial navigation for better language parsing. In Forty-fust International Conference on Machine Learning, 2024.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 DETAILS OF 2D AND 3D PREDICTION TASKS
2D grid prediction task. In the 2D grid environment, the agent can take four discrete actions up, left, right, down), which are encoded as one-hot vectors. Observations are sam- pled from uniform distribution over 64 sensory objects, and in each trial, the observations are drawn independently from this set. Since the environment is discrete and sensory ob- servations are uncorrelated, generalization in this setting primarily depends structural memory-based inference rather than feature abstraction. Consequently, predicting observa- tions at unvisited locations is not meaningful. For all 2D grid prediction lasks, the models are trained on grid environments of sizes 11 x 11.
3D environment, prediction task. The 3D environment built on MemoryMaze31 (Pa- sukonis et al. 2022), with the layout simplified to an empty 2D plane, Environment textures are randomly sampled in each trial. Unlike the allocentric obsorvations in the 2D grid en- vironment, the observations here are egocentric view images, which are inherently more complex. The action space consists of discrete set of actions, including moving forward, lurning left, lurning right, slaying still, and combinations of moving forward with turning. However, the resulting movements are continuous, subject to noise and acceleration. In this environment, unvisited observations can be inferred from nearby spatial information, making feature abstraction a critical factor in the 3D environment prediction tasks.
A.2 DETAILS OF MM- TEM ARCHITECTURE
The mm-TEM architecture primarily consists of a path integration network and relational memory network. Unlike the relational memory TEM and TEM-1, which incorporate hand-designed relational priors TEM uses tensor- tensor-product-A product--bseed Hebbian mechanism, and TEM-t employs key-value pairs - our relational memory does not rely on any mamually injected priors. Instead, we self-supervise the network with a simple auxiliary relational loss, allowing the memory system to learn relational structure autonomously. This approach makes the memory system more flexible and scalable compared to those in TEM and TEM-t.
mm-TEM receives both action vectors and sensory feature vectors as inputs. In the 2D grid environment, sensory features are extracted using 2-layer MLP encoder, whereas in the 3D enviromment they are extracted using a 4-layer convolutional neural network and 1-layer MLP.
For the relational memory network, we follow the standard configuration used in Titans memory. Specifically, we employ 3-layer MLP the core of the meta-MLP network. The relational memory module processes the input using two attention heads, cach with 64 dimensions. The meta-menory weights are updated using a 2-order momentum rule. These settings are kept. identical across both 2D and 3D enviromments, and are also matched to the Titans baseline for fair comparison.
The path integration network processes the action sequences and generates structural codes. we use 64 grid cells for 2D environments and 128 grid cells for 3D environments. Due to noise-driven error accumulation, the path integration network requires correction signals from scnsory observations to recalibrate the structural codes, analogous to the HC-EC system in biological brain. Further error correction details are provided below.
Error correction in the path integration system. In the HC-EC system, the HC pro- vides relational or spatial feedback likely in the form of conjunctive codes that bind spatial sepsory information which can help recalibrate MEC structural representations (Diehl et al., 2018 Mulas et all. 2016)- To emulate this mechanism, from the relational memory network. Specifically, given an action az, the path integration network generates a structural code Igen,t. Together with the sensory code It, a memory query m, [9gen.t; is formed and sent to the relational memory network, which retrieves a structural code Igen,t and a content code é. The retrieved code is then fed back into the path integration network and combined with Igen,t as follows:
introduce feedback loop
Jinf.t = Igen,t + alggen,1 9gen,)) Sdecta (ggen,ts Igen, sg(1 -bal9))
14

---

Published as a conference paper at ICLR 2026
where (-) is two-layer MLP that predict the variance of the integrated structural code, respectively. sg()) means stop gradients. The scalar a controls the integration ratio, which slowly increases in the first 2000 gradient steps. The updated structural code, fin then becomes the new state of the path integration network and is stored in the relational memory alongside the sensory code, St.-
In mm-TEM, the mcta-MLP memory is updated once every mb time steps. At each time t, the model aggregates the recent pairs (ggen,t:8) 8x) collected since the last update. Given an input sequence :..., (Agen,t: 8t), (Igen,t+1, 8641),..., (9gen,t+mmb 91.mb)).... if the previous memory update occurs at time t, the next one occurs at t-mb.
At time t + mb, the path integration network is first corrected using (9gen,r+mb. St+mb). This pair is sent tural code Igen,t-mb- The retrieved code is then combined with the previous struc- tural estimate Igen,t to produce an updated, corrected structural code ginf.ttmb. Af- ter this correction step, the mm-MLP memory is then updated, using the batch St) (ggen,tmbb1: 8140:1 1); [finf.t+mhy 5t-mb)]
the relational memory (mm-MLP), which retrieves a struc-
Importantly, throughout the interval [ -> 1-mmb, the network has access to these "future" (g, pairs. All predictions must rely solely on information available at and before time Cousequently, the model must implicitly predict 1 to mb steps ahead, and larger mb increases the effective prediction horizon.
Objective Functions and Training. The overall prediction and update process in mm- TEM is summarized as follows:
1. Given the action as the path integration network computes the structural code Igen,t-
2. With the input m = eg Oj, the relational memory network retrieves the sensory code, which is further decoded into the observation $ ggen.t. This yields the generative prediction loss:
ggen || - 8c sel
3. Using the joint structural-sensnt code my = 1g sal, the relational memory network predicts the corrected structural code Igen,.- Combining the generative and reetivek-retrieeed structural codes (Agen,as Igen.t), mm-TEM produces the cor- rected structural code f.n using the above error correction process. This gives the consistency loss:
Leon I1ggen,t - finf.ell..
4. The corrected structural code ginf.t is then passed again to the relational memory network with m, [ginf1; The network predicts the sensory code finf which is decoded into the observation Sinft. This defines the inference prediction loss:
Lint = [|8inftt -
- 12
5. Then, the integrated structural code Dinf. and the sensory code 24 are stored in the relational memory network using the online gradient- basod update rulc.
The main objective of mm-TEM to predict the next observation given past sensory inputs and actions. It is trained via self-supe supervied manner. The total loss combines the predictive loss, consistency loss and relational loss:
(7) where "reel Tgens Yoon and Tinf are scale factors of different losses. We train the entire mm- TEM end-to-end on the prediction task by minimizing L. Optimization uses the Adam optimizer with learning rate of 0.001. We apply StepLR schedule with step size 500 and decay factor = 0.9. The warm-up phase lasts for 5,000 steps, and training runs for up to 20,000 steps.
L TretCrel + Tyen Cgen + "coon Coon + fiinf
When combined with Transformer, the model architecture of Hippoformer is shown in more detaied version of Fig. 2A in Fig. A.1.
15

---

Published as a conference paper at ICLR 2026
Visual Decoder
1
I g:
Transformer
mm-TEM
1/
91
Visual Encoder
at
8
a+
Figure A.1: Model architecture Hippoformer, which integrates mm- amn-TEM with a Trans- former in parallel. Related to Fig.
A.3 CONTROL MODEL ARCHITECTURE
We compare mm- TmnTEMM and Hippoformer with two control architectures: Transformers and Titans. The model configurations are summarized as follows.
For the Transformer baseline, we employ three-layer Transformer with two different tem- poral windows: 64 steps and 128 steps. The number of attention heads is set to N 8 and the hidden dimension to NT 128. Although the Transformer with a 64-step window has limited temporal context, its hierarchical three-layer structure still allows capture dependencies across the entire sequence.
For Titans, we adopt the MAC architocture, which incorporates an external memory com- ponent as contextual information. The meta-MLP paramcters keep identical with our re- lational memory modules. Its Transformer component uses the same parameterization as the standard 64 windows Transformer. Both the Transformer and Titans share the same imput layer, which embeds actions and sensory inputs into Nact = 192, N Neense = 192 in 2D environments, and Nact 128, Nernse = 4096 in 3D environments.
In Hippoformer, the short-term memory module is implemented as a one-layer Transformer with a window size of 32. In both model and Hippoformer), the input layer embeds grid representations into Ngria = 64, Nsense 64 in 2D environments, and Ngrid = 128, Nzense 4096 in 3D environments.
A.4 MODEL EVALUATION
In 2D environments, racy for evaluation, following TEM and TEM-t.
one-step prodiction accuracy and multi-step imagination accu-
For one-step prediction, given context sequence of length Ti with state-action in- puts [1), (82, o2),..., (sr, ar)),, the model generates next-state predictions $ [$2, $3...., 82; A predicted state counted as valid only when its corresponding ground- truth visited before t.
For multi-step imagination, after receiving a context of length Ti, the network performs Tz-step imagination. During imagination, action sequence ( ar, +7,] is provided,
16

---

Published as a conference paper at ICLR 2026
and the network produces predicted states $ [81+1, -..-...... &n +r;]. A predicted state counted as valid only when its corresponding ground-truth was visited during the context period. For both one-step and multi-step, all accuracies are computed as the fraction of correct predictions among these valid predictions. Since unvisited states are inherently un- predictable in the 2D disctrete grid state setting, "generalization" is over unseen transitions (edges) rather than unseen states.
In 3D environments, the error metric defined the pixel-wise mean squared error normal- ized over all pixels. The definitions of "one-step" and "multi-step" imagination are similar to those in 2D. Our 3D environment has a size of 9 x For evaluation, we roughly discretize the environment into 9 x 9 spatial grid. Since the network receives egocentric observa- tions, we further discretize the orientation into 12 bins, resulting x 9 12 = 972 distinct states. If an observation falls into a spatial grid and orientation bin that has been encoun- tered previously, we classify as vistble; otherwise, is classified as non-wisible. Note that this discretization is used only during evaluation.
A.5 TRAINING AND PARAMETER DETAILS
All models are implemented in PyTorch and trained on NVIDIA A100 GPUs. The training and parameter configurations corresponding to cach figure are summarized below.
For the results Fig. g. we report the sequence length, learning rate, batch size, memory update frequency (mb), and testing environment size. Detailed settings are provided in Table A.1
Table A.1: Training and parameter details for different experimental results.
Figure
Soquence Length
Learning Rate
Batch Size
mb
Env. Size
Fig.
256
le-3
16
8-11
Fig.
128
le-3
16
8-11
Fig.
256
1e-3
16
1/4/8
8-11
Fig. 256
1e-3
16
8-11
Fig. 128
64
5e-4
16
8
9
Fig-
1e-3
16
8-11
Fig.
128
le-3
16
1/4/8
8-11
A.6
EFFECTS OF HYPERPARAMETER mb MM-TEM
A
B
1-step Accuracy
Multi-step Accuracy
1.0
1.0
0.8
Accuracy
0.8
Accuracy
0.6
0.6
0.4
mb=1
mb=4
0.2
mb=8
27
2
2.1
2
29
211
Context Length (log scale)
Imagination Length (log scale)
Figure A.2: Effects of Memory Update Frequency (mb) on Generalization Capacity of mm- TEM. We investigate how the memory update frequency parameter (mb) affects the gener- alization capacity of mm- am-TEM. All models are trained with sequence length of 128. The generalization performance is evaluated under both one-step prediction (A) and multi-step prediction settings (B).
17

---

Published as a conference paper at ICLR 2026
In mm-TEM, the meta-MLP memory is updated every mb steps; thus, the hyperparameter mb controls the memory update frequency. With a larger mb, the network must rely on information from mà steps ago to predict the next observation, effectively increasing the prediction horizon. Therefore, mb also balance short- and long-range predcitions. As shown in Fig. A.2. under one-step prediction, smaller mà values emphasize short-range predictions but perform worse in multi-step prediction.
Ideally, models should perform well in both short- and long-range prediction and general- ization. This is where Hippoformer demonstrates its advantage: by combining short-term memory from the Transformer with limited window size and structured long-term memory from mm- TEM, Hippoformer achieves good performance across both short- and
predictions, as shown in Fig. 5 in the main text. Furthermore, larger mb also improves training efficiency for mm-TEM. Taken together, this simple combination leverages the complementary strengths of both modules while also achieving high efficiency.
A.7 EFFECTS OF DIFFERENT POSITION ENCODING METHODS ON THE MODEL CAPACITY.
To determine whether mm-TEM and Hippoformer's advantage stems from hippocampus- like computational mechanism, we performed additional experiments and trained Trans- former/ /Titan baselines with three PE variants: (1) Sinusoidal PE, which is static and abso- lute; (2) Rotary PE, which is a relative position embedding; (3) Dynamic PE, implemented by a path integation network identical to mm-TEM's PI module, serving as a powerful recurrent positional embedding. We evaluated both one-step and multi-step prediction and also tested transfer in the circular-gry -rid environment (same as Fig.
A
1-step Accuracy
B 1.0
Multi-step Mult-steepAccuracy
1.0
0.8
Hopofomer
0.8
Accuracy
0.6
0.6
0.4
0.4
0.2
0.2
27 2 2
0.0
2
2
2
Context Length (log scale)
Imagination Length (log scale)
Figure A.3: Comparison of Model Generalization Using Diverse Positional Encodings. Ti- tans and Transformers are evaluated with diverse PEs, inchuding: absolute sinusoidal (c.g.,
Transformer sin), rotary (e.g., Transformer _ott, Dynamic PE (e.g., Transformer dyn),
and the learned axial baseline PE used in the main text (e.g.. Transformer learn) All
models are trained in 2D grid prediction tasks (training length 128) that in Fig. (A)
1-step prediction accuracy vs context length. (B) Multi-step imagination accuracy vs imag- ination length. All results are averaged over seeds.
Table A.2: Model Parameter Comparison for 2D Grid Tasks.
Model
Parameters (M)
Titans Trans
30.62
29.63
mm-TEM
9.11
Hippoformer
10.06
As shown in Fig. A.3 and Fig. A.4. our key findings are as follows. (1) Rotary PE and dynamic PE substantially improve one-step prediction, in some cases reaching performance comparable to mm- mm-TEM with mb = 1. (2) However, none these PE variants allow Trans- formers or Titans to approach mm-TEM's performance on multi-step prediction circular- grid transfer, both of which require robust long-range relational inference. (3) Hippoformer (Transformer mm-TEM with mb 8) consistently outperforms all PF baselines by
18

---

Published as a conference paper at ICLR 2026
A
B
0.935
1.0
0.00d
0.944
1,000
0.939
0.014
Tuaaloma
1994
0.826
Than FET
0,8
0.711
Accuracy
0.612
0-627
19.833
Accuracy
0,57
0.6
0.502
0.525
0.431
0.33
0.294
0.069
CW
CCW
7x7
11x11
13x13
15x15
Walk Direction
Map Size
Figure A.4: Comparison of Model Generalization Circular-grid Environments. All model architectures and task settings are the same as that in Fig. and Fig. (A) Clockwise vs counterclockwise performance across different architectures. (B) Effect of environment size, ranging from 7x7 to 15x15. All results are averaged over 3 seeds.
wide margin, demonstrating the complementary strengths of short-range memory (Trans- former) and long-range, structure-aware memory (mm-TEM). (4) Importantly, 1m-TTEM and Hippoformer achieve those results with fewer paramcters than the Transformer and Titan basclines (scc Tab. A.2 indicating that the gains are not duc to model size, but architecture.
why PE alone cannot explain the gains. Strong positional encodings (rotary or dynamic Pl-based improve short-range prediction, but they do not endow baseline Transformers and Titans with hippocampus-lik computational mechanisms, such as feedback- based error correction relational memory. Simply inserting recurrent path integration module as positional embedding does not yield hippocampus-like abstract structure lcarning or spatial rule inference, which are essential for multi-step relational reasoning and long-range prediction.
A.8 ADDITIONAL ABLATION AND CONTROL STUDIES
A
B
1-step Accuracy
Multi-step Accuracy
1.0
0.8
Accuracy
0.9
Accuracy
0.6
0.4
0.8
Heppffomer
Hipooormer Tranformer
0.7
Thansformer
2'
2
2
0.0
27
2
2
Context Length (log scale)
Imagination Length (log scale)
Figure A.5: Hippoformer using Truncated-BPIT. Transformer using Full-BPTT. Hippo- former has the same architecture as that Fig. 5 Translomer uses the same architecture as that in Fig. 3 All models are trained in 2D grid prediction tasks (training length 512).
BPTT in Hippoformer is truncated at 256 steps, and Full BPTT adopted in Transformer. (A) 1-step prediction accuracy context length. (B) Multi-step imagination accuracy
imagination length. All results are averaged over 3 seeds.
As shown in Fig. A.5 to examine whether full BPTT is necessary, we conducted a new experiment with 512-length sequences. In the task, Hippoformer uses BPTT truncated at 256 steps. Transformer uses full BPTT over all 512 steps. Despite the truncated training, Hippoformer still outperforms the Transformer, demonstrating that full BPTT is not
19

---

Published as a conference paper at ICLR 2026
quired for Hippoformer to capture long-range dependencies. In fact, long-range dependency does not rely solely on long unrolled recurrence. It also relies on the architectural prior to learn abstract structural codes.
A Training Convergenoe (m-stop aoc)
Training Convergenda (m-4top acc)
C
Training Convergance (1-dtop aca)
1.0-
1.0
1.0
0.0
Accunncy
0.0
20.6
0.4
0.2
0.0
0.0
TM
5000 10000 15000 Groslent Stop
20000
20000
60000 60000
30000 15000 20000 Gradient Stup
Time 16)
Figure A.6: Training Efficiency and Multi-step Prediction Performance of mm-TEM ver- sus TEM-t Baselines. Task settings and evaluations are the same as that in Fig. 2 (A) Multi-step prediction accuracy during training (as a function of gradient steps). mm-TEM converges substantially faster and achieves markedly higher accuracy than both TEM and TEM-L. (B) Multi-step prediction accuracy plotted against. wall-clock training time, showing mm-TEM's superior time-efficiency. (C) Both TEM and TEM-t achieve high one-step ahead prediction accuracy during training. One-step prediction was evaluated using a contexxt se= quence length of 128, consistent with the original TEM and TEM-t studies (Whittington et al. 2021; 2020). Notably, TEM-t converges faster than TEM. All results are averaged over 3 seeds.
A
mm-TEM grid score distrbutioon
B
Proportion of Calls (density)
1.50
mb=1
mb=1
1.25
control
1.00
0.75
control
0.50
0.25
contral
0.00
-0.8-0.5
1,7 1.5 Gridness (binned)
Tops neurons
Other typioal nourons
Test-time Effects of (mb). All mm- TEM models are trained with mb = 1. In Control 1, the model is tested with mb = for both sensory error correction and mm-MLP memory updates. In Control 2, the model is tested with mà = 8 only for mm-MLP memory updates, while sensory error correction remains at mb 1. (A) Histogram of grid-cell gridness scores. (B) Autocorrelation maps of grid representations under the different conditions.
Figure A.7: Ablations Separating Training-tino
According to the above detail feedback mechanism, increasing mb reduces both the frequency of memory updates and the frequency of coror-correction feedback. To disentangle these two factors, we performed the following control studies: (1) in Control 1, mm-TEM is trained with mà = 1, but tested with mb 8 for both error correction and mm-MLP memory updates; (2) in Control 2, mm-TEM is trained with mb 1, but tested with mb for mn-MLP memory updates while keeping error correction at mb scale remains identical the mb 1 model, as illustrated in Fig. A.7. This outcome expected. In mm-TEM, the grid scales are determined solely by how the path-integration network maps actions into grid representations and by the specific grid sensory pairings stored in relational memory. The value of mb does not influence either the mapping of path-integration network or the specific grid-sensory pairings, and therefore does not alter the resulting grid scales.
In both cases, grid
Spearman correlation between grid score and model accuracy is shown in Note the non- linear correlation significance more significant than linear.
20

---

Published as a conference paper at ICLR 2026
Grid-Accuracy Spearman corre ation
B
Residuals in rank space
0.3
10
p=0.7633
p=1e-6
Rank (Accuracy)
88
06
Residual (Rank
0.4
95% CI
a.1
High grid-High acc
Low gird-High acc grid-Low ecc
6.0
-03
06 6. 16
Rank (Grid Score)
Rank (Grid Score)
Figure A.8: Spearman correlation between grid score and model accuracy. (A) In the rank space, multi-step prediction accuracy (imagination length 512) positively correlates with grid score in mm-TEM path integration networks (r = 0.7833, p 1e-6), indicating that stronger grid-like regularity supports better generalization. Related to Fig, AA. (B) The residuals.
A
1-step Accuracy
B
Multi-step Accuracy
1.0
1.0l
mm-TEM
Cosa
0.8
Accuracy
0.8
Accuracy
0.6
0.4
0.2
0.4
2'
20
2
2'
2a
2
Context Length (log scale)
Imagination Length (log scale)
Figure A.9: Control Study On the Auxiliary Relational Loss Luze- "With x2x" denotes the variant that adds Cr2e the auxiliary relational loss Crel in mm mm-TEM, whereas "mm- TEM" refers to the original model using Crei without Cæ2a- (A) and (B) report 1-step and multi-step prediction accuracy, respectively, following the same training and evaluation settings as Fig.
A.9 OTHER SUPPLEMENTARY RESULTS
Other supplementary results corresponding to the Results in the main text are shown as follows.
21

---

Published as a conference paper at ICLR 2026
Multi-step Accuracy
1.0
0.8
Accuracy
0.6
0.2
mm-TEM
correction
0.0
27
2
2
Imagination Length (log scale)
Figure A.10: Both mm-TEM variants with and without error correction are trained on sequences of length 256, with mb = 8. The models are evaluated using multi-step prediction, and all reported results are averaged over three random seeds.
1.0
0.9350.950
0.949
70.887
0.915
Transformer
mm-TEM
Titans
0.8
0.632
Accuracy
0.6
0.4
0.2
0.0
2
Prediction #N round
Figure A.11: Generalization Capacity of mm-TER TEM in Long-Conters Conditions. we evalu- ates the generalization capacity of mm- mmm-TEM on the n-round imagination task in a circular grid environment, the same that in Fig. 3D. As the agent explores the environment in clockwise manner and accumulates more rounds of sensory experience, mm-TEM achieves robust performance, maintaining around 90% accuracy in multi-step imagination. In con- trast, both Transformers and Titans exhibit a sharp performance drop as the context input length increases, highlighting the superior generalization ability of mm-TTEM under long- context conditions.
22

---

Published as a conference paper at ICLR 2026
Figure A21: Grid patterns from mm- with parameter mb=1. The upper 4 rows show the ratemaps for each neuron, arranged by grid scores. The lower 4 rows show the auto correlation of ratemaps. Related to Fig, 2C
MISIII
CLISE
AT
CIN
IRGZY
PARI
Figure A.13: Grid patterns from mm-TEM with parameter mb=4. Related to Fig. C.
23

---

Published as a conference paper at ICLR 2026
SUMAN
Figure A.14: Grid patterns from mm- mm-TEM TEM with parameter mb=8. Related to Fig. C.
APIY
YOUR
Figure A.15: Grid patterns in representative mm-TEM model whose gridness and accuracy are high. Related to Fig. B.
24

---

Published as a conference paper at ICLR 2026
NEM
SUR
RIGH
AEXII
AXI
VICAN
MORTN
TARTN
JAPIN
Figure A.16: Grid patterns in reprosentative mm-TEM model whose gridness is low and accuracy is high. Related to Fig. B.
COJI
31
SIMIH
JIRIN
Figure A.17: Grid patterns in representative mm-TEM model whose gridness and accuracy low. Related to Fig. A.
25

---

Published as a conference paper at ICLR 2026
Hodeae
Figure A.18: Visualization of example trajectories of one-step prediction in two 3D environ- ments from different models, with snapshots shown every 5 steps. Related Fig. Gc
Ta Mnder Wedi
Howhner
Teskee
Terbow
Figure A.19: Visualization of example trajectories of multi-step prediction in two 3D envi- ronments from different models, with snapshots shown every steps. Related to Fig, Cc
26


Reviews:
[{"review_id": "XeXuCv81It", "reviewer": "Reviewer_Be6v", "summary": "This paper addresses the limitations of prior hippocampal-inspired models (TEM's computational inefficiency) and modern architectures (Titans' lack of inherent spatial memory). By synthesizing these approaches, the authors propose **mm-TEM** (meta-MLP TEM), demonstrating better training efficiency and revealing a novel link between the memory update frequency and the emergence of biologically meaningful grid-like representations. Furthermore, they introduce **Hippoformer** (mm-TEM + Transformer), a hybrid architecture that effectively integrates structured long-term spatial memory with precise short-term working memory, achieving robust generalization across demanding 2D and 3D prediction tasks.", "questions": "See weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "*   **Clarity and Organization:** The paper is well-structured with clear conceptual figures (e.g., Figure 1), making the overall architecture and training rationale highly intuitive.\n*   **Compelling Rationale:** The core rationale for the proposed method—integrating the computational efficiency of the meta-MLP memory (inspired by Titans) into the theoretically grounded TEM framework—is valid for overcoming scalability issues while retaining biological plausibility.\n*   **Strong Experimental Validation:** The systematic evaluation, including ablations and generalization tests across long context, multi-step imagination, and distribution shifts (circular-grid), robustly supports the claim that mm-TEM captures underlying spatial structure more faithfully than baseline models.", "weaknesses": "**Literature Review Suggestion (Line 40):** The Introduction's discussion of the Transformer's associative memory perspective would be strengthened by citing recent, relevant works that formalize the Transformer's components (e.g., FFNs) as explicit memory systems, such as:\n- Geva, Mor, et al. (2020) on FFNs as key-value memories.\n- Ramsauer, Hubert, et al. (2020) on Hopfield networks' relation to attention.\n\n**Relational Loss Notation and Rationale:**\n- **Notation Clarity:** The auxiliary relational losses (Lines 184-190) require precise clarification. Please map the main text's definitions\\ \n$L_1$ and $L_2$ to the figure notations of $L_{x2g}$, $L_{g2x}$, $L_{g2g}$ and clarify the missing one.\n\n- **Missing Term Rationale:** The authors can define four potential combinations ($g \\to x$, $x \\to g$, $g \\to g$, $x \\to x$). But figure/main text uses 2/3 losses. What is the underlying reason for excluding the other possible terms from the relational losses?\n\n2.  **Path Integration Network Ablation and Role:**\n    *   The core TEM theory mandates a Path Integration (PI) network for structural code generation. Since the authors emphasize the **novel meta-MLP relational memory** as the main source of generalization, could the authors include an **ablation study on the PI network component itself** (e.g., replacing it with a simpler, non-integrated recurrent mechanism or removing the error correction loop)? This would help quantitatively disentangle the contribution of the *novel memory* from the *PI component*.\n    *   Given the PI network's role in generating the structural code ($g_t$) from actions, could this component be viewed as an advanced form of **learned positional encoding**?\n\n3.  **Grid Pattern Neuron Selection:**\n    *   For the analysis of emergent grid-like representations (Figure 2C, Figure 4B), the authors show visualizations of the high-gridness neurons. Could the authors explicitly state **from which module** (the Path Integration Network or the Relational Memory Network) the \"Top-5/Top-3\" high-gridness neurons were selected? Clarification on the specific origin of these analyzed neurons is crucial for interpreting the results.\n\n4.  **Table 1 Completeness (3D Task):**\n    *   Table 1 presents results for Transformer, Titans, and Hippoformer in the 3D environment. Given that **mm-TEM** is the core component providing the long-term generalization in Hippoformer, why are the performance results for the **mm-TEM** model alone **omitted**? (Figure 5 shows on par imagination capability between mm-TEM and hippoformer) Including mm-TEM's 3D performance would provide a necessary direct measure of the Transformer's specific contribution (abstraction) to the final Hippoformer architecture in this complex domain, thereby strengthening the claim of synergy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K8XLMErwHd", "reviewer": "Reviewer_Ny5p", "summary": "This paper introduces Hippoformer, a hybrid architecture combining a novel hippocampus-inspired memory module, mm-TEM, with a Transformer. The work aims to address the lack of inherent spatial priors in standard Transformers. The central contribution lies in mm-TEM, a scalable variant of the Tolman–Eichenbaum Machine that employs a meta-MLP for relational memory, closely resembling the long-term memory module used in Titan. The authors conduct experiments in 2D and 3D environments, reporting that Hippoformer achieves superior generalization on long-horizon spatial prediction tasks compared to Transformer and Titan baselines.\n\nWhile the paper is well-written and tackles an important problem, I have major concerns regarding the experimental methodology and the clarity of the architectural contributions. The central claims about the superiority of the proposed relational memory are not sufficiently supported because the baseline comparisons appear to be confounded by critical differences in how positional information is handled.", "questions": "**1. Clarifications on Baseline Models (Crucial for Rebuttal):**\nThis is the most important area. A clear response here could significantly change my assessment.\n\n*   **Question 1a (Positional Encoding):** How was positional/structural information provided to the Transformer and Titan baseline models? Specifically, did they take as input only `[s_t, a_t]`, or did they also receive the structural code `g_t` from the Path Integration Network, similar to mm-TEM?\n*   **Question 1b (Type of PE):** If the baselines did *not* use the Path Integration Network, what form of positional encoding was used (e.g., sinusoidal, learned, rotary, or none)?\n\n**2. Architectural and Contribution Framing:**\n\n*   **Question 2 (Hippoformer vs. Titans):** Could you please provide a more detailed, side-by-side comparison of the Hippoformer and Titan architectures? What are the key differences in their memory update rules, the interaction between the MLP-based memory and the self-attention component, and the flow of information? A diagram or table would be very helpful.\n\n**3. Implementation Details for Reproducibility and Analysis:**\n\n*   **Question 3 (Training Recurrent Module):** How is the gradient calculated for the recurrent mm-TEM module? Is backpropagation through time (BPTT) applied over the full sequence length (e.g., 256 steps in Fig. 2), or is it truncated to a smaller window? What are the implications of this choice for computational complexity and memory usage during training?\n\n\n**Suggestions**\n* If the Fig. 3A experiment corresponds to the length generalization task and does not use the Path Integrator Network for g_t or employ g_t as a positional encoding, the authors should evaluate multiple positional encoding methods and compare their results to those of mm-TEM.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* **Strong Motivation**: The paper is motivated by a clear and significant limitation of current generative models—their lack of structured spatial memory. The inspiration drawn from the hippocampal-entorhinal system provides a principled foundation for the architectural design.\n* **Interesting Component Design**: The mm-TEM module, with its meta-MLP and auxiliary relational losses, is an interesting and efficient take on prior hippocampus models. The analysis showing the emergence of grid-like representations is a valuable piece of evidence supporting the design.", "weaknesses": "The paper's central claims hinge on the superior performance of mm-TEM and Hippoformer over strong baselines, particularly in length generalization. However, the experimental setup is insufficiently described and may contain confounding variables that invalidate these conclusions.\n\n1.  **Lack of Clarity on Positional Encoding in Baselines:** The most critical issue is the ambiguity surrounding the positional encoding (PE) used for the Transformer and Titan baselines. The mm-TEM model relies on a recurrently updated structural code, `g_t`, from a Path Integration Network, which effectively serves as a powerful, dynamic, and task-specific form of positional encoding. The paper does not specify whether the baselines have access to this same structural code.\n    *   If the baselines use standard, fixed positional encodings (e.g., sinusoidal) or no PE at all, the comparison is fundamentally flawed. The performance gains of Hippoformer could stem entirely from its superior positional information, not its relational memory. As shown by Kazemnejad et al. (NeurIPS 2023, \"The Impact of Positional Encoding on Length Generalization in Transformers\"), the choice of PE is a dominant factor in a Transformer's ability to generalize to longer sequences. To isolate the contribution of the relational memory, the baselines must be equipped with a similarly powerful and dynamic PE.\n    *   This lack of clarity makes it impossible to attribute the performance gains to the claimed source (the relational memory) versus a known, powerful factor (the positional encoding scheme).\n\n2.  **Unclear Architectural Novelty Compared to Titans:** The paper presents Hippoformer as a combination of a Transformer and the mm-TEM module (which is centered around a meta-MLP memory). The Titan architecture is also described as a model leveraging fast MLP weights. From the descriptions provided, the high-level architectural blueprint of Hippoformer appears very similar to that of Titans. The paper needs to explicitly detail the architectural and mechanistic differences. Is the primary novelty of Hippoformer simply the introduction of the auxiliary relational loss as a form of inductive bias for the meta-MLP? If so, the contribution should be framed more narrowly as a novel training objective for existing hybrid architectures on spatial tasks, rather than a fundamentally new architecture.\n\n3.  **Ambiguity in the Training of the Recurrent Module:** The mm-TEM module contains a recurrent update for the structural code `g_t`. This introduces dependencies across the entire sequence. The paper lacks crucial details about how this recurrence is handled during training. Is backpropagation through time (BPTT) performed over the full sequence length? Or is it truncated? This detail has significant implications for the model's computational cost, memory requirements, and its practical ability to capture the long-range dependencies it is being credited for.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dWXLaCynjH", "reviewer": "Reviewer_8yz6", "summary": "This work leverages recent formulations of memory from the Titan framework to improve models of the HPC-EC system. This in turn 1) improves the memory efficiency of the HPC-EC models over long contexts, 2) yields insights into grid scaling, 3) leads to an architecture that performs better than standard Titan and Transformer architectures. The new memory structure is an MLP which is trained in-context to map from keys to values, in contrast to older models based on Hebbian and more recently softmax attention.", "questions": "Typos:\n- 188, 194, 256, 735, 736\n\nDo we have a curve for fig2B tem-t?\n\nCan you clarify in main text that the transformer component of Hippoformer does not receive the path-integration input\n\nSome questions about mb:\n- What is the intuition for larger mb increasing the learnt grid size? A fun result that I don't fully understand! \n- Does mb = 1 really perform that poorly - especially since there is no noise in the observations? If so then it would be nice to see this somewhere (since this is essentially the basis of the argument for including transformer in Hippoformer)\n- Does smaller mb lead to better performance (assuming observations aren't noisy)? - If I understand correctly, a small mb ~ 1 is more similar to a transformer; would a network endowed with multiple memory MLPs each with different mb yield a similar performance to Hippoformer & also get grids at different scales?\n- Linked to question above, could mb relate to oscillation frequency in HPC? I believe there is a dorsal-ventral gradient of oscillation frequencies in hippocampus (matches nicely with the gradient of grid scales). Also wonder if there are interesting findings relating to mb & the discretisation of grid scales that is observed.\n\nAre there biological analogies to the hippoformer? \n\nI've always been curious to see what the path-integrator module adds when applied to tasks that are less obviously cognitive map-like. Have you tried this model on text-based tasks?\n\nAre there relations between the Titans meta-MLP and working memory (in contrast to transformer softmax which seems more episodic memory-like)? If so, maybe some kind of systems consolidation inspired ideas might apply here i.e. selectively exporting things out of hippocampal memory into neocortical memory\n\nLine 453 - should transformers and mm-TEM be the other way round?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This work is a significant and novel combination of recent ideas in neuroscience and machine learning. The improvement of tem-t using memory formulations from the recent Titans work is innovative and yields a fun result relating to grid scale. A hybrid architecture is proposed that blends the strengths of the titans-inspired model and transformer; however I would prefer it if the differential contributions of 1) the path-integration input (an old idea) and 2) the meta-MLP relational memory (new idea) were stated more clearly when comparing between models. The architecture has genuine promise to improve upon existing sequence models and I would be fascinated to see it deployed on language problems. \n\nThe figures are all clear and the text is very well written.", "weaknesses": "Barely a weakness & perhaps more to do with framing, but my understanding is that both Transformer and Titan control models you implemented do not have a recurrent path-integrator. Therefore the improvements demonstrated over each are primarily due to the path-integrator token input that the model receives. Tem-t also has this advantage and so should also perform similarly well, which should be mentioned. It would be nice to see more comparisons between tem-t and mm-tem. The use of the meta-MLP memory instead of softmax attention is elegant, I would be curious to see the gains explored slightly more. Perhaps MLP is more brittle than softmax when faced with a novel but semantically familiar input? Or perhaps MLP imbues memory retrieval with generalisation capacity since it is itself a NN? And does this impact the learnt g representations of the outer-loop meta network? Maybe the MLP allows more interesting operations to be done on memories, e.g. contextual splicing of memories - if memorised spatial envs A and B and then encounter env C in which half the stimuli are from A states and half from B", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8tCKvyboPi", "reviewer": "Reviewer_3ncY", "summary": "This paper proposes mm-TEM, a variant of the Tolman-Eichenbaum Machine using\nmeta-MLP memory, and Hippoformer, which integrates mm-TEM with Transformers. The\nauthors evaluate these models on 2D grid prediction and 3D environment tasks,\nclaiming superior generalization and discovering that memory update frequency\naffects grid-like representation scales.", "questions": "1. Can you provide a direct ablation comparing mm-TEM against a flat associative memory (no factorization) with identical meta-MLP architecture, warm-up procedure, and auxiliary losses?\n2. Can you clarify the evaluation protocol for 2D tasks in Section 3.2? What percentage of positions in the \"imagination\" phase were previously visited during context?\n3. For the mb parameter effect on grid scales: Can you ablate training vs. test-time effects? (Train with mb=1, test with mb=8 and vice versa?)\n4. For Figure 4A: Can you provide non-parametric correlation measures (Spearman's ρ), test for non-linear relationships, and show residual plots?\n5. For Table 1: What specific error metric is used? How are \"visible\" vs. \"not visible\" frames classified in continuous 3D space?\n6. Can you provide wall-clock time comparisons and computational complexity analysis to support efficiency claims?\n7. How does performance scale to truly large environments where 64-step context covers <10% of reachable states?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* Implementation efficiency: mm-TEM trains faster than original TEM (in gradient * steps), making hippocampal-inspired architectures more practical - but see note below regarding \"efficiency\"\n* Integration with Transformers: The Hippoformer architecture combining structured spatial memory with Transformer's working memory is conceptually interesting.\n* Extensive empirical evaluation: The paper includes multiple experimental settings (varying context lengths, environment sizes, circular grids, 2D and 3D tasks).\n* Clear presentation: The paper is generally well-written with good figures illustrating the architecture and results.\n* Emergence of grid-like representations: The spontaneous development of grid patterns provides interesting connections to neuroscience.", "weaknesses": "## Missing Foundational Literature on Hippocampal-Entorhinal Memory Systems\n\nThe introduction would be strengthened by acknowledging the broader foundational\nliterature on hippocampal-entorhinal contributions to episodic and relational\nmemory beyond spatial navigation. Several core ideas presented as novel to the\nTEM framework—namely, factorized MEC-LEC streams, hippocampal binding of\nstructural and sensory codes, and generalization across relational spaces—have\nsubstantial precedent in earlier empirical and computational work. The 'binding\nof items and context' (BIC) model (Eichenbaum et al., 2007) directly addresses\nhow the hippocampus binds contextual and item information. Lesion and recording\nstudies have established MEC vs LEC functional dissociations (Hargreaves et al.,\n2005; Knierim et al., 2014). The Complementary Learning Systems framework\n(McClelland, McNaughton, & O'Reilly, 1995) proposed separate systems for rapid\nrelational binding and structured representations decades earlier. Computational\nmodels have explicitly implemented factorized spatial and sensory\nrepresentations: Hasselmo et al. (2002) modeled grid cells for memory with\nstructural/contextual codes; Franzius, Sprekeler & Wiskott (2007) demonstrated\ngrid/place code emergence with associative mapping to sensory inputs; Bush et\nal. (2015) explicitly separated grid-cell path integration from memory\nassociation in recurrent networks. I recommend that the authors broaden their\nintroductory discussion to acknowledge this literature and clarify precisely\nwhat mm-TEM contributes beyond these established frameworks rather than\nappearing to introduce factorized hippocampal memory de novo.\n\n\n## Terminology: \"TEM Theory\" Should Be \"TEM Model/Framework\"\n\nThroughout the manuscript, the authors refer to \"TEM theory\" (e.g., lines 54,\n134). I suggest replacing this with \"the TEM model\" or \"the TEM framework.\" The\nTolman-Eichenbaum Machine (Whittington et al., 2020) is a computational\ninstantiation of existing theoretical ideas about cognitive mapping and\nrelational memory, rather than a full-fledged theory in itself. The underlying\ntheoretical principles, in particular that the hippocampal-entorhinal system\nimplements factorized memory for flexible generalization, predate TEM by decades\n(see comment above). Calling TEM a \"theory\" risks overstating its conceptual\nnovelty and may mislead readers about its epistemic status. The contribution of\nWhittington et al. (2020) was to provide an elegant computational implementation\nof these principles, not to propose the theoretical framework itself. This\ndistinction matters for accurately situating the current work in the scientific\nliterature.\n\n\n## Missing Related Work on Factorized Memory Architecturs\nThe paper would benefit from acknowledging a broader range of computational\nmodels that have explored factorized memory architectures combining structural\nand sensory representations. Beyond the TEM lineage cited, several prior works\nshare the core principle of separating spatial/structural codes from\nsensory/content codes for flexible memory retrieval. Hasselmo et al. (2002)\nmodeled how entorhinal cortex provides structural/contextual codes while the\nhippocampus performs binding operations. The Complementary Learning Systems\nframework (McClelland, McNaughton, & O'Reilly, 1995), while not specifically\nEC-HC focused, proposed separate systems for rapid relational binding versus\nstructured long-term representations—a conceptual predecessor to factorized\nmemory architectures. Franzius, Sprekeler & Wiskott (2007) demonstrated\ngrid/place code emergence with associative mapping to sensory inputs, providing\nan early computational analogue of separating structural representations from\nsensory mappings. Bush et al. (2015) explicitly implemented computational\nseparation of grid-cell-like path integration from memory association in\nrecurrent networks. Stachenfeld et al. (2017) modeled the hippocampus as a\npredictive map using successor representations, integrating spatial structure\nwith value predictions. Waniek (2020) proposed Transition Scale-Spaces that\nintegrate structural and sensory information in multi-scale frameworks. Even\nwork on sequence memory in recurrent networks (e.g., Rajan, Harvey, & Tank,\n2016) employs separate state/structure representations with readout layers,\nloosely corresponding to mm-TEM's structural versus sensory binding.\nAcknowledging these precedents would provide better context for understanding\nhow mm-TEM's specific implementation choices (meta-MLP memory, auxiliary losses,\nTransformer integration) extend versus replicate prior ideas, and would help\nreaders assess the true novelty of the contribution.\n\n\n## Imprecise Terminology: \"Novelty or Surprisal\"\nThe manuscript refers to \"novelty or surprisal\" in describing the fast-weight\nupdate mechanism (lines 171-172). In formal information theory, these are\ndistinct concepts: surprisal is a probabilistic quantity (−log P(x)), whereas\nnovelty is relative to memory or prior experience (Palm, 2012). What the model\nactually computes is the gradient of reconstruction loss, ∇_Θ L(k_t, v_t), which\nmeasures prediction error -- the mismatch between predicted and actual values. I\nrecommend clarifying which quantity is actually computed and adjusting the\nterminology to reflect the implementation accurately. The most precise term\nwould be \"prediction error\" or \"reconstruction error\" rather than the ambiguous\n\"novelty or surprisal.\" This precision matters for readers seeking to understand\nthe computational mechanism and for those attempting to reproduce or extend the\nwork.\n\n\n## Misleading Description of Hippocampal-Entorhinal Feedback\nIn Appendix A.2 (lines 630-632), the manuscript states: \"visual sensory cues\nprovide feedback from the HC to correct path integration errors in the MEC.\"\nThis description is biologically misleading. While feedback from the hippocampus\ncan indeed help stabilize MEC representations and correct errors in path\nintegration (Diehl et al. 2019, Mulas et al. 2016, and many others), the hippocampus does not send raw visual sensory information to\nMEC. Visual and other sensory inputs primarily reach MEC via cortical pathways\n(particularly through LEC from perirhinal and parahippocampal cortices). A more\naccurate description would be that the hippocampus provides relational or\nspatial feedback, likely in the form of conjunctive codes that bind spatial and\nsensory information, that can help recalibrate MEC structural representations,\nwhile MEC continues to integrate sensory cues from neocortical areas. I\nrecommend rephrasing this passage to clarify that the feedback represents a\ncomputational abstraction where HC provides spatial/relational corrections\nrather than literal sensory signals. This distinction is important for readers\ninterpreting the model in a neurobiological context and for understanding what\naspects of the architecture are biologically plausible versus computational\nconveniences.\n\n\n## Missing Critical Baseline: Direct Associative Memory Comparison\nThe paper's central claim is that factorizing memory into structural codes (via\npath integration) and sensory codes enables superior generalization compared to\nstandard architectures. However, the experimental evaluation lacks a critical\nablation: a direct associative memory baseline that learns state-transition\npairs (state_t, action_t) → state_{t+1} using the same meta-MLP architecture,\nwarm-up procedure, and auxiliary losses, but without structural factorization.\nThe current baselines (Transformer, Titans) differ from mm-TEM in multiple\nconfounded ways -- architecture, training procedures, and task formulation --\nmaking it impossible to isolate whether the performance gains stem from the\nfactorized representation itself or simply from having a meta-trained\nassociative memory.  The ablations in Fig. 3C only remove auxiliary losses, not\nthe core architectural choice of factorization. Furthermore, the claim of\n\"efficiency\" lacks rigor: no computational complexity analysis, wall-clock time\ncomparison, or memory footprint evaluation is provided, only gradient step counts\nagainst the original TEM. The comparison is further confounded by the warm-up\npre-training phase whose application to baselines is unclear. Finally, in small\ngrid worlds (8×8 to 11×11), a 64-step context likely covers a substantial\nfraction of reachable states, meaning the model may largely be performing\nwithin-episode memory retrieval rather than true generalization to novel spatial\nconfigurations. A proper ablation study isolating the contribution of structural\nfactorization is essential to validate the paper's core hypothesis.\n\n## Insufficient Mechanistic Explanation for Grid Scale vs. Update Frequency Relationship\nThe paper claims that the memory update frequency hyperparameter mb controls\ngrid scale through an \"effective prediction horizon\" mechanism (lines 252-256),\npositioning this as a novel insight into grid-scale diversity. However, this\nexplanation lacks mechanistic rigor and conflates multiple confounded factors.\nFirst, mb affects both training dynamics (how often gradients update the\nmeta-MLP weights) and inference behavior (how stale the memory becomes), but the\npaper does not disentangle which factor drives the grid scale effect. The\nobserved correlation could simply be an artifact of optimization\ndynamics. That is, sparser gradient updates naturally produce temporally smoother,\nlower, frequency representations through gradient accumulation, rather than\nreflecting a meaningful \"prediction horizon.\" Second, the feedback mechanism\nfrom relational memory to path integration (Appendix A.2) is central to\nunderstanding this effect but is incompletely described: the function f_delta is\nnot defined, the strength of feedback (α) is not analyzed, and how mb interacts\nwith this feedback is unclear. Third, no ablation studies isolate the causal\nmechanism. For instance, training with mb=1 but testing with mb=8, or analyzing\ngradient flow as a function of mb. Without this mechanistic analysis, the claim\nthat mb reveals insights into biological grid-scale diversity through\n\"multi-timescale predictions\" remains speculative correlation rather than\ndemonstrated causation. The authors should provide: (1) ablations separating\ntraining-time vs. test-time effects of mb, (2) gradient flow analysis explaining\nwhy different update frequencies produce different spatial frequencies, and (3)\ncomplete mathematical description of the feedback mechanism.\n\n## Missing Citation and Overclaimed Novelty on Grid Scale Mechanisms\nThe paper claims to reveal \"a novel mechanism for grid-scale diversity in\nMEC...as a natural consequence of multi-timescale predictions in the brain\"\n(lines 252-256), based on their observation that the memory update frequency mb\naffects grid scale. However, this is not novel -- Waniek (2020, \"Transition\nScale-Spaces: A Computational Theory for the Discretized Entorhinal Cortex\")\nanalytically derived that grid scales emerge from different prediction horizons,\nwith the spatial frequency directly related to the temporal prediction distance.\nThe current paper essentially re-discovers this relationship empirically without\nciting this prior work. Moreover, Waniek's analysis is rigorous in the sense of\nan analytical derivation, compared to the current paper's informal \"effective\nprediction horizon\" argument. Related work by Stachenfeld et al. (2017) and\nDordek et al. (2016) also established connections between temporal prediction\nscales and spatial grid scales. The authors should: (1) cite this prior\ntheoretical work, (2) clarify what is actually novel about their contribution\nbeyond empirical confirmation in a different architecture, and (3) either remove\nthe novelty claim or demonstrate what their mechanism adds beyond existing\ntheory.\n\n## Ambiguity in Generalization Claims Across Sections 3.1 and 3.2\nThe paper creates confusion about what \"generalization\" means across different\nexperiments. In Appendix A.1 (lines 600-602), the authors state that for \"all 2D\ngrid prediction tasks,\" evaluation is confined to previously visited positions\nbecause \"predicting observations at unvisited locations is not meaningful\" given\nthat observations are uncorrelated discrete IDs. However, in Section 3.2, which\nclaims to test \"generalization\" (line 260), this critical constraint is never\nrestated, leaving readers to infer that it still applies. This matters because\nSection 3.2's \"multi-step imagination\" is framed as testing the model's ability\nto \"generalize beyond its training horizon\" (line 268), but if evaluation is\nconfined to previously encountered positions, it's actually testing memory\nretrieval over longer sequences, not spatial generalization to novel states. The\npaper should explicitly clarify for each experiment: (1) whether evaluation\nincludes unseen positions, (2) what fraction of test positions were visited\nduring context, and (3) how \"accuracy\" is computed when positions are/aren't\npreviously seen. The distinction is crucial: the 3D experiments (Section 3.4)\nacknowledge that \"unvisited observations can be inferred from nearby spatial\ninformation\" due to continuous visual features, but no such acknowledgment is\nmade for 2D tasks where this fundamental difference in evaluation regimes should\nbe highlighted upfront in Section 3.2, not buried in the appendix.\n\n\n## Unjustified Causal Claims from Correlational Data (Section 3.2, Figure 4)\nThe paper claims that \"the presence of strongly grid-like cells is a key driver\nfor generalization\" (lines 346-347) based solely on a correlation between grid\nscore and prediction accuracy (r=0.647, Fig. 4A). This is a causal claim\nunsupported by the evidence. Correlation does not establish causality -- the\nrelationship could be due to reverse causation (good generalization enables\nbetter grid formation) or a confounding variable (successful learning produces\nboth regular representations and good generalization). Notably, the paper's own\ndata weakens the causal claim: Figure 4B shows models with low grid scores\nachieving high accuracy through \"alternative but still regular neural\nrepresentations,\" suggesting that regularity (not grid-ness specifically) may be\nwhat matters. To establish causality, the authors should conduct interventional\nexperiments: (1) inject hand-designed grid representations and test if\nperformance improves, (2) add regularization to suppress grid formation and test\nif performance degrades, or (3) explicitly bias learning toward grid patterns\nand compare against controls. As written, the claim that grids \"facilitate\" or\n\"drive\" generalization is speculative. The authors should either provide causal\nevidence or rephrase their claims to accurately reflect the correlational nature\nof their findings (e.g., \"grid scores correlate with generalization performance,\nsuggesting a potential relationship\").\n\n\n## Questionable Statistical Analysis in Figure 4A\nThe correlation analysis in Figure 4A raises several statistical concerns.\nFirst, while reporting r=0.647 (p=0.0002), the authors use linear regression\ndespite evidence of non-linearity: accuracy appears to show a ceiling effect\n(~0.95-1.0) and the relationship exhibits heteroscedasticity (variance in\naccuracy is much higher at low grid scores than high). This violates key\nassumptions of linear regression, making standard errors, confidence intervals,\nand p-values unreliable. Second, with only ~20-25 data points visible, the\nanalysis is sensitive to outliers and the wide confidence interval suggests\nsubstantial uncertainty. Third, r²≈0.42 means 58% of variance in accuracy\nremains unexplained, yet the authors make strong causal claims (\"key driver\")\nfrom this weak-to-moderate correlation. Fourth, the scatter plot shows several\ncounterexamples to the claimed relationship: models with grid scores around\n0.7-0.9 achieve accuracy >0.9, while models with grid scores ~1.0-1.1 have\naccuracy ~0.8. These observations, also noted in the text regarding \"low grid\nscores still achieve high accuracy\" (Fig 4B), directly contradict the linear\nrelationship implied by the regression line. The authors should: (1) test for\nnon-linear relationships (threshold models, saturation functions), (2) report\nnon-parametric correlations (Spearman's ρ) that don't assume linearity, (3) show\nresidual plots and test assumptions, (4) report prediction intervals to\ndemonstrate the large uncertainty in predictions, and (5) acknowledge that grid\nscore alone is a poor predictor of performance. The statistical evidence does\nnot support the strong causal claims made in the text.\n\n## Undefined Error Metric and Inconsistent Terminology in 3D Experiments (Section 3.4, Table 1)\nThe 3D environment evaluation suffers from unclear methodology and inconsistent\nreporting. Table 1 reports \"prediction error in units of 1e-3\" but never defines\nwhat error metric is used. Presumably MSE between predicted and ground truth\negocentric images, but this is not stated. Are errors computed in pixel space,\nnormalized space, or feature space? How are images preprocessed? The text\ncompounds confusion by referring to \"accuracy\" (lines 432-435) while the table\nshows \"error\". These are opposite metrics (higher accuracy vs. lower error is\nbetter). The classification of frames as \"visible\" vs \"not visible\" is\nundefined: what determines if a frame is considered previously seen in\ncontinuous 3D space with egocentric views? The reported standard deviations\nraise questions: one-step errors show std=0.00 across 3 seeds for all models\n(presumably due to rounding, but this should be clarified), while Hippoformer's\nmulti-step std (0.04) is 100× smaller than baselines (4-5). Why is Hippoformer so\nmuch more stable? Finally, without baseline comparisons (chance level, naive\npredictors) or interpretation of error magnitudes (e.g., \"0.001 corresponds to X\nper-pixel deviation\"), the numbers lack context. The authors should: (1)\nexplicitly define the error metric and computation procedure, (2) use consistent\nterminology (error or accuracy, not both), (3) specify the visible/not-visible\nclassification criterion, (4) explain the variance patterns across models, and\n(5) provide baselines and interpretation to make the magnitudes meaningful.\n\n\n## Discussion Section Overclaims Novelty and Omits Critical Limitations\nThe Discussion overclaims novelty and omits acknowledgment of significant\nlimitations identified throughout the paper. First, the claim that mm-TEM offers\n\"a new functional perspective on grid diversity\" (line 445) ignores Waniek\n(2020), who analytically derived that grid scales emerge from prediction\ndistances - the very relationship mm-TEM rediscovers empirically. Second, the\nRelated Work section omits foundational hippocampal-entorhinal literature\n(Eichenbaum et al., 2007; Hasselmo et al., 2002; Stachenfeld et al., 2017;\nFranzius et al., 2007; Bush et al., 2015) that established factorized memory\narchitectures and prediction-timescale-to-spatial-scale relationships decades\nbefore TEM. Third, while the authors acknowledge limited integration and\nsingle-layer design, they fail to address major methodological limitations: (1)\n2D evaluation confined to previously visited positions (Appendix A.1), meaning\n\"generalization\" is actually memory retrieval, not spatial inference to novel\nstates; (2) small environments (8×8 to 11×11) where 64-step context covers\nsubstantial state space; (3) no ablation isolating the contribution of\nfactorization versus direct associative memory; (4) causal claims about\ngrid-generalization relationship based solely on correlation (r=0.647); (5)\nundefined error metrics and inconsistent terminology in 3D experiments. Fourth,\nefficiency and scalability claims lack rigor: no computational complexity\nanalysis, wall-clock time comparisons, or large-scale demonstrations are\nprovided. The Discussion should thus (1) cite prior work and clarify what is\nnovel about the grid-scale finding beyond empirical confirmation, (2)\nacknowledge foundational neuroscience literature on factorized memory, (3)\nexplicitly discuss the limitation that 2D \"generalization\" is constrained to\npreviously visited positions (or clarify the text) (4) acknowledge that\ngrid-generalization causality remains unproven, (5) provide concrete criteria\nfor when mm-TEM/Hippoformer would be preferred over standard architectures, and\n(6) temper claims about efficiency and scalability until rigorous evidence is\nprovided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)