Relating transformers to models and neural representations of the hippocampal formation
Authors: James C. R. Whittington
Joseph Warren
Tim E.J. Behrens
Venue: ICLR 2022 Poster (2022)
Keywords: Neuroscience
representation learning
hippocampus
cortex
transformers

Abstract: Many deep neural network architectures loosely based on brain networks have recently been shown to replicate neural firing patterns observed in the brain. One of the most exciting and promising novel architectures, the Transformer neural network, was developed without the brain in mind. In this work, we show that transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation; most notably place and grid cells. Furthermore, we show that this result is no surprise since it is closely related to current hippocampal models from neuroscience. We additionally show the transformer version offers dramatic performance gains over the neuroscience version. This work continues to bind computations of artificial and brain networks, offers a novel understanding of the hippocampal-cortical interaction, and suggests how wider cortical areas may perform complex tasks beyond current neuroscience models such as language comprehension.

Published as a conference paper at ICLR 2022
RELATING TRANSFORMERS TO MODELS AND NEURAL REPRESENTATIONS OF THE HIPPOCAMPAL FORMATION
James C.R. Whittington"
Joseph Warren, Timothy E.J. Behrens University of Oxford & University College London
University of Oxford Stanford University
ABSTRACT
Many deep neural network architectures loosely based on brain networks have re- cently been shown to replicate neural firing patterns observed in the brain. One of the most exciting and promising novel architectures, the Transformer neural network, was developed without the brain in mind. In this work, we show that transformers, when equipped with recurrent position encodings, replicate the pre- cisely tuned spatial representations of the hippocampal formation; most notably place and grid cells. Furthermore, we show that this result no surprise since it is closely related to current hippocampal models from neuroscience. We addi- tionally show the transformer version offers dramatic performance gains over the neuroscience version. This work continues to bind computations of artificial and brain networks, offers a novel understanding of the hippocampal-corticall interac- tion, and suggests how wider cortical areas may perform complex tasks beyond current neuroscience models such as language comprehension.
INTRODUCTION
The last ten years have seen dramatic developments using deep neural networks, from computer vision (Krizhevsky et al. 2012) to natural language processing and beyond (Vaswani et al.) (017). During the same time, neuroscientists have used these tools build models of the brain that explain neural recordings at precision not seen before (Yamins et al. [0014] Banino et al. 2018 W hitting- ton et al. 2020). For example, representations from convolutional neural networks (Lecun et al., 1998) predict neurons in visual and inferior temporal cortex (Yamins et al. 2014]] Khaligh- Koiegeskorte 2014), representations from transformer neural networks (Vaswani et al.] 2017) predict brain representations in language areas (Schrimpf et al., 2020). and lastly recurrent neural networks (Cueva & Wei 2018 Banino et al.) 2018; [Sorscher et al.) (219) have been shown to recapitulate grid cells (Hafting al.) 2005) from medial entorhinal cortex. Being able use models from ma- chine learning to predict brain representations provides a deeper understanding into the mechanistic computations of the respective brain areas, and offers deeper insight into the nature of the models.
&
As well as using off-tte-shheff machine learning models, neuroscience has developed bespoke deep learning models (mixing together recurrent networks with memory networks) that learn neural rep- resentations that mimic the exquisite spatial representations found in hippocampus and entorhinal cortex (Whitington et al. 2020 Uria et al.) 2020), including grid cells (Hafting et al. 2005), band cells (Krupic et al., 2012). and place cells (e'Keefc & Dostrovsky, [971). However, since these models are bespoke, it is not clear whether they, and by implication the hippocampal architecture, are capable of the general purpose computations of the kind studied in machine learning.
In this work we 1) show that transformers (with a little twist) recapitulate spatial representations found in the brain; 2) show a close mathematical relationship of this transformer to current hip- pocampal models from neuroscience (with a focus on Whittington et al. (2020) though the same is true for Uria et al. (2020)); 3) offer novel take the computational role of the hippocampus, and an instantiation of hippocampal indexing theory (Teyler & Rudy] 2007); 4) offer novel insights on the role of positional encodings in transformers, 5) discuss whether similar computational principles might apply to broader cognitive domains, such as language, either in the hippocampal formation or in neocortical circuits.
"Correspondence to: jerwhi i wwwwwhttiigteon tningtonngmalll coc

---

Published as a conference paper at ICLR 2022
Note, we are not saying the brain is closely related to transformers because it learns the same neural representations, instead we are saying the relationship is close because we have shown a mathe- matical relationship between transformers and carefully formulated neuroscience models of the hip- pocampal formation. This relationship helps us get a better understanding of hippocampal models, it also suggests new mechanism for place cells that would not be possible without this mathematical relationship, and finally it tells us something formal about position encodings in transformers.
2
TRANSFORMERS
Transformer Neural Networks (Vaswani et al., 2017) are highly successful machine learning algo- rithms. Originally developed for language, transformers perform well on other tasks that can be posed sequentially, such as mathematical understanding, logic problems (Brown et al. 2020), and image processing (Dosovitskiy et al., 2020).
Transformers accept a set of observations; X {01 22 "3.... bedding image patch etc), and aim to predict missing elements of that set. The missing ele- ments could be in the future, i.c EtyT, or could be a missing part of a sentence or image, i.c (x1 the, x2 cat, 23 sat, W. = ?, 25 the, Eg mat).
Fr (I,, could a word em-
The core mechanism of transformers self-attention. Self-attention allows each element to 'attend" to all other elements, and update itself accordingly. In the example data-set above, the 4th element (?) could attend to the 2nd (cat), 3rd (sat), and 6th (mat) to understand it should be on. Formally, attend to another element each element (az, row vector) emits query (q: = x; W) and compares it to other elements keys (k, = 2, Ws). Each element is then updated using yı: Σx r(q., k )v, where k(q,, k,) is kernel describing the similarity of q, to k- and v- is the value computed by each element v, = æ, W., Intuitively, the similarity measure n(qe, k,) places more emphasis on the elements that are relevant for prediction; in this example, the keys may contain information about whether the word is a noun, verb or adjective, while the query may 'ask' for any elements that are nouns or verbs elements that match this criteria (large k(qi, kr), i.e. cat, sat, mat) are 'attended" to and therefore contribute more to the output 1/- Typically, the similarity measure a softmax i.c. x(qe, k.) = ".. ".ackr These equations can be succinetly expressed in matrix form, with all elements updated simultane- ously:
Self-attention.
Sut kr
9.K dd
QK
= softmar
V
7 Y = softmmoort
Vd V
(1)
Here Q. K, V are matrices with rows filled by q:. kr, Us respectively, and the softmax is taken independently for each row. After this update, each z is then sent through a deep network (fe(-..)) typically consisting of residual (He et al.) (016) and layer- normalasaaiii (Ba et al. 201 layers to produce 24 = fo ). Z is the output of the transformer which can then be used for prediction, or sent through subsequent transformer blocks.
Position encodings. Self-attention is permutation invariant and so tells you nothing about order of the inputs. Should the data be sequential (i.e. meaning depends the order of elements, such as in language, or navigation as we will see later!), it is necessary to additionally encode the position/ where is in the sequence. This is typically done by adding a 'position encoding" that uniquely identifies each time-step (e, typically sines and cosines) to each input: 2, + te Alternatively the position embedding can be appended i.c. h, = [ ez], with self attention then performed using h as input.
3 TRANSFORMERS LEARN ENTORHINAL REPRESENTATIONS
Here we show that transformers (with a small modification) recapitulate spatial representations grid and band cells when trained on tasks that require abstract spatial knowledge.
Spatial understanding task. The task (more detail in Appendix) is to predict upcoming sensory ob- servations I++1 conditioned on taking action a while moving around spatial environments (Fig- ure Th). For example, after seeing {(z, = cat, aj = North), (x, = dog, a2 = East), (I3 = frog, 0g = South), (2t4 = pig, a4 West), (#s 7, a, = ).. the aim is to predict
2

---

Published as a conference paper at ICLR 2022
Environment
Environmen 2
b
Transformer
Key, Value
Query
posttion
Observations Actions (a)
Vinu
Wige)
Recurrent connections
grid
mape
Cell rate maps
Autocorneiations
Cell rate maps
activations
Autocorreaaiion
Band colls
Figure 1: (a) Sequence prediction in spatial navigation tasks test abstract spatial understanding since some sensory predictions only be done by knowing (generalising) certain rules e.g. North East + South + West = or P Parent + Sibling + Niece = 0. Note, we use sequences drawn from much larger graphs (b) Transformer with recurrent position encodings. (c) Real grid cell rate-maps (Hafting et al. 2005). (d-f) Learned position embedding rate-maps (i.e. average activity each spatial location; plots are spatially smoothed). (d-e) Resembling grid cells with (e) linear activation or (e) ReLu activation post transition. (f) Resembling band cells (Krupic et al. 2012).
25s cat. For simplicity, we treat sensory observations as one-hot vectors, thus the prediction problem is a classification problem.
When faced with unseen stimulus-action pair (e.g. x4 pig. a4 West above; action you have never taken at that stimulus before), successful prediction requires more than just remembering specific sequences of stimulus-action pairs; knowledge of the rules of space must be known; i.c. North + East + South + West = 0 allows prediction of Ø; = cat. Crucially, such rules generalise any 2D spaces and may therefore be transferred to aid prediction in entirely novel 2D environments. This powerful, since unobserved relations between observed stimuli can be inferred in zero-shot manner.
However, these relational rules are not 'known' priori and therefore must be learnt. We therefore train across multiple different spatial environments which share the same underlying 4-connected Euclidean structure (Figure Th)- this means the model must learn and generalise the abstract struc- ture of space to use for prediction in new environments.
To perform on these tasks, the three modifications to the transformer are:
1. Recall equation Y/- soflmar( giK V. where Q = HW g. K = HW V dd,
HW W.. and H matrix of inputs and position encodings (i.e. its rows are h, = [ er)) We restrict these weight matrices such that queries (Q) and keys (K) are the same; Q K EW.. We refer to this matrix as E. Thus the keys and queries only focus on position encodings. Meanwhile, values are exclusively dependent on the stimulus component of H i.e. XW We refer this matrix as X.
K.K
y: softmar
JV
y: =
X
VTa
VTa
This is an extreme version of the realisation that, in transformers, best performance is when position encodings are used to compute keys and queries, but values.
2. We causal transformers; the key and value matrices contain the projected position codings and sensory stimuli respectively at all previous time-steps (i.e. ect and xxt). This is equivalent causal 'unmasking" the agent wanders the environment accumu-
3

---

Published as a conference paper at ICLR 2022
a
C
Samuu (32)
environment
1o--5onooo stimulii (LEG)
environment 2
9o
91
9
conjunctive code (HPC)
conjunctive (HPC)
I
Hemory network
network
Sensery
prediction
Xg
X;
?
structural code (MEC)
structural code (MEC)
b
d
Grid cells
environment 1 HPC conjunction
environment 2 HPC conjunction
sensorry code (LEC)
10000011 1000 (LEC)
Place cells
X- 000000
structural code (MEC)
structural code (MEC)
Figure 2: (a) The TEM model, with a path integration component (equation 3 and memory network component (equation (gnua TEM path integrates g and makes sensory predictions via its memory network (dashed lines are additional connections for inference). (b) TEM recapitulates
a host of empirically described cell representations (Whitiigttnn et al. 2020). Top/bottom row:
example TEM MEC/Hippocampal representations (plots are spatially smoothed). Figures adapted from W Whittington et al. (0220). (c) Schematic of TEM (adapted from Sanders et al. (2020), showing that the same cortical representations (LEC and MEC) are reused different environments allowing for generalisation, facilitated by different hippocampal combinations.
(d) The TEM hippocampal conjunction an outer product- cells receive input from particular MEC and LEC cells.
lating new experiences (not-yee--xpprriened cimmuls-positton pairs are inaccessible to the agent). Meanwhile the query at each time-point is the present positional encoding es.
3. The position encodings are recurrently generated (as in Wang et al. (2019; Liu et al. (2020)): et+1 = (e, where W. a is learnable action-dependent weight matrix, and (.... is a non-linear activation function. This means that unlike traditional transformers, position encodings can be optimised and not the same for every sequence. It becomes interesting to see what representations are learned.
These modifications are sufficient learn spatial representations, in the position encodings, that mimic representations observed in the brain (Figure [ see Appendix for model and training de- tails). The rest of this paper now explains why this not a surprising result; namely we show that transformer with recurrent positional encodings is closely related to current neuroscience models of the hippocampus and surrounding cortex (Whittington et al. [2220] Uria et al.) (2200). Here we focus on the Dolman-Eichenbane Machine (TEM) (Whittinton et al. 2020). though the same principles apply for Uria et al. (2020).
The critical points are: 1) the memory component of TEM can be viewed transformer self- attention, since the TEM memory network analogous to Hopfield network (Hopfield, 1982) which have recently been shown be closely related to transformers (Ramsauer et al. 2020); 2) TEM path integration (see below) can viewed as a way to learn a position encoding.
TEM
The Einnannamm Machine (TEM; Figure 2, further details in Appendix) is a neuroscience model that captures many known neural phenomena in hippocampus (HPC) and entorhinal cortex
4

---

Published as a conference paper at ICLR 2022
(medial/lateral: MEC/LEC). TEM is sequence learner trained on tasks exactly like the one de- scribed in the previous section. TEM consists of two parts;
1) A module that aims understand where it is in space, using representation g to represent location. To update its location, TEM uses vectors enacted in recurrent neural network:
the accumulation of self movement
g++1 o W.)
Where W. is a learnable action dependent weight matrix and o(...) .) is a non-linear activation function. It is in this path-integrating representationg that TEM learns grid and other entorhinal cells for self-localisation (Figure p)
2) To make sensory predictions, location representations g alone are not enough; they must each link to a sensory observation æ, corresponding to the stimulus at that position. Note that these links specific to environment, since each environment consists of a different arrangement of stimuli in space (i.e. different stimulus- posittionss pairings).
The linking is done by binding every element of g with every element of æ, in other words outer product that is flattened back into a vector;
p flatlen(a
(4)
These conjunctive p representations are stored in 'fstt weights' V
Hebbian learning,
t-1
M, DPTP
(5)
T=1
And they can later be retrieved using attractor network (a continuous version of the Hopfield network). Here a query vector q (details next paragraph) is inputted into the network and updated via;
q a- (q)))
where o....) is a non-linear activation function; a ReLu in TEM. Crucially, because the memories formed using both and æ, they can be retrieved (pattem-completed) using just of those representations alone i.e. 'what did see the last time was here' or 'where was the last time I saw this'. To retrieve memorised conjunction p. TEM imagines (path-integrates) the next location and provides this as input to the attractor network in the form q = flatten g). Equation is then iterated until a memory is retrieved.
Finally, to make sensory predictions, the retrieved conjunctive memory (p) (prefrieved) is 'deconjunc- tified into sensory and location components. The sensory component obtained by unflattening retrieved and summing over the dimension (Figure 8):
p,
x retriesed sum Sumfuuflultten(p reetieeed 1,
Finally, to make the sensory prediction x xetrievedd is fed through a MLP 2; = fo fo( to classify (prediet) the upcoming sensory observation.
It is also possible, and often helpful, to project g and x via W,, and Wa; g = gW, and ã = x: W. before they are combined conjunctively
5 TEM AS TRANSFORMER
Here we show that the above equations of TEM can be written so that: the memory retrieval components looks like a transformer self-attention; 2) the path integration representation, look like position encodings.
We note such 'fast weights' have previously been thought of alternative to the LSTM (Ba al. 2016a).
In fact, in the TEM code online, W g and W, are set fixed weight matrices, where W. sub-samples and W. transforms (compresses) x from one-hot two-hot representation.
5

---

Published as a conference paper at ICLR 2022
Transformer (with causal attention mosk)
TEM
Kay, Value
FWL
Query
IW.
Input with position encoding
Figure 3: Self-attention in (a) Transformers and (b) TEM.
1) When considering the TEM memory retrieval process more closely (in this analysis, for direct comparison, we are only considering attractor step in TEM with no non-linearity), we see that the attractor update M q: Σ p. Pr is simply equal to
P
pretrieved_
Elarr
(8)
Since [q.p is just dot-product (q:: Pr)), single step of the attractor just retrieves memories weighted by their similarity (dot product) to the query. As noted by |Ramsauer et al. (2020), this is exactly like transformer but without the softmax scaling the dot-products. Thus the TEM memory retrieval process behaves like transformer self-attention.
2) We however go further since TEM's input to the transformer (i.e. the TEM memories) special; they are learnable and built from an outer product between I and 0 (p, = flatten (i" g.) ). and these memories can be retrieved by a query based on 9 or o alone (e.g. q: flatten (1 9:)). Together, these properties mean we can reduce the above dot product even further;
1qpp T ž g 9-1
ratricued p
gg G A
(9)
Where - = (.r),, and A is a diagonal matrix with elements ã, (see Appendix for an alternative derivation using vector elements). Thus retrieve conjunctive p memory, all that necessary weighting past p representations via 'self-attention' of gt to past representations G.
To simplify this even further, we consider what happens when we 'deconjunctify" p, pretrieved to obtain the sensory component of the memory. Following the TEM procedure described above (Figure 8):
T gretrieved = sum(unflatteng peettet petrienedd 1, = Σ 3. F. 0+1 =
(10)
Where Ag is a diagonal matrix with elements ÿ. (rr)). Now all that is necessary to retrieve the sensory component of the memory weighting past representations with via 'self-attention' of ğr to past representations G. This equation is now very similar to equation | except without the softmax and with additional weightings A, and Ag. These weighting however are likely learned to be constant (a) because otherwise some memories will be preferentially retrieved. In this case TEM is retrieving memories using
G. VTh
a reetrieved
(ag.G") X
cf.
softmar
)
(11)
Which can be seen to be very closely related to the transformer equation (shown the right), and diagrammatically shown in Figure? 3 softmax rule.
The model presented in this paper utilises the full transformer
The TEM-transformer.
Thus the teM-trrasformer (TEM-t; from Section 3 is this transformer that is directly analogous TEM. Additional modelling details (analogous to modelling details in TEM) can be found the Appendix. TEM-t offers dramatic performance improvements over the original TEM model (Figure code will be released on publication). In particular, 1) Sample

---

Published as a conference paper at ICLR 2022
a
b
80
80
Zero-shol accuracy
60
60
Zero-ahof
40
TEM-
TEM-
TEM
TEM
15000
10000
staps
Time (8)
Figure 4: TEM-t is more efficient learner than TEM, both in (a) sample efficiency and (b) time per gradient step. Zero-shot accuracy is prediction accuracy when taking links it has never taken before, but to state it has visited before. Successful accuracy here is only possible with learned and generalised spatial knowledge. We have used the code from TEM from the TEM authors original code https:: / com/ wtwpc///geeeeaaa....e.. wwwssiggstuuuttt struct wal--nowledee so have not optimised for speed of learning etc, so we cannot claim this to be a fair compari- son, nevertheless the difference is stark. We note that in the TEM paper, the authors say it takes up to 50,000 gradient updates for full training, whereas we stopped at 20,000.
and
efficiency is increased; TEM- requires many fewer data samples than TEM, and thus training time reduced 2) TEM-t can tackle much larger problems, with the ability store and retrieve many more memories (not shown here). Additionally to improved performance, TEM-t learns grid cells (Figure D and has potential implications for what place cells are (see next section).
Path integrating position encodings. This leads us to an interesting observation; we see that TEM's representations for path integration g plays the role of position encodings transformers. However the structure of these positional encodings are not hard-coded, but instead learned via path inte- gration (the structure of space!), with the particular position encoding depending on the particular sequence of actions taken. Other (non-spatial) structural representations could also be learned de- pending on the task at hand, i.c. grammar for language. This is a very different (and we think fruitful) re-understanding of position encodings; representing 'location" in (learned) structure that can be inferred on the fly.
PLACE CELLS IN TRANSFORMERS
Here we discuss, and demonstrate, how TEM-1 offers a new interpretation of place representations. To do so we utilise recent suggestion of how the transformer update can be performed in biological hardware (Krotov& Hopfield 2020). particular, self-attention (equation 1 can be split into two
steps which correspond to two pools of neurons (Figure 5 A); 1) calculate softmart k. ). 2) Vd, multiply by V. In this light, K and V can simply be seen as weight matrices between feature neuron (representing the query) and memory neurons (computing the softmax).
Since memory neurons sparsely activated due the softmax, they appear have spatial tuning for each environment resembling hippocampal place cells (Figure 5D-E; note Krotov & Hopfield (2020) stated memory neurons may correspond to place cells but without simulation). Similarly to experimentally recorded place cells, these neurons remap randomly between environments i.e. place cells being neighbours one environment is not predictive of them being neighbours another (unlike grid cells which maintain their phase neighbours across environments).
We can curate this architecture for the specifics of TEM-L TEM-t explicitly considers factorised g and æ representations (e.g. MEC and LEC), which project to feature neurons in hippocampus (or still cortex). Thus the feature neurons consist two separate sub- sub-populations, g = g W and ã æWa, but which connect to the same memory neurons in hippocampus (Figure BB-C). These feature sub-pon cobppopulations are updated alternately rather than simultaneously. depending on the direction of retrieval; for example, when retrieving ă the ÿ feature neurons stay constant while the ă

---

Published as a conference paper at ICLR 2022
A Memory neurons
G
HPC
X
O Feature neurons
9
2
Environment
Environment
W
Remapping
21
MEC LEC
Figure 5: TEM-Transformn neural architecture. (a) Krotov & Hopfield (2020) describe a neu- rally plausible architectural instantiation the 'Hopfield networks is you need" with separation between 'feature' neurons (i.e. h) and memory neurons (i.e. softma((::
(b-c) This can be extended for TEM-t, but now the feature neurons are not all updated simultaneously. but only those across brain regions. (d) Memory neurons resemble hippocampal place cells and (e) remap domly across environments. (f) possible architecture where cortical neurons project to feature neurons in hippocampus which in turn project memory neurons in hippocampus. (g) Additional brain regions can be included easily in this architecture with minimal increase in hippocampal neu- ron number.
neurons are updated (in turn updating æ in LEC). In this vein, hippocampal memories link together cortical representations in potentially disparate brain areas. Thus TEM-t instantiates hippocampal indexing theory (Teyler & Rudy 2007). which states that hippocampus provides an index that binds together cortical patterns across different brain regions.
The randomly remapping place cells described one paragraph ago cannot be the full picture since we know that place cell remapping is not random; instead individual place cells preferentially remap locations consistent particular grid cell firing (as predicted by conjunctive memory cells p in TEM and verified experimentally in Whittington phenomena born from TEM- could be as follows. Should the feature neurons exist in hippocampus (Figure 7 then there will be hippocampal spatial cells ğ that maintain their relationship to grid cells across different environment (as they are inherited from g via projection ğ gWg). Thus across the population of hippocampal cells, there will be those that maintain their relationship to grid cells (e.g. and those that don't (e.g. memory neurons and T), but the population effect will exist, just like what experimentally observed.
al.) (2020)). However another mechanism for this
As a note, the particular relationship of our model to the model of Krotov & Hopfilld (2020), is what they refer to as a 'type B' model. These are models with contrastive normalisation on the memory neurons (via softmax in our case), as opposed to 'type B' models which have power activation function on the memory neurons. TEM (left hand side of Equation [T] corresponds to linear activation function on the memory neurons, and is directly analogous the original Hopfield energy. Secondly, the notion that there are two types of feature neurons that can be bound together in the same memory, was explored in Krotov & Hopfield (2016) where pixel intensities with labels of those images. In TEM-t, one the feature vectors, is learned via a RNN and structures itself according to the underlying task structure.
associated
As an additional aside, we note that Krotov & Hopfield (2020) architectures does not solve the scaling problem of conventional Hopfield networks; the number of memories that original Hopfield networks could store scaled linearly with the dimensionality of the recurrent attractor network (Amil al. 1985). While recent analytical work has shown with exponential power activation functions, the number of memories that can be stored to scale as 2 2 where N is the dimensionality of the feature neurons (Demircigil et al. 2017). This is considerably more favourable scaling. However, unfortunately the architecture from Krotov & Hopfield (2020) instead requires a growing number of memory neurons (one for each memory), so the number of memories is still linear with the number of neurons! We note that mathematically derived scaling law was for an exponential activation function, not with softmax as we use here.
8

---

Published as a conference paper at ICLR 2022
7 DISCUSSION
We have shown that TEM, a current model of the hippocampal formation, is closely related to transformer with recurrent position encodings. We now consider some wider implications for neuroscience.
Multiple cortical inputs to hippocampus. TEM considers hippocampal conjunctions between two cortical regions (g and æ²). It is, however, possible to consider conjunctions of more than two brain regions. Indeed hippocampal neurons often respond to more than two task variables (McKenzie al. 2014). In TEM, the naive approach of a 'triple" (or higher) conjunction would increase the number of hippocampal neurons would increase by a factor of nei the number of neurons from brain region ē. TEM-t does not scale so badly. Instead it just requires an additional ne feature neurons, and the number of memory neurons can stay the same since the each hippocampal memory neuron simply index memory across three (or more), rather than two, brain regions (Figure
With multiple inputs to hippocampus [ 9. C.... any subset of those brain areas can reinstate a memory in the other brain regions i.e. E and 9 reinstate memory alone could reinstate ã and č memories. As an analogy to the TEM triple conjunction. TEM-t proposes that č₁ is updated via Co <- softmar(( O (z, ))) where @ is an element wise product. We note an alternate, and perhaps more intuitive, option could be ē. = softmarlā GT + x2 ) this is equivalent to two 'double' conjunctions in TEM, one between 9 and é, the other between ã and č. This is important for TEM as it requires many fewer hippocampal neurons than 'triple' conjunction.
Beyond hippocampus: Cortex as a Transformer. We have considered transformers as model of hippocampus and its connections. We know, however, that transformer representations predict language areas (Schrimpf et al. 2020), and that patients can talk and comprehend just fine with major hippocampal deficits (Elward & Bargha-Khadee TEM-like models, may also model other brain regions, such as language areas, that are seemingly independent from hippocampus (related ideas discussed in Hawkins et al. (2019): Lewis (221) but specifically for grid cells in neocortex). This raises two questions. Firstly what is the analogue of spatial positional encodings for higher order tasks such as language, and secondly what takes the role of the memory neurons if not hippocampus. We offer some thoughts in the following two paragraphs.
2018). This indicates that the transformer, and
In spatial tasks, TEM and TEM-t learn positional encodings that mirror the structure of space. The implication is that positional encoding should reflect the abstract underlying properties the task at hand. In language for example, this structure is grammar. This contrasts to the typical positional en- codings in Transformers sines and cosines which represent linear structure, It is our contention that positional encodings that are inferred on the fly and consist of previously learned structures (like the spatial case we have considered) would offer an interesting and potentially fruitful research direction in problems of language, maths, and logic.
If the transformer were solely instantiated in cortex, then what about the memory neurons? It is possible that the memory neuron equivalent exists in cortex too, but for these tasks, since it is not necessary to store long term memories or bind knowledge across multiple brain areas hippocampus is not required; so short term cortical memory neurons suffice.
8 CONCLUSION
We have shown that transformers with recurrent positional encodings reproduce neural represen- tations found in rodent entorhinal cortex and hippocampus. We then showed these transformers are close mathematical cousins to models of hippocampus that neuroscientists have developed over the last few years. We hope this work brings neuroscience and machine learning closer together, and offers understanding for both sides; for neuroscientists road map to understanding cortical areas beyond the hippocampal formation; for machine learners a greater understanding of positional encodings in transformers.

---

Published as a conference paper at ICLR 2022
REFERENCES
Daniel J. Amit, Hanoch Gutfreund, and H. Sompolinsky. Storing infinite numbers of patterns in spin-glass model of neural networks. Physical Review Letters, 1530531550-, 1985. ISSN 00319007. doi: 10.1 103/PhysRevLeet.55.55
Jimmy Ba, Geoffrey Hinton, Volodymyr Mnih, Joel Z. Leibo, and Catalin Ionescu. Using Fast Weights to Attend to the Recent Past. Advances in Neural Informa-
tion Processing Systems 29, 29:4331-4339, 10 2016a. ISSN 10495258. URL
6057
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer Normalization. arXiv preprint, 2016b. ISSN 1607.06450. doi: 10.1038/maturee14233. URL http://arxiv.ot wwwzxxivvorggapo..
1 607. 06450
Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Pi- otr Mirowski, Alexander Pritzel, Martin J Chadwick, Thomas Degris, Joseph Modayil, Greg Wayne, Hubert Soyer, Fabio Viola, Brian Zhang, Ross Goroshin, Neil Rabinowitz, Razvan Pascanu, Charlie Beattie, Stig Petersen, Amir Sadik, Stephen Gaffney, He- len King, Koray Kavukcuoglu, Demis Hassabis, Raia Hadsell, and Dharshan Kumaran. Vector-based navigation using grid-like representations in artificial agents. Nature, 557 doi: 1180088/-1156-01--000000
(7705):429 433, 5 2018. ISSN 0028-0836.
URL
do1 000.:/000..00000.011100111110110 com/ 2atpa/aaticceeee41118----------0
018 118-11 0102
shttp:
www. nature.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Ncelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. arXiv preprint, 2020.
ISSN 23318422.
Yoram Burak and Ila R. Fiete. Accurate path integration in continuous attractor network models of grid cells. PLoS Computational Biology. 100021, 2009. ISSN 1553734X. doi: 10. usii//ouma...pbi.00000 1000291
URL httpp:: pttp://xxxppp.oop////omoo/ooooom org. http:///w...ll....1111111/11//1/ pcbi.
Christopher
Cueva and Xue-Xin Wei. Emergence of grid-like representations by training recurrent neural networks to perform spatial localization. International Conference on Learning Represen- fations, 0:1-19, 2018. URL http:t//:////xii/
abs 07770
Mete Demircigil, Judith Heusel, Matthias Löwe. Sven Upgang, and Franck Vermet. On a Model of Associative Memory with Huge Storage Capacity, Journal of Statistical Physics, 168(2):288-22)) 2017. ISSN 00224715. doi: 10. 10010777719955---7118886
Yedidyah Dordek, Daniel Soudry, Ron Meir, and Dori Derdikman. Extracting grid cell char- acteristics from place cell inputs using non-negative principal component analysis. elife, (MARCH2016):1336, 2016. ISSN 2050084X. doi: 10. (0.7554/LLifc.10044,
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An Image Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv preprint, pp. 1-21, 2020.
/ abs 201 1929
Rachael L. Elward and Faraneh Vargha-Khadem. Semantic memory in developmental amnesia. me.utt1...
Neuroscience Letters, A30) 2018. ISSN 18727972. doi: 10.1016/J nww.let.2088.00
040. URL htRL:ttps:/ / /wi. wwg. 1016// neulet 2018 04.00..00 040
Dileep George, Rajeev V. Rikhye, Nishad Gothoskar, J. Swaroop Guntupalli, Antoine
Dedieu, and Miguel L.raaroGGredilla.
Clone-structured graph representations enable
10

---

Published as a conference paper at ICLR 2022
flexible learning and vicarious evaluation of cognitive maps. Nature Communicatiins, 12(1):2392, 12 2021. ISSN 2041-1723. doi: 10.
URL
Nicholas Gustafson and Nathaniel D. Daw. Grid Cells, Place Cells, and Geodesic Generalization for Spatial Reinforcement Learning. PLoS Computational Biology, 7(10):e1002235,, 10 2011. ISSN 1553-7358. doi: 10.137 10.1371/jouraalpper peb....iii........ 1002235. URL /tt.tttppe///o..ooooo.oo////////
1371/journal pcbi .102235
Torkel Hafting, Marianne Fyhn, Sturla Molden, May-britt Britt Moser, and Edvard I. Moser. Mi- crostructure of spatial map in the entorhinal cortex. Nature, 436(7052):801-806 2005. ISSN 00280836. doi: 10. N0.1038/naturc0372..
Jeff Hawkins, Marcus Lewis, Mirko Klukas, Scott Purdy, and Subutai Ahmad. A Framework for Intelligence and Cortical Funetion Based on Grid Cells in the Neocortex. Frontiers in Neural
Circuits, 12(Janarry:1-15, 2019. ISSN 1662-5110. doi: 10.3339/fncrr221180002222 URL htttpss//www.nnmentaa..ccoo ottpe/aaaaa//oooaaaaaaaaaaaaaaaa http://ww.iiiaaaaooaaaaaaassaaaa
ort.rhaii...aaaaaeeaaaaaaaaaaaor and-eeeeena---oiiccccee Fun Furctciionn Based- on--on-----
Full
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image
Recognition. Proc. Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 12 2016. URL
J J Hopfield. Neural networks and physical systems with emergent collective computational abili- ties (associative memory/paralle
wtt.ugggrriitiiooooommmaaaaaaaaa addresssabd memory/fail devices). Biophysics, :9(A 1982.
Seyed Mahdi Khaligh-Razavi and Nikolaus Kriegeskorte. Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation. PLoS Computational Biology, 10(11), 2014. ISSN 15537358. doi: 10.137 wbb.uma...pcii.00039911111
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet Classification with Deep Con- volutional Neural Networks. Advances In Neural Information Processing Systems, pp. 1-9, 2012. ISSN 10495258. doi: http://dx.doi http///xddo.org////// wwwp.../////.... wttpg//..001/.jrrr.. 20140,
Dmitry Krotov and John Hopfield. Large Associative Memory Problem in Neurobiology and Ma- chine Learning. arXiv preprint, pp. 1-12, 2020. URL http::::://rxvvvoor////aa www.v.owp//p//////./.
http://axxvv..o
06 06996
Dmitry Krotov and John Hopfield. Dense Associative Memory for Pattern Recognition. Advances in Neural Information Processing Systems, 11N0p)):1800-11888 180-1188, 6 2016. ISSN 10495258. URL
atttp://arxiv: wttg/bbs//iee
01164
Julia Krupic. Neil Burgess, and John O' Oeeeff Neural Representations of Location Composed of Spatially Periodic Bands. Science, 337((096) 8 2012. ISSN 0036-8075. doi: 10. 10.1126/science. 1222403.
URL http: www. sc wwtp::/wcieeccem....omom
6096/ 00/66/853.00 full BUS5///..full.ppttttttt////aaa sci.sccienn http://ttaaaa//....o..o.oooaaaaa science. 1222403
Yann Lecun, Le'on Bottou, Yoshua Bengio, and Parick Haffner. Gradient-based learning applied to document recognition. proc. OF THE IEEE, 1998.
Mareus Lewis. Hippocampal Spatial Mapping As Fast Graph Learning. arXiv preprint, 2021. URL http:://ttpiii////////// org/ abs 2w1. 00567
Xuanqing Liu, Hsiang Fu Yu, Inderjit S. Dhillon, and Cho Jui Hsieh. Learning to encode position for transformer with continuous dynamical model. 37th International Conference on Machine Learning, ICML. 2020, PartF1681 4:6283-6291, 2020.

---

Published as a conference paper at ICLR 2022
Sam McKenzie, Andrea J. Frank, Nathaniel R. Kinsky, Blake Porter, Pamela D Rivie, Pamela D. Rivière, Howard Eichenbaum, Pamela D Rivie, Pamela D. Rivière, Howard Eichenbaum, Pamela D Rivie, Pamela D. Rivière, and Howard Eichenbaum. Hippocampal representation of related and opposing memories develop within distinct, hierarchically organized neural schemas. Neuron, ((1))2202-215, 7 2014. ISSN 10974199. doi: 10.1016/..eeunn.2114.000000
URL
www.eelleeviiee com htt.://c....c...eeeeee//////////
John Keeeeee and J. Dostrovsky. The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Research, 34(1):171-175, 11 1971. ISSN 00068993. doi: 10.1016/0006 9993(7))993388)) /ttt::t:t/ nigub. el sevier. com/ http:/eeerrleeee/i//0/333333333
Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Lukas Gru- ber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, Victor Greiff, David Kreil, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. Hopfield networks is all you need. arXiv preprimt, 2020. ISSN 23318422.
Honi Sanders, Matthew Wilson, Mirko Klukas, Sugandha Sharma, and lla Fiete. Efficient Infer- ence in Structured Spaces. Cell, 183(5):1147-11443 11 2020. ISSN 00928674. doi: 10.1016/j. cell.2020. 11.020.11.0008 11.008. http:::::/....ogg////////
2020.
008https:
Martin Schrimpf, Idan Blank, Greta Tuckute, Carina Kauf, Eghbal Hosseini, Nancy Kanwisher, Joshua Tenenbaum, and Evelina Fedorenko. The neural architecture of language: Integrative reverse-engineerrig converges on model for predictive processing. bioRxiv preprint, 2020. doi: 10.1101/2020.06.26.. 174444
Ben Sorscher, Gabriel C Mel, Surya Ganguli, and Samuel A Ocko. A unified theory for the origin of grid cells through the lens of pattern formation. Advances in Neural Information Processing Systems 32, (20000rP) 2019.
Kimberley L. Kimberly L. Kimberley L. Kimberly L. Stachenfeld, Matthew M. Botvinick, and Samuel J. Gershman. The hippocampus as a predictive map. Nature Neuroscience, 20(1 (((11):166 14:1643- 1653, 2017. ISSN 15461726. doi: 10. 10388/n..650..
Timothy Teyler and Jerry W Rudy. The hippocampal indexing theory and episodic mem- ory: Updating the index. Hippocampus, 17(12):1158- 1169, 12 2007. ISSN 10509631. doi: URL /tttps://onlieelibrr wwtp:aaarrw..l..l.... wwwew. www. wttp:eey.oooooo//.oooooo. /ot//0.i1011111 1002/
10.1002/hipo.20350. /doi. wi www.ww.w........................ wtt..ccmm11..000///1pp000 www.//pp............. 20350
hi
Benigno Uria, Borja Ibarz, Andrea Banino, Vinicius Zambaldi, Dharshan Kumaran, Demis Hass- abis, Caswell Barry, and Charles Blundell. The spatial memory pipeline: A model of egocen- tric to allocentric understanding in mammalian brains. bioRxiv preprint, pp. 1 1-52, 2020. ISSN 26928205, doi: 10. 10.1101/2020. 1..20......... 1..1..37844.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Informa- fion Processing Systems, 2017 (07TDDeem(Nipss:599990000
2017. ISSN 10495258.
Zhiwei Wang, Yao Ma, Zitao Liu, and Jiliang Tang. R- R-Transformer: Recurrent Neural Network En- hanced Transformer. arXiv preprini, 0:1-11, 2019. HRRLhttp:// org/ abs /1007.
055722
James CR Whittington, Timothy H. Muller, Shirley Mark, Caswell Barry, Neil Burgess, Timothy E.J. EJ Behrens, Guifen Chen, Caswell Barry, Neil Burgess, and Timothy E.J. EJ Behrens. The Folman-Fichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation. Cell, 1249 11 2020. ISSN 00928674. doi: 10.1016/
J.cel.2020.10.022 /Rt.htpps///o.i.oo//.00..0.
ce 20. 12..0......222thtee 024https: el sevier....... www.:ieee.....rreeeeeeeeeeee pttps//er..oom/rerrreeeee/////// ottpaa/eeeeppr11.oo0202002211131
Daniel L K Yamins, Ha Hong. Charles F. Cadieu, Ethan A. Solomon, Darren Seibert, and James J.
DiCarlo. Performance-optimize hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences of the United States of America, 111
(23):8619-8624, 2014. ISSN 10916490. doi: 10. 10.1073/pnas. 1403| 11111,
12

---

Published as a conference paper at ICLR 2022
A APPENDIX
A.1 THE MATHS USING ELEMENTS
Here we derive the main results using vector and matrix clements. Since each element in p is multiplicative combination of every pair of elements in æ and g, then
Pij 9
(12)
Where we label the elements of vector p with two indices even though it is vector. Similarly, the memory matrix uses 4 indices;
Miskt = Eanjon
(13)
Retrieving a memory via path integration. Querying the network becomes 4τ = g.' which reduces to
Σ 15h
Mishl
= ΣΣ
9
9
Σ 27
9k
PI
9 Σ kj)
(14)
PI 0.
X
A
Retrieving memory using a sensory observation alone. Similarly when the query is the sensory observation alone 9i = Izj
Σ E2
nú Mijki
Σ
85tr
Σ कe
Σ
3
o4
(15)
AG
Retrieving a memory using a sensory observation and path integration. When the query is o 2j, and we want to retrieve a position encoding memory 4s of
Σ Eij
Σ
Σ
gh Σ
3h P1 ) [
(16)
X
x
gt
PI TT
G
A.2 DETAILS IMPLEMENTATION
Code
will
be made available
at
https:://ithubb.cmm
Scaling beta. Since the number of memories is not constant, we use an adaptive / parameter in the softmax. This because normalisation term the softmax sums the number of memories, and so more memories down-weights probabilities. We want self-attention value to not be affected by the number elements in the set. In particular, we weight the softmax by log mmmmories)
Losses. We same set of losses in TEM:
a one-step sensory prediction cross entropy loss, i.c. using g," PT as imput to the memory retrieval process
13

---

Published as a conference paper at ICLR 2022
sensory prediction cross entropy loss using I: input to the memory retrieval process
a squared eтrоr loss between 9, and g.!!
12 weight regularisation
12 regularisation on gr
Normalisation. We find that using layernorm (Ba et 2016b) on the positional encodings (not in the RNN, but on the input to transformer) to be beneficial, since the memory retrieval process can then be standardised no one memory is up-weighted relative others. Using layernorm before self-attention in transformers is common practice (Vaswani et al.) (017). For simplicity, we use fixed weights on the layer norm, i.e. just z score of g, Since we use a one-hot encodings of our sensory representations, they are already normalised.
Stabilising position representations. Recurrently generated positional encodings accumulate noise and drift. While bespoke path integration networks from neuroscience mitigate noise by enforcing their neural representations to stay close to a neural manifold (Burak & Fiete, 2009), this can not be guaranteed in learned recurrent networks. One method of stabilisation is via sensory landmarks - i.e. 'what positional encoding did have the last time I saw this landmark'. In this vein TEM uses the following query to the memory network; qt flatten (ā) 1), and memories of positional encodings are retrieved as;
gretriered = sum(unflattens (q, 0) = Σ g. 9. [+ ãrl ã. x. AAAA, g:
(17)
The final position encoding, g is computed on a basis of path integration (g." equation 3 and stored landmark information etreevvedd equation 17 i.e. gt p1 + retrieeeedd P! ") g where g retriccedd = fo fg (gratrieved) and fe( are different MLPs.
retrieved
We note that better query the memory network would be q: flatten( g. "f²²) since sensory observations may be aliased, including 96" can help disambiguate such aliasing. In this case, the retrieved memory is (ā, &g,"" A, This is perhaps. different to what would be anticipated; using an element wise product rather than an addition. Translating this other TEM memory retrieval process into transformers we get g retrieved So (sof fommarrrre (ã, X " this can be thought of as another attention head i.e. from input [g. æ²) the key and query 'attends' to while the value 'attends" to g.
When to add memories. Memories should not be added at every step, otherwise the self-attention mechanism would be biased towards memories (locations) that have been visited more frequently. This is not problem typically addressed in transformer, but here to mitigate this issue memories are only added when existing memories for that particular conjunction do not already exist i.e. if there is memory already with the present combination of and (similarity determined by dot product) then no new memory added]
Though in our simulations we only used a single iteration of the transformer block, it is possible to use multiple iterations - just like Hopfield network (as in Ramsauer et al. 2020 Here the retrieved sensory memory क aretrieved can be fed into the next iteration, along with positional encoding i.e. the path integrated g. In this case (see Appendix for derivation), TEM suggests that ã, is iteratively updated via o; + softmaz( X², 3 iteration via -. t sof coftmar( G). X. The multiplicative term in the softmax is perhaps unexpected it may be thought that should be additive instead.
Multiple iterations:
G) but with the first
Place-like representations without softmax: Here we chose softmax activation function on the memory neurons to make the relationship between TEM and transformers exact. However was this choice necessary to observe place cells in the memory neurons? While we have not examined this experimentally, we suspect piace-like representations would emerge for a variety of activations, since memories must still be sparsely activated for correct prediction. We leave this for future work to verify, but note again that power and exponential activations have been shown to have a
'The TEM model actually does something very much like this memories are only added if they did not already exist M = E, (p- - (pp) Pr), where Pr is the memory that was retrieved at i.e. only the un-predicted components of the memory p- are added.
14

---

Published as a conference paper at ICLR 2022
Hidden structure
Passive actions
Sensory observations
I &. ?
Trials
Environments
Figure 6: Learning to predict the next sensory observation in environments that share the same structure but differ in their sensory observations. TEM only sees the sensory observations and associated action taken, it is not told about the underlying structure - this must be learned.
much greater memory capacity (Krotov & Hopfield 2016 Demircigil et al. 2017) than traditional Hopfield nets with linear activation.
A.3 ADDITIONAL DETAILS OF THE SPATIAL TASK
We formalise a task type that not only relates known hippocampal function, but also tests the learning and generalising of abstract structural knowledge. We formalise this via relational under- standing on graph structures (a graph is a set of nodes that relate to each other).
Should one passively move a graph (e.g. Figure 6). where each node is associated with non- unique sensory observation (e.g- an image of a banana), then predicting the subsequent sensory observation tests whether you understand the graph structure you in. For example, if you return to previously visited node 6 by new direction it is only possible to predict correctly if you know that right down left up of such loop closures equivalent to understanding the structure of the graph.
you'l back in the same place. Knowledge
We thus train models these graphs with it trying predict the next sensory observation. Our models are trained on many environments sharing the same structure, e.g. 2D graphs (Figure (). however the stimulus distribution is different (each vertex is randomly assigned stimulus). Should it be able to learn and generalise this structural knowledge. then it should be able to enter new environments (structurally similar but with different stimulus distributions) and perform feats of loop closure on first presentation.
Formally, given data of the form D (( atrr)) with (1,...,AV) (which environment is in). where ø<r and acr are sequence of sensory observations and associated actions/relations (Figure 6), N is the number of environments in the dataset, and T is the duration of time-steps in each environment, our model should maximise its probability of observing the sensory observations for each environment.
The sensory stimuli are chosen randomly, with replacement, at each node. We understand that this not like the real world, where adjacent locations have sensory correlations most notable in space (though names in family tree will be less correlated). Sensory correlations help with sensory predictions, thus if we use environments with sensory correlations, we would not know what was causing the leamned representations, sensory correlations, transition structure. To answer this question cleanly, and know that transition structure is the sole cause, we not use environments with sensory correlations.
15

---

Published as a conference paper at ICLR 2022
A.4 ADDITIONAL DETAILS OF TEM
We present more detailed model schematic of TEM in Figure 7 We see there are two components to TEM a RNN for understanding position (g. in green top of Figure 7 that also indexes memories via "queries" q yg A memory network that binds together and g. via an outer product (middle green in 7 with more detail in Figure AA). When the memory network is queried (red in Figure 7, it undergoes attractor dynamics to retrieve the full memory. To make sensory prediction, the retrieved memory is "deconjunctified" into a sensory representation (Figure 8C).
Model flow. TEM transitions through time and infers 9. and Pt each time-step. 9: is inferred before forming each new memory Pt- In other words variables and are inferred in the following order gt. Pts 1, Pr+1, 9++2 .... This flow of information is shown in a schematic in Figure 7
Independently, at each time-step, the model model asks 'are the inferred variables what would have predicted given my current understanding of the world (weights)'. L.c. 1) Is the inferred g: the one I would have predicted from 1t-1. 2) Is the inferred P: the one I would have predicted from 9t. 3) Is x, what would have predicted from pt. This leads errors (at each timestep) between inferred and predicted variables Jr and pa. and between sensory data 2; and its prediction.
At then end of a sequence, these errors are accumulated, with model parameters updated along the gradient (from back paooppropagttion through time) that matches each others variables and also matches the data.
Since the model runs along uninterrupted, it's activity at one time-step influence those at later time- steps. Thus when learning (using back-propagaan through time BPTT), gradient information flows backwards in time. This is important as, should bad memory be formed at one-time step, it will have consequences for later predictions thus BPTT allows us to learn how to form memories and latent representations such that they will be useful many steps into the future.
A.5 RELATED WORK
Here we discuss and compare other models that recapitulate spatial representations found in the brain. These models fall into several categories. 1) Auto-encoder like models trained on spatial representations (Dordek et al., 2016). 2) Graph representation models (Gustafson & Daw [201T] Stachenfeld et al.) ad.[(017). 3) Latent state inference models (George et al.. (021). 4) Path integrating models with RNNs trained on spatial representations (Cueva & Wei 2018 Banino et al., 2018). 5) Models mixing RNNs and memory networks that are trained on sequences of sensory observations.
We note that models (1-4) are learned using curated spatial representation, i.e. the modeller has already worked how how space works, and the model is finding an alternate representation of that space. This is not how you or learn. Instead, we learn by extracting regularities from the sensory world, and transfer/generralis this knowledge from domain to domain. Model category (5) does unsupervised learning by predicting sequences of sensory observations alone. From just sensory observations, it can slowly build up a picture of how space is structured, and then transfer this knowledge to situations that share the same underlying spatial structure. Our model TEM-t fall in the model (5) category, along with TEM (Whittinton et al. 2020) and other similar models Uria al. (2020).
For model to make sensory predictions as fast as possible, it is necessary to have two components. a) A RNN that understand position, and b) a memory network that can remember what you see and where you see them. Now the model can be asked what 'what will you see when heading East' and correctly respond Cat even it has never gone East at that location before. This is because it can simulate going East via its RNN and then retrieve the memory it had stored at that location (it must have visited the Cat location before thought!). The other models, e.g. an autoencoder, could not solve this task since it learns slow statistical structures between sensory observations over many many training examples. Thus would be clueless to the entirely new configuration of sensory observations in the new environment. In sum, is necessary to have understanding of position, and the ability to make and retrieve memories to successfully make sensory predictions as fast as possible.
We describe the relationship between TEM (and thus the hippocampallenttrhiia. system) as close not because it produces the same representations, instead saying the relationship is close
16

---

Published as a conference paper at ICLR 2022
Transition with weight matrix
+ action at-1
gt
Path integrate new g
Wa
Wa
qt
Wa
Query memory network, and retrieve memory via attractor dynamics
M
Form memory p from and via outer product, then change weights M
Pt
Wx
Predict sensory from p
Sensory data
Step =t
Step = t+1
Figure 7: Schematic show the model flow. Depiction of TEM two time-points, with each time-point described at different level of detail. Timepoint / shows network implementation, + describes each computation in words. Red is for model predictions, green is for updating model variables. We do not show the stabilising position encodings module here. Circles depict neurons (blue is g. red is æ, blue/red p); shaded boxes depict computation steps; arrows show Iearnable weights; looped arrows describe recurrent attractor. Black lines between neurons in attractor de- scribe Hebbian weights M. W. are learnable, action dependent, transition weights. W, and Wz learnable projection matrices. Yellow arrows show training errors.
because we have shown a mathematical relationship between the two models. This could not happen for most ML models most ML systems do not have both RNN structured representations and a memory system and so is not mathematically relatable to current models of the hippocampal say that the memory
formation. However, equipped with the mathematical relationship, part of the transformer is related to the hippocampal memory system in TEM. And the positional encodings are related to the entorhinal grid RNN system in TEM.
A.6 FURTHER LEARNED CELL REPRESENTATIONS
17

---

Published as a conference paper at ICLR 2022
A
Conjunctivo code
B
2
Colours
concoretion The value net oi each
Nave
Ol morines My.
M,
p Battten
D
rishapeia, Mi-In.
Qvery q- flatteed ag)
Retrieved M,
Marroiee Mi
Σ TOSS
X constant
P.S. pls
sindlantt woig
Oj M; 1/1R
Boop Fattrerse rebwcrkk consengee
Innel product
Figure 8: Memory formation and retrieval in TEM. (a-b) Memory formation. (a) Projected sensory sensory code & and projected grid code Đ are combined via an outer-produce 57 g. which flattened obtain vector of place cells p. Each place cell (denoted by single diagonally divided cell) is a conjunction of an element from each of æ and ğ (denoted by the two colours composing each cell). The activity of the place cell is the product of the values of these elements. (b) new Hebbian memory p" is added to the existing memory matrix M. (c-d) Memory retrieval. (c) Multiplication of the query q with the memory matrix M retrieves place code p. This retrieved code is the sum of previously experienced codes, weighted by their similarity to the present query. This may be repeated iteratively to converge the stored p that is most similar to q. (d) The retrieved place code is reshaped and summed along the rows average-out the g components. The result is gē.
18

---

Published as a conference paper at ICLR 2022
gnd score disoibution
grid soore distibution
0
O 0
O
Figure 9: Learned grid cells ordered by grid score. We only show cells that are both active and have a grid score above 0. A gird score of 0.3-0.5 is generally considered to be grid cell. The panels on the right hand side are the show the grid scores for the whole population of cells (though in some cases the grid score was not calculable e.g. the cell has no activity; these cells are ignored in the analysis). (a) Grid cells learned with a linear activation function. (a) Grid cells learned with ReLu activation function.
distribution
Figure 10: Learned grid cells in hexagonal 6-connected world.
19

---

Published as a conference paper at ICLR 2022
A
matric distribution
p ace cells
grid cells
Figure 11: Place cells ordered by novel place cell metric. This metric assess how place-like the firing of each cell is. In particular, metric is the ratio of 'firing mass" in the largest connected component versus all connected compo- nents. This metric is if all the firing in single component, and it lower if the firing is spread between components, (a) TEM-t learned memory place cells. (b) Our novel metric distinguishes between TEM-t RNN neurons (grid cells), and TEM-t memory neurons (place cells).
look the connected components of the firing rate map, and
20


Reviews:
[{"review_id": "U23vd1brxlO", "reviewer": "Reviewer_K1VL", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Major issues:\n- Many papers have reported finding learned representations that resemble place cells and some have found grid cells as well. It would be quite helpful to contextualize the findings in this manuscript with others, as well as more rigorously evaluate the degree of evidence for these types of response properties, by: (1) defining a metric of place/grid cell-ness, and then quantifying the extent to which a given model unit resembles a place cell or grid cell, and then (2) quantifying how common these representations are across all model units, rather than simply showing one or two example units (and ideally comparing these numbers to estimates of the prevalence of such units in the relevant [biological] neural populations). And then (3) showing the top K units that score on this rather than one or two (ideally k=50 or 100 and then is just a big appendix figure). If the authors also wish to discuss band cells then they should do this for that type as well. Lastly, an obvious gap is that the authors don't mention boundary cells -- do they find any units that mimic those?\n\n- I found myself wanting a full end-to-end diagram of both models, but particularly the TEM, as I am (and I imagine the ICLR audience more broadly is) less familiar with it. Obviously, space is tight, but including a full schematic of each model would be  particularly helpful. If need be, this could be in the appendix.\n\n- Further clarity on what the task exactly is. The authors briefly discuss it briefly in a few paragraphs, but the manuscript would benefit from more details on the exact nature of the task -- e.g., example diagrams of the environments, and perhaps even examples of performance over training. Since space is tight, this could just be in the appendix.\n\n\nMinor issues:\n- What do the authors mean that TEM-t both has better sample efficiency and has reduced training time? Naively, one would imagine these things are largely equivalent statements, but I assume they mean something more here.\n- Typos: \n  - Pg 5: \"via via Hebbian learning\" --> \"via Hebbian learning\"\n  - Pg 6: \"TEM can tackle much larger problems\" --> \"TEM-t can tackle...\"\n  - Pg 7: \"relation ship\" --> \"relationship\"\n", "summary_of_the_paper": "This well-written and clear paper clarifies the relationship between transformers and a recent exciting model of the medial temporal lobe in neuroscience. The authors find that a transformer with recurrent positional encodings trained on a spatial navigation task ends up learning representations that resemble some classic findings from the medial temporal lobe in the neuroscience (e.g., grid and place cells). This finding alone is not that surprising -- as the authors note, a variety of groups have previously found such representations are learned in navigation tasks with other models (particularly place cells). The major contribution of this paper is that the authors then demonstrate the connection between transformers and a contemporary model of the hippocampal formation, the Tolman-Eichenbaum Machine (TEM). Specifically, the memory retrieval process in the TEM resembles self-attention and the path integration representations are comparable to a transformer's learned position encodings. I believe that by offering a novel perspective on transformers and models of biological memory systems, this work may lead to fruitful future work both in machine learning and neuroscience.\n", "summary_of_the_review": "By offering a novel perspective on transformers and models of biological memory systems, this work may lead to fruitful future work both in machine learning and neuroscience.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ShJwEOl6xd8", "reviewer": "Reviewer_hKkg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In general I am very curious and supportive of the effort to find similarities between the methods and representations that biological and artificial learning systems discover during their learning process. However we must be very strict with ourselves when we declare that such a similarity has been found:\n  * (a) Is it sufficiently similar to be non-trivial?\n  * (b) Can the emergence of this similarity be explained by other factors, such as experimental setup?\n  * (c) Does the similarity occur only when we are using one particular ML algorithm or would it also emerge when attempted using other learning methods / algorithms / architectures that are appropriately configured?\n\nThe emergent patterns resembling grid cell structure that are shown to emerge in this paper closely resemble the patterns that emerged in an RL agent in this work: https://deepmind.com/blog/article/grid-cells. In that work Transformers were not used, but the pattern has emerged nevertheless. This begs the question whether Transformers are indeed *the* architecture that encodes spatial representation closest to hippocampus, or whether any ML model, properly incentivized, will form such representations?\n\nAuthors mention in the abstract that \"this result is no surprise since it is closely related to current hippocampal models from neuroscience\" about the claim \"transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation\". Could we have written the same paper by just replacing Transformers with something else that could mimic neuroscientific model of hippocampal grid cells? If yes, then Transformers have no special significance here and should not enter the discussion.\n\nI am looking forward to the discussion and am very to open to changing my rating upon further exploration of these matters.\n\n\nMajor concerns / criticisms / discussion\n----------------------------------------\n\n* Seeing how Transformers are very flexible models with high capacity, wouldn't it be the case that (with a little twist) one could fit them to become good representations of almost any phenomena? The theory that is put forward in this paper is that Transformers model representations that are close to those used by hippocampal formation. What kind of result would invalidate such a theory? Let's say that instead of Transformers we would use a black box method \"X\" that would tune its parameters in such a way, that they correlate well with hippocampal responses. Would that mean that \"X\" is good model of representations that are being used by hippocampus? Note that the more flexible \"X\" is and the more time and computational resources we have, the closer we can bring \"X\"'s activations to correlate with hippocampal responses. What would be a hypothetical empirical indication from your experiments that would lead you to conclude that \"nah, Transformers are nothing like hippocampus\"?\n\n* Our brain did not learn in an environment with a 4-connected Eucledian structure. Wouldn't having this structure most assuredly lead to a representation that reflects the pattern of the structure? Let's say you would conduct the same experiment in an environment with a some kind of 10-connected star-shaped Eucledian structure -- wouldn't the patterns of representations formed by the Transformer would try to capture that star-shape?\n\n\nQuestions\n---------\n\n* page 3: \"It now becomes interesting to see what representations are learned\" -- taking into account the properties of the task and its spatial structure, what would we expect the representation that captures that structure to look like? Does it have to be a Transformer to capture that representation or could be captured, for example, by an autoencoder trained to find an efficient representation of the samples constituting the task?\n\n* page 4: \"Hopfield network (Hopfield, 1982) which have recently been shown to be closely related to transformers\" -- Could you please elaborate what is meant by being related and how this relationship pertains to the claim that \"a transformer with recurrent positional encodings is closely related to current neuroscience models of the hippocampus and surrounding cortex\"?\n\n\n\nMinor remarks\n-------------\npage 2: typo \"Crucially, such rules generalises\"", "summary_of_the_paper": "The paper postulates a task that requires the ML model to capture the spatial nature of the task in order to perform well. The chosen ML method is a Transformer with positional encoder, where the encoding is not fixed, but is learnable, making it possible for the transformer to \"choose\" what kind of representation would it \"prefer\" for the task at hand. The Transformer is modified to act comparably to a neuroscientific model of hippocampal function (TEM). After the learning process the authors discover, that, when visualized, the representation that has emerged resembles the grid-cell pattern similar to the one that is being encoded by hippocampal place and grid cells. From there the works suggests the existence of a special relationship between the Transformer architecture and the mechanism by which hippocampus encodes spatial information.", "summary_of_the_review": "In my current evaluation (prior to discussion with the authors) it seems to me that the claim that Transformers have a special kind of relation to hippocampal grid cells is not supported by the evidence. The evidence does show that the condition is \"sufficient\" (Transformers do form grid-like representations), but it does not show that it is also \"necessary\" (meaning it does not have to be a Transformer to form a grid-like representation). If my understanding of the evidence is correct, then it does not support the claim of the paper and I would recommend rejecting this result.In my current evaluation (prior to discussion with the authors) it seems to me that the claim that Transformers have a special kind of relation to hippocampal grid cells is not supported by the evidence. The evidence does show that the condition is \"sufficient\" (Transformers do form grid-like representations), but it does not show that it is also \"necessary\" (meaning it does not have to be a Transformer to form a grid-like representation). If my understanding of the evidence is correct, then it does not support the claim of the paper and I would recommend rejecting this result.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "Oy7_Z_baZHA", "reviewer": "Reviewer_skR7", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*Strengths*\n\nThe authors show the full analogy between the Transformer architecture and the TEMs – unlike the existing literature which has only identified some similarities between the two (e.g. the similarity between the attractor update rule in TEM and the matrix product in self-attention).\n\nThe Transformer architecture proposed in this paper inherits the biological plausibility of TEMs and, at the same time, is considerably more sample-efficient, allowing for building computationally efficient models of the hippocampus and entorhinal cortex.\n\nThe proposed architecture is expected to scale well with an increase in the number of cortical inputs to the model thus potentially enabling the studies of multisensory integration.\n\nThe paper has a nice introduction to the Transformer architecture.\n\n*Weaknesses*\n\nWhereas the authors, due to their results, expect the Transformer architecture to inform the future models of the hippocampus – and the other way around – such work has not been done in the scope of the current paper.\n\nI think it would be nice to include a slightly more detailed description of the TEM model in the paper. I like the equation-based description, offered here, better than the text-based one in the original TEM paper; however, this approach would benefit from defining the variables and, perhaps, briefly commenting on their role before using them (e.g. the q-variable)\n", "summary_of_the_paper": "The authors show that the Transformer architecture, with a specific choice of parameters, is analogous to the Tolman-Eichenbaum Machine (TEM), a neuroscientific model successful at explaining phenomena related to grid cells in the entorhinal cortex, place cells in the hippocampus, and their coordinated remapping in new environments. The authors show that, compared to TEM, its Transformer counterpart is more sample-efficient and should better generalize to multisensory inputs, thus offering a computationally efficient basis for models of the hippocampus and entorhinal cortex.", "summary_of_the_review": "The paper proves a parallel between the Transformers and the Tolman-Eichenbaum Machines, thus offering a computationally efficient and scalable implementation for this biologically-grounded model of the hippocampus. Even though no new biological insights were obtained in the scope of this paper using that knowledge, the parallel between the two frameworks deems useful for models of the hippocampus in follow-up works and highly relevant to the community. I, therefore, recommend accepting the paper. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "A_gOpDK994b", "reviewer": "Reviewer_RQx3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper nicely connects several ideas discussed recently in the AI and comp. neuroscience communities, such as transformers, Hopfield Networks, memory systems, place cells in the hippocampus, grid cells, etc. I want to give a couple of pointers to the literature that might make these connections more transparent and mathematically more precise. \n\n1. The left equation in (Eq 11) seems to be closer related to the class of models that have power activation function (see for example equation 10 in Dense Associative Memory for Pattern Recognition by Krotov & Hopfield 2016 https://arxiv.org/abs/1606.01164, and assume that f is linear, or more generally a power function) than to models with f=softmax. In the classification scheme of Krotov & Hopfield 2020, the former is a class of models that they call models A, while the latter is the class of models that they call models B. The same applies to (Eq 8). \n\n2. The network motif with two kinds of feature neurons, similar to the one shown in Fig.5 B is commonly used in models of associative memory in the hetero-associative setting. When there are two kinds of features that need to be associated with each other in one memory. For instance, this idea was used in Krotov & Hopfield 2016 for associating pixel intensities of images with the labels of those images, see for example Figure 1A. The pixel intensities are presented to the network and hold clamped, while the label part of the feature neurons are updated to compute the desired output. \n\n3. The sentence “It has been shown that the use of a softmax corresponds to a variant of the Hopfield energy (Demircigil et al., 2017) where the number of memories is untethered from dimensionality of the attractor and therefore are potentially unbounded.” on page 8 is not entirely correct. First, the model by Demircigil and coauthors has a bounded capacity, which is at most $2^{(N/2)}$, see their theorem 1.3 in https://arxiv.org/abs/1702.01929. Second, the activation function in that model is f(x) = exp(x), rather than the softmax. I understand that the confusion comes from the derivation given in Ramsauer et al., who start with the model with f(x)=exp(x), then take a logarithm of the energy function for that model, and add an extra term to enforce the lower bound on the energy. The problem with this logic is that the addition of the quadratic term changes the mathematical properties of the model. For this reason, the model with the softmax activation, and the model with f=exp(x) are two mathematically different models.   \n\nI also wish the paper had some kind of quantitative metrics of how closely the learned representations resemble place cells. Right now the similarity is only qualitative, e.g. Figure 5D. If the evaluation was somewhat more quantitative the authors might be able to ask interesting questions. For example how does the choice of the activation function (softmax, power, exponent) change the similarity of the learned representations to biological place cells? Maybe some activation functions model the biological system better than others?   \n", "summary_of_the_paper": "This paper demonstrates emergence of grid and place cells in transformer architectures with positional encodings. Transformers are related to TEM models and Hopfield networks in computational neuroscience. A neurobiological model of TEM-transformers is presented, which contains feature neurons and memory neurons. It is shown that memory neurons resemble place cells in the hippocampus. Remapping phenomenon is also discussed. ", "summary_of_the_review": "In general, I like the scope of the paper, and I am inclined to vote accept provided that the authors can address the comments above.  ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper received 4 unanimous accept (including 1 marginal accept). This well-written and clear paper clarifies the relationship between transformers and a recent exciting model of the medial temporal lobe in neuroscience. There was some clarifications requested by the reviewers that were addressed during the revision. This paper will make a great computational neuroscience contribution to this year ICLR!
Decision: Accept (Poster)