Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells
Authors: Rylan Schaeffer
Mikail Khona
Tzuhsuan Ma
Cristobal Eyzaguirre
Sanmi Koyejo
Ila R Fiete
Venue: NeurIPS 2023 poster (2023)
Keywords: self-supervised learning
unsupervised learning
grid cells
neuroscience
systems neuroscience
representation learning
TL;DR: SSL learning representations of space yields multi-modular grid cells

Abstract: To solve the spatial problems of mapping, localization and navigation, the mammalian lineage has developed striking spatial representations. One important spatial representation is the Nobel-prize winning grid cells: neurons that represent self-location, a local and aperiodic quantity, with seemingly bizarre non-local and spatially periodic activity patterns of a few discrete periods. Why has the mammalian lineage learnt this peculiar grid representation? Mathematical analysis suggests that this multi-periodic representation has excellent properties as an algebraic code with high capacity and intrinsic error-correction, but to date, synthesis of multi-modular grid cells in deep recurrent neural networks remains absent. In this work, we begin by identifying key insights from four families of approaches to answering the grid cell question: dynamical systems, coding theory, function optimization and supervised deep learning. We then leverage our insights to propose a new approach that elegantly combines the strengths of all four approaches. Our approach is a self-supervised learning (SSL) framework - including data, data augmentations, loss functions and a network architecture - motivated from a normative perspective, with no access to supervised position information. Without making assumptions about internal or readout representations, we show that multiple grid cell modules can emerge in networks trained on our SSL framework and that the networks generalize significantly beyond their training distribution. This work contains insights for neuroscientists interested in the origins of grid cells as well as machine learning researchers interested in novel SSL frameworks.

Spatial information fundamental animal survival, particularly in animals that return to home base. Mammalian brains exhibit menagerie of specialized representations for remembering and Self-Supervised Learning of Representations for Space Generates Multi-Modularr Grid Cells
Rylan Schaeffer" Computer Science
Mikail Khona"
Physics MIT mikail@mit.eem
Stanford University
rschaeffcs www.ssst.nnoeeeeoommmom
Tzuhsuan Ma
Cristóbal Eyzaguirre
Janelia Research Campus Howard Hughes Medical Institute mat @ anelia. hhmi org
Computer Science
Stanford University
coyzaguirre@stanfordccom
Sanmi Koyejo
Ila Rani Fiete
Computer Science
Brain and Cognitive Sciences
Stanford University
MIT fietelmit.edi
sanmi @cs. stanford.edd edu
Abstract
To solve the spatial problems mapping, localization and navigation, the mam- malian lineage has developed striking spatial representations. One important spatial representation the Nobel-prize winning grid cells: neurons that represent self- location, local and aperiodic quantity, with seemingly bizarre non-local and spatially periodic activity patterns of a few discrete periods. Why has the mam- malian lineage learnt this peculiar grid representation? Mathematical analysis suggests that this multi-periodic representation has excellent properties as an alge- braic code with high capacity and intrinsic error-correction, but to date, synthesis of multi-modula grid cells in deep recurrent neural networks remains absent. this work, we begin by identifying key insights from four families of approaches answering the grid cell question: dynamical systems, coding theory, function opti- mization and supervised deep learning. We then leverage our insights propose new approach that elegantly combines the strengths of all four approaches. Our approach is self-supervised learning (SSL) framework including data, data aug- mentations, loss functions and a network architecture motivated from a normative perspective, with access to supervised position information. Without making assumptions about internal or readout representations, we show that multiple grid cell modules emerge in networks trained networks generalize significantly beyond their training distribution. This work contains insights for neuroscientists interested in the origins of grid cells as well machine learning researchers interested in novel SSL frameworks.
SSL framework and that the
Introduction
Denotes equal contribution and co-first authorship.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

a
b
firing fields
C
multi-modular representation
*
rat's path inside box
Ord souding Jom
uniform phase coverage
places where cell fired
Figure 1: Grid Cells in the Mammalian Medial Entorhinal Cortex. (a) A grid cell fires at the vertices regular triangular lattice space, regardless of direction or speed. (b) Examples of two grid cells' firing fields (top) and spatial autocorrelations (bottom). (c) The grid code is multimodular: across the population, grid cells cluster into discrete modules such that all cells within a module share the same spatial periodicity and orientation, and only a small number of spatial periods are expressed. (d) Grid cells within one grid module uniformly cover all phases. Figs b-d from 26 59]
reasoning about space. Of these, grid cells (whose discovery earned a Nobel prize) are a striking and important example. Each grid cell fires in a spatially periodic pattern, at every vertex of regular triangular lattice that tiles all two-dimensional environments (26) (Fig. [bb) This neural code for space appears bizarre since position is local and aperiodic variable, but the neural representations are non-local and periodic. Why has the manmalian brain learnt this peculiar representation?
Broadly, there have been two directions to answering this question. The first direction uses theoretical analysis to deduce the mathematical properties that the grid code possesses [2 56 64 40 [58) This
direction yields many insights about the coding properties of grid cells, but has not validated these insights by proposing an optimization problem under which grid cells emerge in deep recurrent neural
networks. The second direction trains deep recurrent neural networks (RNNs) [66 5 [54] with the goal of identifying the optimization problem that grid cells solve. However, recent work 48 51 showed that these models require highly specifie, hand-engineered and non-biological readout representations to produce grid cells; removing these representations results in loss of grid cells even though the optimization problem is equally well-solved.
In this paper, we make three contributions. First, we extract critical insights from prior models of the grid code spanning diverse fields including coding theory, dynamical systems, function optimization and supervised deep learning. Second, we use those insights to propose a new self-supervised learning (SSL) framework for spatial representations, including data, data augmentations, loss functions and network architecture. Third, we show that multi- modular grid cells can emerge as optimal solutions to SSL framework, with no hand-engineered inputs, internals or outputs. We also perform ablations to show under what constraints grid cells do and do not emerge.
By creating a minimal, for path-integrating deep recurrent networks through SSL, we close the loop from the discovery of biological grid cells, to the study of their theoretical properties, to their artificial genesis.
motivatee and developmentally plausible learning problem
2
Background
Grid cells in the mammalian medial entorhinal cortex Grid cells [26] are neurons found in the mammalian medial entorhinal cortex tuned represent the animal's spatial location as it traverses 2D space. Each grid cell fires if and only if the animal is spatially located at any vertex of a regular triangular lattice that tiles the explored space, regardless of the speed and direction of movement through space (Fig. [bb). As population, grid cells exhibit several striking properties that provide support for specialized and modular circuit: Grid cells form discrete modules (clusters) such that all cells within a module share the same spatial periodicity and orientation, while different modules express a small number of discretely different spatial periods (Fig. (T)) [59]. Grid cells in the same module (i.c., sharing the same lattice periodicity) exist for all phase shifts [68] (Fig. ld).

---

Self-Supervised Learning (SSL) and its applications in Neuroscience SSL is subset of unsuper-
vised learning in which data is used to construct its own learning target [2] 27 61 T0 14 69
[9]. Broadly, SSL methods belong to one of three families: deep metric learning, self-distillation or canonical correlations analysis; we refer the reader to a recent review 4 SSL. is increasingly gaining popularity as normative approach in neuroscience since SSL mitigates the need for large-scaled su- pervised data that biological agents do not possess. SSL has been used in vision to explain the neural representations of ventral visual stream [32 770,, dorsal visual stream [41] and an explanation for the emergence of both []. In this paper, we provide the first model of SSL applied to the mammalian brain's 's spatial representations with an emphasis sequential data and recurrent networks that the best of our knowledge is not well explored in either the machine learning or neuroscience literature.
3 Notation and Terminology
We consider temporal sequences of three variables: spatial positions Se R2, spatial velocities V E I22 and neural representations on the surface of the positive orthant of the N N-dimensional
sphere g: E s del G. Note that any neural representation g E & has unit norm ||gl|2 = 1. We interchangeably refer representations as states, codes or embeddings. We denote sequences of any of these variables from time i = to T as (-1, '2,..., 'T), e.g., (V), Ur).
4 Insights from Grid Cell Modeling Approaches
The question of why the mammalian brain uses a high-iighhheeee disensiimsii,,,, multiperiodic code to encode a low-dimensional, aperiodic variable has led rescarchers to theoretically analyze the grid code's 'ss properties from multiple mathematical perspectives: coding theory, dynamical systems, function optimization, and supervised deep learning. In this section, extract key insights that suggest convergence towards a SSL approach.
4.1 Insights from Coding Theory
Many interrelated coding- theoorcttt properties of grid cells have been pointed out before: Exponential capacity: The coding states form a space-filling curve (ID) (Fig, 2 or space-filling manifold (2D) with well-separated coding states in the full coding volume defined by the number of cells. Linearly many distinet grid modules and therefore linearly many neurons generate exponentially many unique coding states for generie choices of periods [21 [6]. The grid code enables high dynamic- range representation (i.e., large value of range divided by resolution) of space using individual parameters that do not vary over large ranges 21 (Fig. -it)). Separation: Excluding the closest spatial locations, the coding manifold for spatial positions is interleaved in the coding volume, such that relatively nearby states are mapped faraway locations (Fig. [b) and relatively nearby spatial locations are mapped to faraway states (Fig. (2);; this gives the grid code strong crror-correcting properties (56). Decorrelation: Grid cells maximally decorrelate their representations of space beyond a small scale that is roughly half the period of the smallest module, so that all correlations are roughly equally small beyond that scale 21 56 (Fig. 2 Equinorm coding states: The grid coding states are equi-norm and lie on the surface of hypersphere. Ease of update: The grid code permits carry-free updating arithmetic so that individual modules update their states without communicating with each other, meaning updates can be computed in parallel rather than sequentially (21]. Algebraic: The grid code is a faithful algebraic code (21 (56, in that the grid code for composition of displacements is equal to the composition of grid codes for each of the displacements. This property enables the grid code to colf-consistently represent Euclidean displacements (Fig. 2 Whitened representation: All modules (coding registers phases) update by similar amounts for given displacement in real space, as a consequence of module periods being similarly sized (in contrast with fixed-base number system representations, where the "periods" for each register are hierarchically organized in big steps as powers of the base, and the representation is not whitened).
These properties are all intrinsic to the grid code, without regard to other types of spatial neurons, e.g. place cells. The lesson is that the grid code likely originates from
properties, not from predicting place cells (c.f., 5 55). The challenge before us is identifying which subset of these partially inter- mathematical function of grid cells forms the minimally sufficient subset for grid cell emergence.
properties of the grid code all useful for the
3

---

a
4
high-capactty representation
e
well- seli-separated coding states
rapid decorrelatin
b
neartnoural space) far(real space)
Fg
g
qa
algebraic coding
near(real space) fan neuurll space)
11g
lig, iolationn oownaoor
noural represeniation
space
Figure 2: Mammalian grid cells and their coding theoretic properties: Schematics in (a)-(e) depict the coding of ID rather than 2D space for simplicity, with phases of grid modules. (a) Continuous locations translate to a continuous coding line that wraps around the torus defined by the two grid module phases. Right: torus cut open sheet with boundary conditions. (b) Near-to-far mapping from phases to locations. (c) Near-to-far mapping from locations to phases. (d-e) High capacity and separability: For generic choices of grid module periods, the code space-filling curve on the phase torus (d), with capacity (before curves overlap for inite-thicknnes, curves) that grows exponentially with the number of modules. The coding lines tend to be well sepaaaepa for any range of real-space being encoded (2 shown) because of the interleaving property of the code (e). (f) The distance between coding representations g(0) and g(x) very quickly increases (around X/2 where À is the scale of the smallest module.) (g) Path-independent encoding of displacements.
4.2 Insights from Dynamical Systems
Consider the following general framework for defining neural code that is dynamically updated on the basis of a velocity signal: An agent starts with some neural representation go E G, then moves along a spatial trajectory given by the sequence of velocities (Vi, UT). The sequence of neural representations (gl,..., .... gT) is updated based on velocity through the (arbitrary, non-linear and
possibly time-dependent) dynamical translational operation f G x R² + G: def f(g--1, v)). In order for the neural representations to be good codes for Euclidean space, what properties should f and G possess? The first property is that the representations should be path invariant: gr should be the same regardless of whether the agent took path vi..... Ur or path Ux(1), .... Un(T) for any permutation desire that Vg. U. Uy f([(g, by) [[](g, ,,),,,, Note that path independence automatically implies that the coding states form continuous attractor [21]. because when the velocity O. the state should remain constant: Vg. f(g. 0) = g. Further, the grid cell representation is translation translationally invariant 39 31: representations at different positions are related by shift operator that only depends on the relative separation between the positions, i.e. the correlational structure thus twofold: the data and loss function(s) should together train the network learn path invariant representations and continuous attractor dynamics, and the dynamical translation operation of the recurrent neural networks should architecturally depend on the velocity.
: [T] (T). Specifically,
+ v)) f(v), where f is a function only of v. The lessons are
4.3 Insights from Spatial Basis Function Optimization
third approach to explaining grid cells via optimization of spatial basis functions. The spatial basis function optimization approach is non-dynamical one that posits spatial neural code g(x) of

---

a particular parametric form, e.g., g(x) def then define loss function and optimize parameters to discover which loss(es) results in grid-like This functional form used because reflects weight matrices, then one must use that functional form [18] While sensible, these insights have not yet crossed to deep recurrent neural networks. The challenges are that recurrent networks given velocity inputs are not explicit functions of space, and their representations need not be comprised Fourier basis functions. We seek to bring such insights. which requires discovering how to train recurrent networks that simultancously Iearn a consistent representation of space (i.e., learn a continuous attractor), while at the same time, shape the learnt representations to have desirable coding properties.
do a assinkk - + by cos(ks æ). One can
ag
tuning, possibly under additional constraints 39 18.
the constraints of path-integration: if one path- inth-integrates with action-depended
4.4 Insights from Supervised Deep Recurrent Neural Networks
Deep recurrent neural networks sometimes learn grid-liks like representations when trained in supervised manner on spatial target functions [3] 344
67]. and studies regarding the successor representation reinforcement learning 157 42 show similar findings. These results share common origin based on the eigendecomposition of the spatial supervised targets (putatively: place cells) which results in periodic vectors and was earlier noted in shallow nonlinear autoencoders 17 This understanding was used by [54] to hand-design better supervised targets.
23
65
22
While high profile, this line of work has fundamental shortcomings. Firstly, because the networks learn to regurgitate whatever the researcher encodes in the spatial target functions, the learned representations
by the researcher, not by the task or fundamental constraints 449 Secondly, by assuming the agent already has access to its spatial position (via the supervised targets), this approach bypasses the central question of how agents learn how to track their spatial position. Thirdly, previous supervised path-integrating recurrent neural networks either do not learn multiple grid modules or have them inserted via implementation choices, oftentimes due to statistics in the supervised targets [54] 49). Fourthly, supervised networks fare poorly once the spatial trajectories extend outside the training arena box (i.e., where supervised targets have not been defined).
The lesson we extract from this is to remove supervised targets altogether, and instead directly optimize the neural representations via self-supervised learning. Supervised networks pick up whatever statistics exist in the supervised targets, and struggle to generalize beyond their supervised targets, meaning that if we wish avoid putting grid cells in by hand or if we wish to obtain networks that generalize, we must remove the supervised targets altogether. Previous work lightly explored this direction by introducing unsupervised loss terms, but kept the supervised targets [22 67]. In this work, we fully commit to this lesson and take purely SSL approach.
5 Self-Supervised Learning of Representations for Space
We deduce that learning useful representations of space a SSL problem requiring an appropriate SSL framework (comprising training data, data augmentations, loss functions and architecture). We hypothesize that three loss functions, extracted from the coding, path integration and optimization per- spectives synthesized above, when combined with particular data and recurrent network architecture, will generate multiple modules grid cells. See App. A for experimental details.
5.1 Data Distribution and Data Augmentations
For each gradient step, we sample sequence of velocitis (V), U2.... UT), with Lddd p(v), then construct batch by applying B randomly sampled permutations 1/ To : [ + [T] the sequence of velocities to obtain B permuted velocity trajectories; doing so ensures many intersections between the trajectories exist in each batch (SI Fig. (b). We feed each trajectory through recurrent network (Sec. (5.2) with shared initial state Iu. producing sequence of neural representations (1mo()) (2) ..., (.) This yields a batch of training data for single gradient step.
gradient =(((ms())
(Voo())) (z((2). U (T)(T)), (I))(1) (aa(i))-mmm0 Im.()) ..... )mmm)) B
}
(I)
5

---

Separation
Path Invariance
Architectur ReLU
nEzE iggezutaiio
and
Vepee ropedry
Figure 3: A self-supervised framework for learning representations of space. (a) Separation incentivizes neural representations corresponding different spatial locations be well-separated. (b) (Path) Invariance incentivizes neural representations corresponding to the same spatial location to match. (c) The architecture of the recurrent neural network used. Inputs are 2D Cartesian velocities 2, E I2 and the non- non-linen con-iinarrtty is Morm-ReLU No positional readout exists. (d) For each batch, input trajectory velocities drawn i.i.d. from uniform distribution, then randomly permuted create batch; three trajectories shown.
5.2 Recurrent Deep Neural Network Architecture
Motivated by our insights from dynamical systems (Sec. interaction matrix given by:
4.2
we use an RNN (SI Fig.
8a) with
Ww.,) MLP((t)
(2)
where i E 2 is the velocity input to the recurrent network and W R NN for N hidden RNN units. The dynamics of the recurrent network are defined as:
g: (WW(v.
The neural representations are enforced to be non-negative and unit-norm via the non-linearity (.) def Norm ()))) (-) (-) R (U))) Constraining representations to be constant norm is common technique in SSL to avoid representation collapse [4] and also was used in previous grid cell modelling work [67]. By choosing this particular architecture, we ensure that representations different points in space related only function of velocity, thereby ensuring that the transformation applied to the state is the same so long as the velocity is the same.
5.3 Loss Functions
Neural representations corresponding to different spatial locations, regardless of trajectory, should well-separated (Fig- (p). Here, "different spatial locations" is with respect to spatial length scale > 0 and "well-separated" is with respect to neural length scale 0g > 0.
Separation Loss
Lsep
def
Σ
exp
1 (t) (m)(ee) (e')
20
VIL. [[xag(())) =
Path Invariance Loss Neural representations corresponding to the same spatial location, regardless of trajectory, should be identical (Fig (3)) i.c., neural representations should be invariant to the particular path the agent takes. If two trajectories are permuted versions of one another such that they begin at the same position and end the same position (measured with respect to a given spatial length scale og > 0). the neural representations after both trajectories should be attracted together.
Lio0
def
Σ Wab.s 1 (1 <22
gru()) (my(()) |
Capacity Loss Neural representations should be as high capacity as possible. That means that given finite neural representation space, the number of unique spatial positions that the neural representations can uniquely encode should be maximized. In our particular setting, because our representations lie on the hypersphere, we can incentivize high capacity in the following manner:
Lcap -
BT 9
1
Σ
(6)

---

a 1m
Figure 4: Multiple modules grid cells in trained RNN: (a) The RNN's hidden units learn three discrete mod- ules of spatially periodic representa- tions. Four cells from each module are displayed.
This capacity loss pushes together all neural embeddings, regardless of path traveled or point in time, encouraging little of the embedding space to be allocated for the training data as possible.
Conformal Isometry Loss Consider a velocity er that shifts the neural code from gr to 9++11 A neural code is conformally isometric if the distance between the neural codes remain constant if vr changes direction (but not magnitude). Motivated by [22 [67], we propose novel and simpler conformal isometry loss that minimizes the variance in ratios of changes in code space to changes in physical space for small velocities:
def v Ilg.
g 9c-1l
LConI
where V denotes the variance. Intuitively, LCoutso asks that neural representations change by some constant amount for the same magnitude velocity, regardless the velocity's s direction. Our overall self-supervised loss is then weighted combination of these four losses:
def Xsep Lsep Aino Liav + ACap LCap + AConlsofConIso
(8)
We term this framework suited to setting where neural representations are generated by a deep recurrent network which receives velocity inputs. Unlike previous (supervised) approaches, our SIC framework requires information about absolute spatial position, only relative spatial separation, and does not require any tuned supervised target, thus mitigating the shortcomings of previous works identified recently [48).
(SIC). This SIC framework is naturally
6 Experimental Results
6.1 Emergence of multiple modules of periodic cells
Under our SIC SSL framework, find that trained networks learn neurons with spatially multi- periodic responses. each run, most neurons in the network exhibit periodic tuning. Moreover, the network clearly cxhibits few discrete periods. We used grid search over hyperparameters with roughly 1/2 an order of magnitude variation for key hyperparameters (az, dg. Asep, Xtne, XCap) and found that multiple modules robustly emerge across this range. Computational limitations prevented us from sweeping more broadly. Example units from a run are shown in Fig. Below, we characterize different facets of the results.
6.2 Generalization to input trajectory statistics
drawback previous (supervised) deep learning grid cell works was poor generalization when the networks are evaluated in arenas larger than their training arenas. Grid cells in our networks
7

---

1000
Tempore sep soparration
CUI probabiliig
neuml sauce distanne
(Umestepp
9.6
dig/0)
-0.4
0.2
X(m) training speed distribution
0.0
spatial separation
eval speed distribution
count
4.0 05
656
eidn 030
speed (m/8)
speed (m/s)
1m
d
1.5m
2m
Figure 5: Grid tuning generalizes to different velocity statistics and to environments much larger than the training environment. (a) [left] A kernel density estimate plot of pairwise distances neural space for randomly chosen pairs of points real space showing rapid decorrelation over length scale that is R d cumulative CDF of the pairwise distance distribution over training batch (black), showing that the network generalizes to environment sizes much larger than it has seen during training. [right] A plot of pairwise neural space distance as a function of both spatial separation and temporal separation. Red vertical line at 0 spatial separation denotes perfect path integration. (b) Training trajectory statisties (left) and evaluation trajectory statistics (right) (c-e) Ratemaps for trajectories in boxes sizes 2m, 3m, 4m.
generalize to arenas many times larger than the distances explored during training (Fig. - (training trajectories extend < 1.22m from the starting position (Fig. 5a)). Secondly, the training trajectories were drawn i.i.d. from uniform velocity distribution, Fig. [5 (left), while evaluation trajectories were constructed to be smoother and avoid the walls of a confined arena (Fig. 5p, left). As found in experiments, the periodicities of the emergent grid cells remained the same, regardless of the size of the arena (Fge))e)).
6.3 State space analysis of trained RNNs
We turn to a more-detailed characterization the grid cells in each module and their cell-cell relationships/populatioo states. For big enough set of cells for comparison with experimental results, we needed sufficiently many co-modular cells and so used the network corresponding to Fig. 77p in which most of the units formed grids of the same period.
found that cells with similar periods were tightly clustered to have essentially identical periods and orientations (Fig. 64). Moreover, this population tiled the space of all phases (the Fourier PSD contains a hexagonal structure with wavevectors k1 k2, ky; we defined three non-orthogonal phases satisfying E.=122. of 0 mod 2n for each unit from its ratemap r(x): of phase [] dxe "r, "r:())) exhibiting uniform phase distribution (Fig. 6D).

---

a mean fourier power
toroidal state space
module onentation
righ
b
phase coverage
projections
d
Kg
Figure 6: Analysis of low-dimensional structure in one of the grid cell modules: (a) (left) Mean Fourier power spectral density of all 128 units in the module, showing hexagonally arranged peaks. (right) Histogram of cell periods and orientations. (b) Distribution of phases with phases Φ1, d22, d3 calculated by evaluating the Fourier transform of each ratemap along cach lattice vector. (c) Cells from one module exhibit a toroidal manifold (visualization by non-linear dimensionality reduction (AppDD). (d) Projection of the state space manifold onto the three principal directions defined by vectors k2, 12, ky shows 3 rings.
In theoretical models and real data, grid cells exhibit low-dimensional dynamics and the states of all cells in module lie 2D torus. We used spectral embedding 44 to find that the emergent grid cell population states also fell on toroidal manifold (Fig. (4). Following (55), we projected the activity manifold onto 3 axes defined by each set of phases calculated for each unit. This revealed the existence rings, Fig. 6d, confirming that the attractor manifold has a 2d twisted torus topology.
(6)..(6)).
Ablation experiments We also ablated each of the three loss terms, the data augmentation, i.e., trajectory permutations and the hyperparameter ag reveal when multiple modules of grid cells changes to single module and then to a place-cell like code (Fig a-f).
Discussion & Future Directions
In this work, we extracted insights from previous first principles coding theoretic properties of grid cells and mechanistic models that produce grid-like firing. We then proposed novel set of self-supervised loss functions that with the architectural constraint of a recurrent neural network can produce multiple modules grid cells when trained on spatial navigation task. doing so, we have highlighted the distinction between separation and capacity for neural representation and shown that optimizing for capacity leads to efficient representations that are better for generalization to larger amounts of test data.

---

a
capacity loss ablation
b
capacity loss ablation + +
capacity loss ablation + Og=02
Ser 6a
O
Mi- 200
Max 300
Max dem
Mo: encodo
INOOO
5.30
1
RV: AUE
Mir 30
d trajectory permutations ablation
path invariance loss ablation
separation loss ablation
Figure 7: Ablation experiments: (a) Removal of capacity loss leads to a single module of grid cells. (b-c) Reducing og makes the representation place cell-like, with size of place fields decreasing with increasing gg- (d-f) Ablating other key ingredients leads to loss of spatial tuning.
Looking to the future, we see four broad directions, two in neuroscience and two in machine learning. Firstly, we believe significantly more work is necessary to understand these trained recurrent networks and connect them back grid cells. Key questions include: What components of the data, augmentations, losses and architecture control the number of modules, the distribution of the module periods, the period ratios, the structure (square or hexagonal) of the lattices? What do the state spaces of these networks look like? Can we reverse-engineer the learnt mechanisms, using mechanistic Another interesting question is what underlying connectivity leads discrete modular responses (29]. How do the learnt representations change when ex exposed to experimental manipulations, e.g., environmental deformations or rewards [33 34
analyses [50 30 3
35 28 g 7 What new experiments can these networks suggest in biological agents? Secondly, we see many opportunities in neuroscience broadly. The biological brain cares about high capacity representations, and grid
cells have been implicated in many diverse cognitive and non-spatial tasks [15] 1 6 45
63 44)
How might our grid cell recurrent networks, and more generally, SSL principles be applied to drive computational neuroscience forward?
Thirdly, are excited push SIC framework further towards machine learning. We envision empirically exploring whether our SIC framework can improve performance in other popular SSL domains such vision and audition, well as mathematically characterizing the relationship between SIC and other SSL frameworks. Fourthly, one puzzling aspect of our capacity loss is that it can be viewed as minimizing the entropy of the neural representations in representation space, which flies against common information theoretic intuition arguing for maximizing the entropy of the
53 [22. In some sense, the capacity loss akin to "dual" minimum description length problem 25 instead of maximizing compression by having a fixed amount of unique data and trying to minimize the code words' length, here we maximize compression by having fixed length code words and trying to maximize the amount of unique data. How can this apparent contradiction be resolved?
representations 666 43 62
36.
10

---

8 Acknowledgements
IRF is supported by the Simons Foundation through the Simons Collaboration on the Global Brain, the ONR. the Howard Hughes, Medical Institute through the Faculty Scholars Program and the K. Lisa Yang ICoN Center. SK acknowledges support from NSF grants No. 2046795, 1934986, the Alfred P. Sloan Foundation,
2205329, and NIH 1622616222-0011 NIFA 0020-67021-32799 and Google Inc. RS is supported by a Stanford Data Science Scholarship. MK is supported by Math Works Science Fellowship and the MIT Department of Physics.

---

References
[1]
Dmitriy Aronov, Rhino Nevers, and David W. Tank. Mapping of a non-spatial dimension by the hippocampal-entorhine circuit. Nature, 543(7647):719-722) March 2017. Number: 7647 Publisher: Nature Publishing Group.
[2] Philip Bachman, R Devon Hjelm, and William Buchwalter. Learning Representations by Maximizing Mutual Information Across Views. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[3] Shahab Bakhtiari, Patrick J. Mineault, Tim Lillicrap, Christopher C. Pack, and Blake Aaron Richards. The functional specialization of visual cortex emerges from training parallel pathways with self-supervised predictive learning. October 2021.
[4] Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, et A cookbook of self-supervised learning. arXiv preprint arXiv: /24./22211 10, 2023.
[5] Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel, Martin Chadwick, Thomas Degris, Joseph Modayil, Greg Wayne, Hubert Soyer, Fabio Viola, Brian Zhang, Ross Goroshin, Neil Rabinowitz, Razvan Pascanu, Charlie Beattie, Stig Petersen, Amir Sadik. Stephen Gaffney, Helen King, Koray Kavukeuoglu, Demis Hassabis, Raia Hadsell, and Dharshan Kumaran. Vector-based navigation using grid-like representations artificial agents. Nature, 557((705))429 433,, May 2018.
[6] Xiaojun Bao, Eva Gjorgieva, Laura K Shanahan, James D Howard, Thorsten Kahnt, and Jay A Gottfried. Grid-like neural representations support olfactory navigation of a two-dimensional odor space. Neuron, (02(5):1066-1))
2019.
[7] Jacob LS Bellmund, William De Cothi, Tom A Ruiter, Matthias Nau, Caswell Barry, and Christian F Doeller. Deforming the metric of cognitive maps distorts memory. Nature human behaviour, 4(2): 177-188, 2020.
[8] Charlotte N Boccara, Michele Nardin, Federico Stella, Joseph O'Neill, and Jozsef Csicsvari. The entorhinal cognitive map is attracted to goals. Science, 3636643))1143--444,, 2019.
[9] William N Butler, Kiah Hardcastle, and Lisa M Giocomo. Remembered reward locations restructure entorhinal spatial maps. Science, 363(6434 2019..
[10] Mathilde Caron, Piotr Bojanowski, Julien Mairal, and Armand Joulin. Unsupervised Pre- Training of Image Features on Non-Curated Data. arXiv:9905.01278 [cs]. August 2019. arXiv: 1905.01278.
[11] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised Learning of Visual Features by Contrasting Cluster Assignments. arXiv:2006. 09882 [cs], January 2021. arXiv: 2006.09882.
[12] Rishidev Chaudhuri and Il Fiete. Computational principles of memory. Nature neuroscience, 19(3):394-403 2016.
[13] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Contrastive Learning of Visual Representations. arXiv:2002.05709 [cs, stat], June 2020. arXiv: 2002.05709.
[14] Xinlei Chen and Kaiming He. Exploring Simple Siamese Representation Learning, November 2020. atXiv:2011..0560 [cs].
[15] Alexandra O. Constantinescu, Jill X. O' O'Reilly, and Timothy E. J. Behrens. Organizing concep- tual knowledge in humans with gridlike code. Science, Publisher: American Association for the Advancement of Science.
June 2016.
[16] Christopher Cueva and Xue Xin Wei. Emergence of grid-like representations by training recurrent neural networks to perform spatial localization. International Conference on Learning Representations, page 19, 2018.
12

---

[17] Yedidyah Dordek, Daniel Soudry, Ron Meir, and Dori Derdikman. Extracting grid cell character- istics from place cell imputs using non-negative principal component analysis. eLife, 5xe10094, March 2016. Publisher: elfef Sciences Publications, Ltd.
[18] William Domell, Peter E Latham, Timothy EJ Behrens, and James CR Whittington. Actionable neural representations: Grid cells from minimal constraints. International Conference on Learning Representations, 2023.
[19] Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, and Nicu Sebe. Whitening for Self-Supervised Representation Learning. page 10, 2021.
[20] William Falcon and The PyToch Lightning team. PyTorch Lightning, March 2019.
[21] lla R. Fiete, Yoram Burak, and Ted Brookings. What Grid Cells Convey about Rat Location. Journal Neuroscience, 28(27):6858-66)) July 2008. Publisher: Society for Neuroscience Section: Articles.
[22] Ruiqi Gao, Jianwen Xie, Xue-Xin Wei, Song- Long-Chun Zhu, and Ying Nian Wu. On path integration of grid cells: Group representation and isotropic scaling. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 28623-28635. Curran Associates, Inc., 2021.
[23] Ruiqi Gao, Jianwen Xie, Song- Mhng Zhu, and Ying Nian Wu. LEARNING GRID CELLS AS VECTOR REPRESENTA- TION OF SELF-POSITION COUPLED WITH MATRIX REPRE- SENTATION OF SELF- SELF-MOTION, page 24, 2019.
[24] Richard J. Gardner, Erik Hermansen, Marius Pachitariu, Yoram Burak, Nils Baas, Ben- jamin A. Dunn, May-Britt Moser, and Edvard I. Moser. Toroidal topology of population activity grid cells. Nature, 602(7895); 128-128, February 2022. Number: 7895 Publisher: Nature Publishing Group.
[25] Peter D Grünwald. The minimum description length principle. MIT press, 2007.
[26] Torkel Hafting, Marianne Fyhn, Sturla Molden, May-Britt Moser, and Edvard I. Moser. Mi- crostructure of spatial map in the entorhinal cortex. Nature, 436(7052):801-800) August 2005. Number: 7052 Publisher: Nature Publishing Group.
[27] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9726-9735, Seattle, WA, USA, June 2020. IEEE.
[28] Alexandra Keinath, Russell A Epstein, and Vijay Balasubramanian. Environmental deforma- tions dynamically shift the grid cell spatial metric. Elife, 7:e38169. 2018.
[29] Mikail Khona, Sarthak Chandra, and Ila R. Fiete. From smooth cortical gradients to discrete modules: spontaneous and topologically robust emergence of modularity in grid cells. Technical report, bioRxiv, January 2022. Section: New Results Type: article.
[30] Mikail Khona, Sarthak Chandra, Joy Ma, and lla R Fiete. Winning the lottery with neural connectivity constraints: Faster learning across cognitive tasks with spatially constrained sparse mns. Neural Computation, pages 1-20, 2023.
[31] Mikail Khona and Ila R Fiete. Attractor and integrator networks in the brain. Nature Reviews Neuroscience, pages 1-23, 2022.
[32] Talia Konkle and George A Alvarez. A self-supervised domain-general learning framework for human ventral stream representation. Nature communications, 13(1):1-12, 2022.
[33] Julija Krupic, Marius Bauza, Stephen Burton, Caswell Barry, and John Keeff. Grid cell symmetry is shaped by environmental geometry. Nature, 518(7538):232-25) 2015.
[34] Julija Krupic, Marius Bauza, Stephen Burton, and John Keefe. Framing the grid: effect of boundaries grid cells and navigation. The Journal of physiology, 594(22)x6489-644)) 2016.
13

---

[35] Julija Krupic, Marius Bauza, Stephen Burton, and John O' Oeeefe Local transformations of the hippocampal cognitive map. Science, 199(6380):113311111 2018.
[36] Xin Liu, Zhongdao Wang. Ya-Li Li, and Shengjin Wang. Self-supervised learning via maximum entropy coding, Advances in Neural Information Processing Systems, 35:34091-34105 2022.
[37] Ziming Liu, Mikail Khona, Ila R Ficte, and Max Tegmark. Growing brains: Co-emergence of anatomical and functional modularity in recurrent neural networks. arXiv preprint arXiv:2310.07711 2023.
[38] Nya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 1r 2017.
[39] Tzuhsuan Ma. Towards a theory for the emergence of grid and place cell codes. PhD thesis, Massachusetts Institute of Technology, 2020.
[40] Alexander Mathis, Andreas V. M. Herz, and Martin Stemmler. Optimal Population Codes for Space: Grid Cells Outperform Place Cells. Neural Computation, 24(9):2280-233) September 2012.
[41] Patrick Mineault, Shahab Bakhtiari, Blake Richards, and Christopher Pack. Your head is there to move you around: Goal-driven models of the primate dorsal pathway. In Advances in Neural Information Processing Systems, volume 34, pages 28757-28771. Curran Associates, Inc., 2021.
[42] Ida Momennejad. Learning structures: predictive representations, replay, and generalization. Current Opinion in Behavioral Sciences, 32:155-166, 2020.
[43] Serdar Ozsoy, Shadi Hamdan, Sercan Arik, Deniz Yuret, and Alper Erdogan. Self-super learning with an information maximization criterion. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 35240-2500 -32533. Curran Associates, Inc., 2022.
[44] Seongmin A Park, Douglas S Miller, and Erie D Boorman. Inferences on multidimensional social hierarchy use grid-like code. Nature neuroscience, 129--1139213311 2021.
[45] Seongmin A Park, Douglas S Miller, Hamed Nili, Charan Ranganath, and Erie D Boorman. Map making: constructing, combining, and inferring on abstract cognitive maps. Neuron, (0766):1226-1233) 2020.
[46] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, nigh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[47] Lawrence K Saul, Kilian Q Weinberger, Fei Sha, Jihun Ham, and Daniel D Lee. Spectral methods for dimensionality reduction. Semi-supervised learning, 3, 2006.
[48] Rylan Schaeffer, Mikail Khona, and Ila R Fiete. No free lunch from deep learning in neu- roscience: A case study through models of the antorhinal-hippocaapaal circuit. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[49] Rylan Schaeffer, Mikail Khona, and 1l Rani Fiete. No Free Lunch from Deep Learning in Neuroscience: A Case Study through Models of the Hiipooaplll Circuit. August 2022. Pages: 0022.08.07.5033.
Section: New Results.
[50] Rylan Schaeffer, Mikail Khona, Leenoy Meshulam, Brain Laboratory International. and Ila Fiete. Reverse-engineering recurrent neural network solutions to a hierarchical inference task for mice. Advances in Neural Information Processing Systems, 33:4584 4596, 2020.
[51] Vemund Schøyen, Markus Borud Pettersen, Konstantin Holzhausen, Marianne Fyhn, Anders Malehe-Sprenssen, and Mikkel Elle Lepperød. Coherently remapping toroidal cells but not grid cells are responsible for path integration in virtual agents. bioRxiv, pages 2022-08, 2022.
14

---

[52]
Ravid Shwartz-Ziv, Randall Balestriero, Kenji Kawaguchi, Tim G. J. Rudner, and Yann LeCun. An information-thheortiic perspective on variance-invarriace--com invariance-covarianee regularization, 2023.
[53] Ravid Shwartz-Ziv, Randall Balestriero, and Yann LeCun. What do we maximize in self- supervised learning?, 2022.
[54] Ben Sorscher, Gabriel C Mel, Surya Ganguli, and Samuel A Ocko. A unified theory for the origin of grid cells through the lens of pattern formation. Advances in Neural Information Processing Systems, page 18, 2019.
[55] Ben Sorscher, Gabriel C Mel, Samuel A Ocko, Lisa M Giocomo, and Surya Ganguli. A unified theory for the computational and mechanistic origins of grid cells. Neuron, 111(1):21-1377, 2023.
[56] Sameet Sreenivasan and Ila Fiete. Grid cells generate an analog error-correcting code for singularly precise neural computation. Nature Neuroscience, 14(10):1300-1337 October 2011. Number: 10 Publisher: Nature Publishing Group.
[57] Kimberly L Stachenfeld, Matthew M Botvinick, and Samuel J Gershman. The hippocampus as a predictive map. Nature neuroscience, 20(1 16:163--165,, 2017.
[58] Martin Stemmler, Alexander Mathis, and Andreas VM Herz. to decode the population activity of grid cells. Science Advances, 1(1 (11))ee500086, 2015.
Connecting multiple spatial scales
[59] Hanne Stensola, Tor Stensola, Trygve Solstad, Kristian Frøland, May-Britt Moser, and Edvard I. Moser. The entorhinal grid map is discretized. Nature, 492(7427):72-77, December 2012. Number: 7427 Publisher: Nature Publishing Group.
[60] Joshua B Tenenbaum, Vin de Silva, and John C Langford. A global geometric framework for nonlinear dimensionality reduction. science, 290(5500):2319-23333 2000.
[61] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive Representation Distillation. arXivv: /910. 10699 [es, stat), January 2020. arXiv: 1910.10699
[62] Yao-Hung Hubert Tsai, Yue Wu, Ruslan Salakhutdinov, and Louis-Philippe Morency, Self- supervised learning from a multi-view perspective, 2021.
[63] Simone Viganò, Valerio Rubino, Antonio Di Soccio, Marco Buiatti, and Manuela Piazza. Grid-like and distance codes for representing word meaning in the human brain. Neurolmage, 232: 11776, 2021.
[64] Xue-Xin Wei, Jason Prentice, and Vijay Balasubramanian. A principle of economy predicts the functional architecture of grid cells. eLife, 4:e08362, September 2015. Publisher: eLife Sciences Publications, Ltd.
[65] James R. Whittington, Timothy H. Muller, Shirley Mark, Guifen Chen, Caswell Barry, Neil Burgess, and Timothy E. J. Behrens. The Tolman-Finn Eicheenauc Machine: Unifying Space and Relational Memory through Generalization the Hippocampal Formation. Cell, 183(5):1249- 1263.e23, November 2020.
[66] Mike Wu, Chengxu Zhuang, Milan Mosse, Daniel Yamins, and Noah Goodman. On mutual information in contrastive learning for visual representations, 2020.
[67] Dehong Xu, Ruiqi Gao, Wen-Hao Zhang, Xue-Xin Wei, and Ying Nian Wu. Conformal isometry of lie group representation in recurrent network of grid cells. arXiv preprint 22..v:2210.0266:. 2022.
[68] KiJung Yoon, Michael Buice, Caswell Barry, Robin Hayman, Neil Burgess, and Ila R Fiete. Specific evidence of low-dimensional continuous attractor dynamics in grid cells. Nature neuroscience, 16(8): 1077--0477, 2013.
[69] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow Twins: Self- Supervised Learning via Redundancy Reduction, June 2021. arXiv:2103.03230 [es, q-bio].
[70] Chengxu Zhuang. Siming Yan, Aran Nayebi, Martin Schrimpf, Michael C. Frank, James J. DiCarlo, and Daniel I.. K. Yamins. Unsupervised neural network models of the ventral visual stream. Proceedings of the National Academy of Sciences, 10019): January 2021.
15

---

A
Experimental Details
Architecture and training data augmentations
Our code was implemented in PyTorch [4] and PyTorch Lightning [20]. Hyperparameters for our experiments are listed in Table
Our code will be made publicly available upon publication.
Hyperparameters
Values
Batch size
130
Trajectory length
60
Velocity sampling distribution
Ut IR? aiidd Uniform" 2 (-0.15, 0.15) meters
RNN nonlinearity
Norm Re (U())
Number of RNN units
128
Number of ML. layers
3
Spatial length scale xe Neural length scale dy
0.05 meters
0.4
Separation loss coefficient Asep
1.0
Invariance loss coefficient Aine Capacity loss coefficient ACap
0.1
0.5
Optimizer
AdamW (38
Optimizer scheduler
Reduce Learning Rate on Plateau
Learning rate Gradient clip value
2e-3
0.1
Weight decay
None
Accumulate gradient batches
2
Number of gradient descent steps
2c6
Table 1: Hyperparameters used for training the networks.
Architecture
Data and Augmentations
Norm-ReLU
start end
self-supervised SIC loss
velocity inputs
(((
(t)
V.V.VVV.VV
Figure A trained recurrent neural network learns multiple modules of grid cells. (a) The architecture of the recurrent neural network used. Inputs are 2D Cartesian velocities E IR² and the non-linearity is Norm-Rell ReLU, No positional readout exists. (b) For each batch, input trajectory velocities drawn i.i.d. from uniform distribution, then randomly permuted to create batch; trajectories shown.
16

---

b
Figure 9: All 128 ratemaps evaluated trajectories inside 2m box. (a) Ratemaps from the
corresponding to Fig. (b) Ratemaps corresponding to the run in Fig 6
B All Ratemaps
17

---

a
2
0=0
R
Figure 10: (a) Example cells from modules, with preferred phase 0 and (b) Visualizing the state space defined by {0,, as a torus (left) and on a square with periodic boundary conditions (right), which an equivalent construction of torus.
C Construction of the grid code
To explain the structure of the grid code, consider idcalized tuning curves in 1d.
Each cell is defined by its periodicity X and preferred phase Φ; S All cells in the same module a have the same periodicity and uniformly tile all allowed phases. For position r E R.
2T x + Φ5
rf (x) Rmmax ReLU
(9)
Aa
The tuning curves corresponding to this module can be seen in Fig, 10a.
2xT
For this module, can define dº (x) So the firing can be written Aa
modulo 2m. Here d S
(x) max ReLU [cos (0)(x) + di)]
(10)
All information about the current state of the module is encoded in the single variable dº Thus the set of phases (dx) )a S x S uniquely define the coding states of the set of grid modules.
For 2 modules, defined by (d",d²), these states can be visualized as being on a torus S S1 Fig 10p.
D Nonlinear Dimensionality, Reduction
For Fig. 6 we qualitatively followed the methodology used by seminal experimental papers exam- ining the topology of neural representations [12] 24]: used principal components analysis dimensions followed by non-linear dimensionality (in our case, spectral embedding) reduction to dimensions. Similar results obtained if one uses Isomap (60).
18


Reviews:
[{"review_id": "plh21oVMNA", "reviewer": "Reviewer_keKQ", "summary": "The paper shows that recurrent networks trained with a \"self-supervised\" loss leads to units of the internal representations that organize as grid cells. In particular, the paper defines a loss that promotes separations between neural representations encoding different spatial locations, encourages a representation to be invariant to different possible paths taken leading to it's representation, and maximizing the capacity of the representation, and the authors use paired velocities and neural representations as their dataset.\n", "questions": "- Do all the internal units have grid cell properties, or only a fraction?\n- Related, but rather than show example units that look like grid cells, are there any metrics that quantify the extent to which a cell is a grid cell?\n- Are there any predictions that can be made, for example, what might happen for animals that explore 3d space?\n- Can the authors comment on the relationship between the toroid's and the encoding of space to make the paper self-contained (e.g. schematics from Fig. 2)?", "limitations": "Yes", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper was very clearly written and organized, with nice visuals that supported the text. Moreover, the paper provided a helpful background of previous research that was relevant to the formulation used in the paper.\n- The paper defines a loss that is nicely linked to existing theories for the emergence of grid cells, and shows that this loss, optimized using gradient descent leads emergence of grid cells in artificial recurrent networks.\n- The authors performed ablations of the hyperparameters in their loss to show the dependence of their results on the different terms", "weaknesses": "- How robust are the results to other hyperparameters, like batch-size, learning rate, etc?\n- While the authors criticize previous work that identified the emergence of grid cells using supervised RNN to specifics of the target function (line 55), the authors do not seem to properly explain how their setup differs, and does not lead to similar implicit assumptions. For example, what is the difference between the velocities being used as a supervised signal (which the authors criticize), versus incoporating implicitly into their dataset and self-supervised loss? Are there similar assumptions with respect to the creation of the dataset (e.g having a sufficient number of examples with overlapping positions)\n- Further, the authors assert that \"SSL mitigates the need for large scale supervised data\", but it is unclear to me how different it is to incorporate the velocities as a paired dataset rather than a target variable for a supervised objective. \n- (Minor) The claim in the discussion \"how might ... SSL principles be applied to drive computational neuroscience forward\" seems too general.\n- (Minor) Difficult to read text in Figure 7\n- (Minor) extra italics on t in line 138", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Author Response\n\nWe thank Reviewer keKQ for their time, feedback and high score.\n \nReplying to each comment, suggestion and/or question in turn:\n\n> How robust are the results to other hyperparameters, like batch-size, learning rate, etc?\n> e.g having a sufficient number of examples with overlapping positions\n\n\nThis is a good question. We have some initial answers but more work is necessary and already underway! For context, SSL is notoriously compute hungry (see Balestriero et al. 2023), and training one of our SSL networks requires a 16 GB GPU for ~7 days, whereas a supervised path-integrating network from previous papers takes an ~8 GB GPU for ~6 hours. \n\nWe have found that batch sizes of 120, 130 and 150 all work equally well; we haven’t tried smaller batch sizes, and larger batch sizes can result in OOM errors. We tested learning rates in {0.002, 0.0002, 0.00002} x optimizers in {Adam, AdamW, NAdam, NAdamW} x learning rate schedulers in {Reduce LR on Plateau, Linear Warmup with Cosine Annealing} and found different combinations have different learning curves but reach the same result. We’re currently running additional experiments and ablations. \n\nRegarding sufficiently many overlapping positions, we have not yet tested this rigorously. We conjecture that some fraction of overlapping positions is necessary for two reasons: (1) no overlapping positions is akin to contrastive SSL without a sufficient number of positive pairs, and (2) in the past, we tried sampling trajectories without overlapping positions and random representations emerged because the network does not need to path integrate; rather, it only needs to ensure it never repeats a code word. We intend to test what fraction of overlapping points is necessary to put in the appendix but have not yet been able to prioritize that experiment.\n\n\n> While the authors criticize previous work that identified the emergence of grid cells using supervised RNN to specifics of the target function (line 55),\n\nLine 55 was unintentionally harsh. We will rewrite this. \n\n>  the authors do not seem to properly explain how their setup differs, and does not lead to similar implicit assumptions. \n\nThis is good feedback! We will do a significantly better job at distinguishing our approach from the previous supervised learning approach. We have devoted our Global Response (Approach 2) to this point.\n\n - To your question about distinguishing ourselves from previous approaches, please see the Global Response.\n\n - To explain generally the shortcomings we see with supervised path-integrating networks see Approach 2 of the Global Response\n\n> Further, the authors assert that \"SSL mitigates the need for large scale supervised data\"\n\nTo contextualize that sentence, the sentence is in our Background and reads: “SSL is increasingly gaining popularity as a normative approach in neuroscience since SSL mitigates the need for large-scaled supervised data that biological agents do not possess.” We think this is a general viewpoint across multiple modalities (vision, language, audition) that many neuroscience & ML papers have made previously. We will add citations to prominent neuroAI work that uses self-supervised learning across modalities that make a similar point.\n\nIn the particular modality of spatial navigation, we can comment on the feasibility of the self-supervised versus supervised setup. Recall that the goal is to learn a self-consistent representation of spatial position that is updated by velocity. It’s unrealistic to assume that biological agents have access to their absolute spatial position at all times; if they did, why would they need to learn their spatial position? Consequently, the supervised target bypasses the learning problem. In contrast, permuting trajectories is generally straightforward. For instance, if you walk to a grocery store twice, you might go North then West on the first trip, and you might go West then North on the second trip. We think our setup is much more realistic for biological agents.\nTo further contrast supervised & SSL, we do not include velocities as a paired dataset. Rather we use velocities and convert it to a binary tensor of 1’s and 0’s, where 1 denotes that positions are within $\\sigma_x$ of each other. This is a very low information, coarse-grained and impoverished learning target.\n\n>  are there any metrics that quantify the extent to which a cell is a grid cell?\n\nThere is indeed a metric! Unfortunately, the “grid score” used by previous works (such as Banino et al. 2018, Nayebi et al. 2021, Schaeffer et al. 2022) are applicable either for _hexagonal_ lattices or _square_ lattices but our multi-periodic lattices are all sheared. This means that the square or hexagonal grid score cannot be used for our networks to identify grid-like units.  We suspect that the ``conformal isometry” loss function [first identified by Dehong Xu, Ruiqi Gao et al. 2022] might make our lattices more hexagonal and we are currently exploring this direction.\n\n\n> Predictions for 3d space\n\nWe thank you for this question! To the best of our knowledge, the majority of previous work for grid cells assumes flat 2d Euclidean space.  It is certainly an interesting point of discussion. Grid cells have been observed in 3D [Yartsev 2011, Ginosar et al. 2021] and at least one paper has studied them in higher dimensional spaces [Klukas et al. 2020]. However, we follow the majority of previous modeling work that focuses on 2D Euclidean space. We leave 3D for future work; we will add this to our Discussion. \n\n>  The claim in the discussion \"how might ... SSL principles be applied to drive computational neuroscience forward\" seems too general.\n\nWe’ll rework this sentence. We weren’t trying to say something grandiose, merely that we hope our paper might be useful to the field.\n\n> Difficult to read text in Figure 7\n\nWe will make the fonts in all our figures larger.\n\n\n\n\n\n"}, {"review_id": "55FBicpwUc", "reviewer": "Reviewer_dHUQ", "summary": "This work reviews some of the issues with existing models of grid cells (cells in mammalian brains that fire when the animal is located at the vertices of a hexagonal grid) and suggests a new model based on recurrent neural networks (RNNs). The model is self-supervised, eliminating the worry that the structure of the readout in a supervised task could influence the outcome. Besides leading generically to grid-like firing, the model also exhibits multiple grid scales organized in modules such that the scale is the same within each module but the phase varies.", "questions": "1. The permutation invariance discussed around eq. (2) assumes Euclidean space, where the order of the steps does not change the outcome of a path. This seems overly restrictive since a typical animal's habitat is unlikely to feature a flat 2d plane, but instead is more likely to contain obstacles, hills, tunnels, etc. Can the authors comment on how their method might adapt to such cases?\n2. The motivation for the capacity loss in eq. (9) seems a bit obscure. This was touched upon briefly in the Discussion, but it would be useful for the authors to indicate how they came up with this particular formulation. Also, were other capacity loss functions attempted, and how did they work?\n\nMinor comments:\n* line 127: $f$ is used here with a different meaning from $f$ in, e.g., eqns. (2) or (3), which is a bit confusing\n* Figure 6, panel c: what non-linear dimensionality reduction technique was used?", "limitations": "The authors have adequately discussed limitations of their work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "*Originality:*\nThe paper suggests a way for grid cells to emerge from a self-supervised learning (SSL) paradigm, in contrast to previous work which works mostly in a supervised regime.\n\n*Quality:*\nThe paper does a good job of surveying prior work and includes a fair amount of simulations.\n\n*Clarity:*\nThe presentation is generally clear.\n\n*Significance:*\nGrid cells are of tremendous interest in neuroscience and there is a considerable volume of work attempting to explain their properties and the reasons behind their existence. This paper provides a novel model for how grid cell may emerge – as a self-supervised means of keeping track of an animal's location in space – and is thus of great interest to neuroscientists.", "weaknesses": "1. The authors present this work as a significant advance over methods based on supervised learning because the latter depend on specific design choices. However, the same seems to be true in the new approach: for instance, while the separation and path-invariance loss are pretty natural, the capacity loss is counter-intuitive, as mentioned even by the authors in the Discussion. Excluding the capacity loss eliminates the multi-scale nature of the solution. Moreover, the emergence of grid cells is sensitive to the parameters used in the loss, as shown in Figure 7. It is thus not immediately obvious that the proposed method requires any less fine tuning to lead to grid cells than prior models.\n2. Ideally the code used to run all the simulations would have been included with the supplementary material. It was promised for after acceptance, but this seems hard to justify, since the code exists already, and it could be useful for a thorough evaluation of the paper.\n3. Most of the figure panels should be significantly larger – Figure 7 is a particularly bad example. I understand that space is a limiting factor, but some progress can be made by including fewer ratemaps (don't see the need for more than 3x3 or 4x2 examples of each kind). Also ensure that font sizes don't go below 7 or 8 – when they do, it may be better to just remove the text because it is very inconvenient to read.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Author Response\nWe thank Reviewer dHUQ for their time and feedback. Replying to each comment, suggestion and/or question in turn:\n\n> The authors present this work as a significant advance over methods based on supervised learning because the latter depend on specific design choices. However, the same seems to be true in the new approach [...] It is thus not immediately obvious that the proposed method requires any less fine tuning to lead to grid cells than prior models.\n\nThank you for giving us a chance to clarify! We want to make three points.\n1. Firstly, the prior supervised path-integrating network papers are insightful and valuable. Our paper would not be possible without their contributions.\n2. Secondly, we want to reiterate that our paper is a significant contribution not just to supervised path-integrating networks but to three other lineages of understanding grid cells: basis function optimization, coding theory and continuous attractor models . As we explain in our Global Response, all four lines of work have at least one significant limitation, and our paper solves all four lineages’ limitations simultaneously. This is why we feel our paper is a novel & significant contribution. \n3. Thirdly, the limitations of supervised path-integrating networks that we identify are more nuanced than they “depend on specific design choices.” To explain generally the shortcomings we see with supervised path-integrating networks, there are several. See Approach 2 of the Global Response for an extensive discussion.\n\n> the capacity loss is counter-intuitive, as mentioned even by the authors in the Discussion.\n> The motivation for the capacity loss in eq. (9) seems a bit obscure. This was touched upon briefly in the Discussion, but it would be useful for the authors to indicate how they came up with this particular formulation. \n\nWe thank the reviewer for the opportunity to better explain the capacity loss. The capacity loss is one of our conceptual breakthroughs, and a key component of our paper. Previous theory work [Fiete, Burak, Brookings 2008; Sreenivasan & Fiete 2011] identified a high coding capacity as one of the key properties the grid code provides.\n\nThe capacity loss demands from the network: “Use as little of your total available coding volume as possible on the training data (subject to separation and path invariance).” This is linked to generalization. On the training data, the network learns how to dynamically evolve its representations in a manner consistent with velocity inputs. The network could learn a ‘shortcut solution’ (single scale solution) and use up all its coding volume on the training data, but then it will fail to generalize to longer trajectories (Fig. 3eg). The capacity loss provides a top-down inductive bias to prevent this ‘shortcut solution’. \n\n> Also, were other capacity loss functions attempted, and how did they work?\n\nNo, no other capacity loss functions were attempted. We can explain how this capacity loss came to be. We found that the first two loss terms reliably produced one grid module (i.e. all units sharing one period, as shown in our ablation figure) but that the period scaled to match the training trajectory length. The problem is that there’s no incentive to ensure that a location outside the training distribution has a different representation than a location inside the training distribution, but that’s precisely what we needed. How could we incentivize generalization? Fig 3efgh was the conceptual breakthrough: Fig3e and Fig3f both achieve the same _training_ loss, but only Fig3f generalizes. Thus, we needed a loss term to prefer Fig3f over Fig3e, and once we conceptualized Fig 3e and 3f the first loss function we thought of was the paper’s current capacity loss.\n\n> 2. Ideally the code used to run all the simulations would have been included with the supplementary material. It was promised for after acceptance, but this seems hard to justify, since the code exists already, and it could be useful for a thorough evaluation of the paper.\n\nWhile NeurIPS does encourage submitting code, doing so is not a requirement and it seems unfair to penalize us for failing to do something that is not required. Moreover, submitting code is not without risk: unintentionally failing to completely anonymize the code can result in an immediate rejection.\n\n> 3. Most of the figure panels should be significantly larger – Figure 7 is a particularly bad example. \n\nAgreed. We will improve the figures as suggested.\n\n> 1. The permutation invariance discussed around eq. (2) assumes Euclidean space [...] This seems overly restrictive since a typical animal's habitat is unlikely to feature a flat 2d plane [...] Can the authors comment on how their method might adapt to such cases?\n\nWe thank you for raising this question! To the best of our knowledge, previous modeling work typically assumes flat 2d Euclidean space.  It is certainly an interesting point of discussion. Grid cells have been observed in bats navigating in 3D (Yartsev & Ulanovsky 2011 and Ginosar et al. 2021) and at least one paper has studied grid cells in higher dimensional spaces (Klukas, Lewis, Fiete 2020). However, in this work, we follow the majority of previous modeling work that focuses on flat 2D Euclidean space. We leave 3D for future work. We have added this to our discussion section.\n\n> Figure 6, panel c: what non-linear dimensionality reduction technique was used?\n\nWe qualitatively followed the methodology used by seminal experimental papers examining the topology of neural representations e.g., Chaudhuri et al. Nature neuroscience 2019 and Gardner et al. Nature 2022: we used principal components analysis to 6 dimensions followed by a non-linear dimensionality reduction to 3 dimensions (in our case, spectral embedding). Similar results are obtained if one uses Isomap.  We will add an explanation and citations to these works in the appropriate section.\n\n"}, {"review_id": "NBTdD6J0gz", "reviewer": "Reviewer_cRXg", "summary": "The paper proposes a computational framework for the emergence of grid cells in the mammalian cortex through self-supervised learning. The learning objective is formulated combining requirements of path independence for location code, error-correcting coding, efficient coding. Validity of the approach is demonstrated through numerical experiments on a recurrent neural network. Resulting cells reproduce important properties of grid cells observed in biology: cells organized into modules with common spatial frequency and orientation; modules exist for a range of frequencies; cells in a single module regularly tile the space. ", "questions": "Do all learning batches start from the same implied position x? Otherwise it's hard to imagine coordination of position codes between batches.\n\nReader needs at least some elaboration on updates of W.\n\nWhat's a ratemap? Is it a reception field?\n\nI could not decipher figure 5a, left.\n\nline 199-200: do you mean arena is larger in testing than in training? how can the value for the low spatial frequency be found in testing?\n\n", "limitations": "See weaknesses", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The paper formulates a principled and biologically meaningful optimization problem and arrives at a representation that manifests important properties of grid cells. \n\nThe role of each component of the objective function is studied experimentally.\n\nRich future directions outlined in Discussion section.\n", "weaknesses": "Authors do not discuss relation of their work to the literature arguing for grid cells role in predictive representation (e.g., Stachenfeld et al, 2017; Momannejad 2020). \n\nThe biological plausibility of proposed learning procedure is also not discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Author Response\n\nWe thank Reviewer crXg for their time, positive feedback and high score. Replying to each comment, suggestion and/or question in turn:\n\n> Authors do not discuss relation of their work to the literature arguing for grid cells role in predictive representation (e.g., Stachenfeld et al, 2017; Momannejad 2020).\n\nStachenfeld et al, 2017; Momannejad 2020 are based on the successor representation (in reinforcement learning) for learning predictive representations, resulting in a place cell-like population code as the successor map) are a normative model for place cell formation. Given a place cell representation, they show that eigendecomposition of their place cell representation results in vectors that are periodic. This point is essentially the same as Dordek et al. 2016 (non-negative PCA of place cells results in grid cells), which was subsequently used by Sorscher et al. 2019 to design better supervised targets for recurrent neural networks. We will add a connection to this line of work with appropriate citations to our Discussion. Thank you for helping us improve the paper!\n\n> The biological plausibility of proposed learning procedure is also not discussed.\n\nTo clarify, are you inquiring about the biological plausibility of the data (i.e. the trajectories and their permutations), of the network (i.e. the architecture), of the learning algorithm (i.e. backpropagation), or of something else?\n\n> Reader needs at least some elaboration on updates of W.\n\nCould you please clarify what additional information you’re looking for? W is a square matrix output by an MLP whose inputs are 2D cartesian velocity. We’d be happy to provide whatever additional information once we understand.\n\nCould the reviewer provide us with more specific questions or doubts about W?\n\n> What's a ratemap? Is it a reception field?\n\nIndeed, a ratemap is a receptive field over spatial position. Specifically, one ratemap shows one neuron’s (average) activation values at different spatial locations. A ratemap is computed by partitioning physical space into square bins (typically ~5 cm by 5 cm) and then computing the neuron’s average activation, averaged over all instances that the animal/network passes through the spatial bin.\nRatemaps are a standard tool in the neuroscience of spatial navigation, used to visualize the spatial tuning of neurons. We will add a small section in the appendix describing the construction of ratemaps.\n\n> Do all learning batches start from the same implied position x? Otherwise it's hard to imagine coordination of position codes between batches.\n\nYes, all batches start from the same implied position. We have added this clarification in the main text.\n\n> line 199-200: do you mean arena is larger in testing than in training? how can the value for the low spatial frequency be found in testing?\n\nYes, the testing trajectories are much longer than the training trajectories.\n"}, {"review_id": "80u0DxaSLN", "reviewer": "Reviewer_1Nr3", "summary": "Shows how grid modules of grid cells can emerge as solutions to a self-supervised learning framework, implemented as a recurrent neural network.\nThey take insights from Continuous Attractor models (velocity dependent weights), Dorrell et al (2023) representation theory (path invariance), and ideas about efficient coding (Sreenivasan & Fiete), and put them all within a single SSL framework, based on three loss functions for RNNs - maximising separation of distinct locations, path invariance and high capacity for encoding locations.   \nThe simulation results show modules of grid cells are formed. ", "questions": "Given the eventual aim to understand what is necessary for the emergence of grids in RNNs, should Sorscher et al have been cited earlier? Or the aim to understand the effect of environmental manipulations such as rewards, should Nayebi et al (NeurIPS 2021) be cited? \nFor the eventual aim of pushing this framework towards machine learning in general domains such as vision, audition etc, what will correspond to velocity, given the reliance on a framework based on path integration? \n", "limitations": "yes", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The single theoretical framework for explaining various aspects of the grid code is a strength", "weaknesses": "The advance on previous work is not so clear, given that each aspect has been presented previously, with representation theory covered by Dorrell et al., nice analysis of the emergence of grid codes in RNNs in Sorscher et al., and coding efficiency in Sreenivasan and Fiete, and using the basline continuous attractor model of RNNs with velocity dependent weights for path integration.  ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer 1Nr3 for their time and feedback and suggestions. Replying to each comment, suggestion and/or question in turn:\n\n> The advance on previous work is not so clear, given that each aspect has been presented previously, with representation theory covered by Dorrell et al., nice analysis of the emergence of grid codes in RNNs in Sorscher et al., and coding efficiency in Sreenivasan and Fiete, and using the basline continuous attractor model of RNNs with velocity dependent weights for path integration.\n\nWe thank the reviewer for the opportunity to better explain why our work is a significant and novel contribution. **_We devoted our entire Global Response to answering this question_**. We refer the reviewer to this response. In short, there are four approaches (that we identify and that you correctly reiterate) and each has at least 1 significant limitation. We use insights from all four perspectives and *solve all their limitations* by contributing a unified model that elegantly combines the strengths of these approaches. Please let us know if you have any follow up questions!\n\nAdditionally, the difference between supervised and self-supervised learning is _significant_: In the supervised setting, for each time point, the loss function has access to the absolute position of the agent in the environment. In the self-supervised case, for each pair of positions, the loss function has 1 coarse-grained bit of information: Whether the current position is within $\\sigma_x$ of the previously considered position. Consequently, showing a successful solution in self-supervised learning is a significant advance beyond a supervised learning solution.\n\n> Given the eventual aim to understand what is necessary for the emergence of grids in RNNs, should Sorscher et al have been cited earlier?\n\nWe’re not quite sure we understand this suggestion. We cite Sorscher et al. 2019 on Page 1 in the second paragraph, after introducing grid cells in the first paragraph, and again on Page 2. Could you please clarify?\n\n> Or the aim to understand the effect of environmental manipulations such as rewards, should Nayebi et al (NeurIPS 2021) be cited?\n\nNayebi et al. 2021’s primary contribution is regressing candidate artificial neural networks’ activations against mouse electrophysiology recordings from medial entorhinal cortex. As you correctly state, their Section 5 also studies how trajectory statistics (e.g., a preference of moving to a particular location, understood to be a “reward” location) affect the artificial neural population similarly to reward-driven remapping in biological neural populations. Our paper does not regress artificial and biological representations, nor does our paper study how those regressions are affected by trajectory statistics, and so respectfully, we feel Nayebi et al. 2021 is a bit distant to merit a citation.\n\n\n> For the eventual aim of pushing this framework towards machine learning in general domains such as vision, audition etc, what will correspond to velocity, given the reliance on a framework based on path integration?\n\nThat’s a great question and one that we see as wide-open! In vision, one might imagine an agent following some “path” in view-space i.e. moving in a way such that the agent returns to the agent’s original vantage point. This can be related to visual saccades produced by primate vision. In audition, one might take inspiration from Aronov, Nevers & Tank Nature 2017, although the analogy is admittedly less clear. This is why we suggest these other modalities as possible future directions.\n"}]

Author rebuttal:
## Global Response to Reviewers

Here we take the opportunity to better motivate our paper, and to explain why we view it as a novel and significant contribution. In short, multiple approaches have been taken to understand grid cells, but each contains at least one limitation. Our paper combines their strengths to simultaneously solve their limitations.


Broadly, there are four approaches to understanding grid cells:


**Approach 1: Basis Function Optimization** (e.g. Dorrell et al. 2023)

**Perspective:** Given functions of space constructed by linear combinations of sine and cosine basis functions, provides loss function(s) that makes these functions look grid-like.

**Limitation(s):**

1. The central question is how one should learn a self-consistent representation of space updated by velocity inputs. This learning of an attractor with a continuum of fixed points is known to be a hard problem, covered in both classic papers (e.g. Seung 1996) and recent reviews (e.g. Khona and Fiete 2022). By explicitly starting with functions of space, Dorrell et al. 2023 avoid this problem.
2. There is no neural network component - this approach is pure basis function optimization
3. The assumed sine and cosine basis functions lie in the correct function class to learn periodic representations and thus provide an advantageous inductive bias that artificial networks lack.


**Approach 2: Supervised Reconstruction of Supervised Spatial Targets by recurrent networks** (e.g., Banino et al 2018, Sorscher et al. 2019)

**Perspective:** Supervised learning on spatial target functions sometimes leads to grid-like representations.

**Limitation(s):**

1. By assuming access to supervised learning targets encoding privileged absolute spatial position, these approaches again bypass the central question. If biological agents already had their own absolute spatial position, they wouldn’t need to learn how to track their own absolute spatial position. 
2. The design choices between our work and previous works are qualitatively different. Previous supervised learning papers crafted supervised targets to insert grid-like representations into the networks, sometimes contradicting known biological properties of place cells (Schaeffer et al. NeurIPS 2022). In contrast, our learning setup is motivated by first-principles properties of the grid representation identified by previous theory work.
3. Previous supervised papers claimed extreme generality, far beyond what the numerical results support. Schaeffer et al. 2022 showed that their results are tuned and their theory only sometimes predicts empirical results. The criticism of supervised path-integrating networks is this mismatch between claims and empirical results. We were very careful to write our paper in the language of “There exists” and not in the language of “For all” and have avoided using sweeping terms such as “generically” to describe our results.
4. Lastly, our results are different. Despite best efforts, previous supervised path-integrating recurrent neural networks did not learn multiple grid modules. In contrast, our networks do learn multiple modules. Additionally, our networks generalize outside the training distribution, whereas supervised networks fare poorly away from the training arena box where supervised targets have not been defined.

Thus, this line of work is not about the task of path integration, but rather about the shape of targets to obtain grid-like tuning curves. Stachenfeld et al. 2017 and Momennejad et al. 2020 are related in that given a place cell representation (derived from a predictive approach), they show that eigendecomposition of their place cell representation results in vectors that are multiperiodic. This point is essentially the same as Dordek et al. 2016 (non-negative PCA of place cells results in vectors that are grid-like), which was subsequently used by Sorscher et al. 2019 to design better supervised targets for recurrent neural networks and was analytically shown.

**Approach 3: Coding Theory** (e.g., Srinivasan and Fiete 2011)

**Perspective:** Given the grid cell representation, what are the unique coding theoretic properties that this representation provides?

**Limitation(s):**

Lists many of the coding-theoretic properties of grid cells, but doesn’t test the necessity or sufficiency of those properties for generating grid cells via an optimization problem or provide explicit loss function(s).


**Approach 4: Continuous Attractors** (e.g., Burak and Fiete, 2009)

**Perspective:** What is the physical principle underlying grid cells? How do we build a mechanistic model that reproduces grid-like patterns using this principle?

**Limitation(s):**

1. Does not provide an optimization problem that leads to grid cells, rather builds a mechanistic model by hand. Neuroscientists would say that there is no “normative” answer to why mammals have learned this particular representation over an evolutionary time scale.
2. Produces only a single module of grid cells.
3. This approach relies on the principle of pattern formation and continuous attractor dynamics: This needs the interaction kernel between neurons to be fine-tuned to produce a continuum of fixed points.


Our work uses insights from all four approaches and solves their limitations by contributing a unified model that elegantly combines their strengths. *We provide an optimization problem motivated from a normative perspective that does not require privileged access to supervised position information, such that networks trained on the optimization problem reliably learn multi-modular periodic representations and generalize significantly beyond their training distribution.*


Meta-review: The paper shows that a self-supervised learning objective can generate a multi-modular grid cell code for representing space. All reviewers agree that the paper tackles an important question and makes a valuable contribution. One of the main concerns raised during the review and discussion period was how robust this finding is. While this issue was not fully resolved by experiments, the paper still appears to be a valuable contribution and code will be available for others to build upon the work.
Decision: Accept (poster)