Title: Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells
Venue: NeurIPS 2023
Accepted: yes
Keywords: self-supervised learning

unsupervised learning

grid cells

neuroscience

systems neuroscience

representation learning
Abstract: To solve the spatial problems of mapping, localization and navigation, the mammalian lineage has developed striking spatial representations. One important spatial representation is the Nobel-prize winning grid cells: neurons that represent self-location, a local and aperiodic quantity, with seemingly bizarre non-local and spatially periodic activity patterns of a few discrete periods. Why has the mammalian lineage learnt this peculiar grid representation? Mathematical analysis suggests that this multi-periodic representation has excellent properties as an algebraic code with high capacity and intrinsic error-correction, but to date, synthesis of multi-modular grid cells in deep recurrent neural networks remains absent. In this work, we begin by identifying key insights from four families of approaches to answering the grid cell question: dynamical systems, coding theory, function optimization and supervised deep learning. We then leverage our insights to propose a new approach that elegantly combines the strengths of all four approaches. Our approach is a self-supervised learning (SSL) framework - including data, data augmentations, loss functions and a network architecture - motivated from a normative perspective, with no access to supervised position information. Without making assumptions about internal or readout representations, we show that multiple grid cell modules can emerge in networks trained on our SSL framework and that the networks generalize significantly beyond their training distribution. This work contains insights for neuroscientists interested in the origins of grid cells as well as machine learning researchers interested in novel SSL frameworks.

Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells

Abstract

To solve the spatial problems of mapping, localization and navigation, the mammalian lineage has developed striking spatial representations. One important spatial representation is the Nobel-prize winning grid cells: neurons that represent selflocation, a local and aperiodic quantity, with seemingly bizarre non-local and spatially periodic activity patterns of a few discrete periods. Why has the mammalian lineage learnt this peculiar grid representation? Mathematical analysis suggests that this multi-periodic representation has excellent properties as an algebraic code with high capacity and intrinsic error-correction, but to date, synthesis of multi-modular grid cells in deep recurrent neural networks remains absent. In this work, we begin by identifying key insights from four families of approaches to answering the grid cell question: dynamical systems, coding theory, function optimization and supervised deep learning. We then leverage our insights to propose a new approach that elegantly combines the strengths of all four approaches. Our approach is a self-supervised learning (SSL) framework - including data, data augmentations, loss functions and a network architecture - motivated from a normative perspective, with no access to supervised position information. Without making assumptions about internal or readout representations, we show that multiple grid cell modules can emerge in networks trained on our SSL framework and that the networks generalize significantly beyond their training distribution. This work contains insights for neuroscientists interested in the origins of grid cells as well as machine learning researchers interested in novel SSL frameworks.

1 Introduction

Spatial information is fundamental to animal survival, particularly in animals that return to a home base. Mammalian brains exhibit a menagerie of specialized representations for remembering and

trajectory

spike

cell 1

cell 2

cell 3

0.03

30

40

50

60

70

80

90

100

Grid spacing (cm)

0

38.8

48.4

65.0

98.4

rat's path

inside box

places where

cell fired

a

b

c

d

multi-modular representation

uniform phase coverage

firing fields

*

reasoning about space. Of these, grid cells (whose discovery earned a Nobel prize) are a striking and important example. Each grid cell fires in a spatially periodic pattern, at every vertex of a regular triangular lattice that tiles all two-dimensional environments [26] (Fig. 1ab) . This neural code for space appears bizarre since position is a local and aperiodic variable, but the neural representations are non-local and periodic. Why has the mammalian brain learnt this peculiar representation?

Broadly, there have been two directions to answering this question. The first direction uses theoretical analysis to deduce the mathematical properties that the grid code possesses [21, 56, 64, 40, 58]. This direction yields many insights about the coding properties of grid cells, but has not validated these insights by proposing an optimization problem under which grid cells emerge in deep recurrent neural networks. The second direction trains deep recurrent neural networks (RNNs) [16, 5, 54] with the goal of identifying the optimization problem that grid cells solve. However, recent work [48, 51] showed that these models require highly specific, hand-engineered and non-biological readout representations to produce grid cells; removing these representations results in loss of grid cells even though the optimization problem is equally well-solved.

In this paper, we make three contributions. First, we extract critical insights from prior models of the grid code spanning diverse fields including coding theory, dynamical systems, function optimization and supervised deep learning. Second, we use those insights to propose a new self-supervised learning (SSL) framework for spatial representations, including data, data augmentations, loss functions and network architecture. Third, we show that multi-modular grid cells can emerge as optimal solutions to our SSL framework, with no hand-engineered inputs, internals or outputs. We also perform ablations to show under what constraints grid cells do and do not emerge.

By creating a minimal, theoretically-motivated and developmentally plausible learning problem for path-integrating deep recurrent networks through SSL, we close the loop from the discovery of biological grid cells, to the study of their theoretical properties, to their artificial genesis.

2 Background

Grid cells in the mammalian medial entorhinal cortex Grid cells [26] are neurons found in the mammalian medial entorhinal cortex tuned to represent the animal's spatial location as it traverses 2D space. Each grid cell fires if and only if the animal is spatially located at any vertex of a regular triangular lattice that tiles the explored space, regardless of the speed and direction of movement through space (Fig. 1ab). As a population, grid cells exhibit several striking properties that provide support for a specialized and modular circuit: Grid cells form discrete modules (clusters) such that all cells within a module share the same spatial periodicity and orientation, while different modules express a small number of discretely different spatial periods (Fig. 1c) [59]. Grid cells in the same module (i.e., sharing the same lattice periodicity) exist for all phase shifts [68] (Fig. 1d).

Self-Supervised Learning (SSL) and its applications in Neuroscience SSL is a subset of unsupervised learning in which data is used to construct its own learning target [2, 13, 27, 61, 10, 11, 14, 69, 19]. Broadly, SSL methods belong to one of three families: deep metric learning, self-distillation or canonical correlations analysis; we refer the reader to a recent review [4]. SSL is increasingly gaining popularity as a normative approach in neuroscience since SSL mitigates the need for large-scaled supervised data that biological agents do not possess. SSL has been used in vision to explain the neural representations of ventral visual stream [32, 70], dorsal visual stream [41] and an explanation for the emergence of both [3]. In this paper, we provide the first model of SSL applied to the mammalian brain's spatial representations with an emphasis on sequential data and recurrent networks that to the best of our knowledge is not well explored in either the machine learning or neuroscience literature.

3 Notation and Terminology

We consider temporal sequences of three variables: spatial positions x t ∈ R 2 , spatial velocities v t ∈ R 2 and neural representations on the surface of the positive orthant of the N -dimensional sphere g t ∈ S N -1 ≥ 0 def = G . Note that any neural representation g ∈ G has unit norm || g || 2 = 1 . We interchangeably refer to representations as states, codes or embeddings. We denote sequences of any of these variables from time t = 1 to T as ( · 1 , · 2 , ..., · T ) , e.g., ( v 1 , v 2 , ..., v T ) .

4 Insights from Grid Cell Modeling Approaches

The question of why the mammalian brain uses a high-dimensional, multiperiodic code to encode a low-dimensional, aperiodic variable has led researchers to theoretically analyze the grid code's properties from multiple mathematical perspectives: coding theory, dynamical systems, function optimization, and supervised deep learning. In this section, we extract key insights that suggest a convergence towards a SSL approach.

4.1 Insights from Coding Theory

Many interrelated coding-theoretic properties of grid cells have been pointed out before: Exponential capacity : The coding states form a space-filling curve (1D) (Fig. 2a) or space-filling manifold (2D) with well-separated coding states in the full coding volume defined by the number of cells. Linearly many distinct grid modules and therefore linearly many neurons generate exponentially many unique coding states for generic choices of periods [21, 56]. The grid code enables high dynamic-range representation (i.e., large value of range divided by resolution) of space using individual parameters that do not vary over large ranges [21] (Fig. 2d-e). Separation : Excluding the closest spatial locations, the coding manifold for spatial positions is interleaved in the coding volume, such that relatively nearby states are mapped to faraway locations (Fig. 2b) and relatively nearby spatial locations are mapped to faraway states (Fig. 2c); this gives the grid code strong error-correcting properties [56]. Decorrelation : Grid cells maximally decorrelate their representations of space beyond a small scale that is roughly half the period of the smallest module, so that all correlations are roughly equally small beyond that scale [21, 56] (Fig. 2f). Equinorm coding states : The grid coding states are equi-norm and lie on the surface of a hypersphere. Ease of update : The grid code permits carry-free updating arithmetic so that individual modules update their states without communicating with each other, meaning updates can be computed in parallel rather than sequentially [21]. Algebraic : The grid code is a faithful algebraic code [21, 56], in that the grid code for composition of displacements is equal to the composition of grid codes for each of the displacements. This property enables the grid code to self-consistently represent Euclidean displacements (Fig. 2g). Whitened representation : All modules (coding registers or phases) update by similar amounts for a given displacement in real space, as a consequence of module periods being similarly sized (in contrast with fixed-base number system representations, where the "periods" for each register are hierarchically organized in big steps as powers of the base, and the representation is not whitened).

These properties are all intrinsic to the grid code, without regard to other types of spatial neurons, e.g. place cells. The lesson is that the grid code likely originates from coding-theoretic properties, not from predicting place cells (c.f., [5, 55]). The challenge before us is identifying which subset of these partially inter-related mathematical coding-theoretic properties of the grid code - all useful for the function of grid cells - forms the minimally sufficient subset for grid cell emergence.

b

f

d(g(0),g(x))

e

a

c

d

algebraic coding

high-capacity

representation

near(neural space)

far(real space)

n

neural representation

space

n

n

1

2

3

g

f(f(

g

,

v

),

v

) = f(f(g ,

v

),

v

)

2

1

f(

g

,

v

)

f(

g

,

v

)

f(

g

,

v

) = translation operator

2

1

2

1

0

0

0

0

0

R

0

R I

R

0

R I

Φ

1

Φ

2

Φ

2

Φ

1

R

0 R

I

min

σ    (R )

l

near(real space)

far(neural space)

well-separated

coding states

rapid decorrelation

x

g

error(

Φ

)

min

min

2

Φ

1

Φ

2

σ

σ

4.2 Insights from Dynamical Systems

Consider the following general framework for defining a neural code that is dynamically updated on the basis of a velocity signal: An agent starts with some neural representation g 0 ∈ G , then moves along a spatial trajectory given by the sequence of velocities ( v 1 , ..., v T ) . The sequence of neural representations ( g 1 , ..., g T ) is updated based on velocity through the (arbitrary, non-linear and possibly time-dependent) dynamical translational operation f : G × R 2 → G : g t def = f ( g t -1 , v t ) . In order for the neural representations to be good codes for Euclidean space, what properties should f and G possess? The first property is that the representations should be path invariant : g T should be the same regardless of whether the agent took path v 1 , ..., v T or path v π (1) , ..., v π ( T ) for any permutation π : [ T ] → [ T ] . Specifically, we desire that ∀ g , v i , v j , f ( f ( g , v i ) , v j ) = f ( f ( g , v j ) , v i ) . Note that path independence automatically implies that the coding states form a continuous attractor [21], because when the velocity v = 0 , the state should remain constant: ∀ g , f ( g , v = 0 ) = g . Further, the grid cell representation is translationally invariant [39, 31]: representations at different positions are related by a shift operator that only depends on the relative separation between the positions, i.e., the correlational structure 〈 g ( x ) g ( x + v ) 〉 x = f ( v ) , where f is a function only of v . The lessons are thus twofold: the data and loss function(s) should together train the network to learn path invariant representations and continuous attractor dynamics, and the dynamical translation operation of the recurrent neural networks should architecturally depend on the velocity.

4.3 Insights from Spatial Basis Function Optimization

A third approach to explaining grid cells is via optimization of spatial basis functions. The spatial basis function optimization approach is a non-dynamical one that posits a spatial neural code g ( x ) of

a particular parametric form, e.g., g ( x ) def = a 0 + ∑ d a d sin( k d · x ) + ∑ d b d cos( k d · x ) . One can then define a loss function and optimize parameters to discover which loss(es) results in grid-like tuning, possibly under additional constraints [39, 18]. This functional form is used because it reflects the constraints of path-integration: if one path-integrates with action-dependent weight matrices, then one must use that functional form [18]. While sensible, these insights have not yet crossed to deep recurrent neural networks. The challenges are that recurrent networks given velocity inputs are not explicit functions of space, and their representations need not be comprised of Fourier basis functions. We seek to bring such insights, which requires discovering how to train recurrent networks that simultaneously learn a consistent representation of space (i.e., learn a continuous attractor), while at the same time, shape the learnt representations to have desirable coding properties.

4.4 Insights from Supervised Deep Recurrent Neural Networks

Deep recurrent neural networks sometimes learn grid-like representations when trained in a supervised manner on spatial target functions [5, 54, 23, 65, 22, 67], and studies regarding the successor representation in reinforcement learning [57, 42] show similar findings. These results share a common origin based on the eigendecomposition of the spatial supervised targets (putatively: place cells) which results in periodic vectors and was earlier noted in shallow nonlinear autoencoders [17]. This understanding was used by [54] to hand-design better supervised targets.

While high profile, this line of work has fundamental shortcomings. Firstly, because the networks learn to regurgitate whatever the researcher encodes in the spatial target functions, the learned representations are set by the researcher, not by the task or fundamental constraints [49]. Secondly, by assuming the agent already has access to its spatial position (via the supervised targets), this approach bypasses the central question of how agents learn how to track their own spatial position. Thirdly, previous supervised path-integrating recurrent neural networks either do not learn multiple grid modules or have them inserted via implementation choices, oftentimes due to statistics in the supervised targets [54, 49]. Fourthly, supervised networks fare poorly once the spatial trajectories extend outside the training arena box (i.e., where supervised targets have not been defined).

The lesson we extract from this is to remove supervised targets altogether, and instead directly optimize the neural representations via self-supervised learning. Supervised networks pick up whatever statistics exist in the supervised targets, and struggle to generalize beyond their supervised targets, meaning that if we wish to avoid putting grid cells in by hand or if we wish to obtain networks that generalize, we must remove the supervised targets altogether. Previous work lightly explored this direction by introducing unsupervised loss terms, but kept the supervised targets [22, 67]. In this work, we fully commit to this lesson and take a purely SSL approach.

5 Self-Supervised Learning of Representations for Space

We deduce that learning useful representations of space is a SSL problem requiring an appropriate SSL framework (comprising training data, data augmentations, loss functions and architecture). We hypothesize that three loss functions, extracted from the coding, path integration and optimization perspectives synthesized above, when combined with particular data and recurrent network architecture, will generate multiple modules of grid cells. See App. A for experimental details.

5.1 Data Distribution and Data Augmentations

For each gradient step, we sample a sequence of T velocities ( v 1 , v 2 , ..., v T ) , with v t ∼ i.i.d. p ( v ) , then construct a batch by applying B randomly sampled permutations { π b } B b =1 , π b : [ T ] → [ T ] to the sequence of velocities to obtain B permuted velocity trajectories; doing so ensures many intersections between the trajectories exist in each batch (SI Fig. 8b). We feed each trajectory through a recurrent network (Sec. 5.2) with shared initial state g 0 , producing a sequence of neural representations ( g π b (1) , g π b (2) , ..., g π b ( T ) ) . This yields a batch of training data for a single gradient step.

b

looped trajectory

a

start

end

Δx<σ

x

x

σ

g

σ

g

neural representation

space

real space

n

n

n

1

2

3

n

n

n

1

2

Path Invariance

Separation

positive

orthant

v

v

velocity inputs

c

self-supervised

SIC loss

Norm-ReLU

d

x

y

start

end

v

1

v

2

v

3

v

4

v

5

v

6

v

7

v

8

v

9

Architecture

Data

3

5.2 Recurrent Deep Neural Network Architecture

Motivated by our insights from dynamical systems (Sec. 4.2), we use an RNN (SI Fig. 8a) with interaction matrix given by:

where v t ∈ R 2 is the velocity input to the recurrent network and W : R 2 → R N × N for N hidden RNN units. The dynamics of the recurrent network are defined as:

The neural representations g t are enforced to be non-negative and unit-norm via the non-linearity σ ( · ) def = Norm ( ReLU ( · )) = ReLU ( · ) / || ReLU ( · ) || . Constraining representations to be constant norm is a common technique in SSL to avoid representation collapse [4] and also was used in previous grid cell modelling work [67]. By choosing this particular architecture, we ensure that representations at different points in space are related only by a function of velocity, thereby ensuring that the transformation applied to the state is the same so long as the velocity is the same.

5.3 Loss Functions

Separation Loss Neural representations corresponding to different spatial locations, regardless of trajectory, should be well-separated (Fig. 3a). Here, 'different spatial locations" is with respect to spatial length scale σ x > 0 and 'well-separated" is with respect to neural length scale σ g > 0 .

Path Invariance Loss Neural representations corresponding to the same spatial location, regardless of trajectory, should be identical (Fig 3b), i.e., neural representations should be invariant to the particular path the agent takes. If two trajectories are permuted versions of one another such that they begin at the same position and end at the same position (measured with respect to a given spatial length scale σ x > 0 ), the neural representations after both trajectories should be attracted together.

Capacity Loss Neural representations should be as high capacity as possible. That means that given a finite neural representation space, the number of unique spatial positions that the neural representations can uniquely encode should be maximized. In our particular setting, because our representations lie on the hypersphere, we can incentivize high capacity in the following manner:

1m

This capacity loss pushes together all neural embeddings, regardless of path traveled or point in time, encouraging as little of the embedding space to be allocated for the training data as possible.

Conformal Isometry Loss Consider a velocity v t that shifts the neural code from g t to g t +1 . A neural code is conformally isometric if the distance between the neural codes remain constant if v t changes direction (but not magnitude). Motivated by [22, 67], we propose a novel and simpler conformal isometry loss that minimizes the variance in ratios of changes in code space to changes in physical space for small velocities:

where V denotes the variance. Intuitively, L ConIso asks that neural representations change by some constant amount for the same magnitude of velocity, regardless of the velocity's direction. Our overall self-supervised loss is then a weighted combination of these four losses:

We term this framework Separation-Invariance-Capacity ( SIC ). This SIC framework is naturally suited to the setting where neural representations are generated by a deep recurrent network which receives velocity inputs. Unlike previous (supervised) approaches, our SIC framework requires no information about absolute spatial position, only relative spatial separation, and does not require any tuned supervised target, thus mitigating the shortcomings of previous works identified recently [48].

6 Experimental Results

6.1 Emergence of multiple modules of periodic cells

Under our SIC SSL framework, we find that trained networks learn neurons with spatially multiperiodic responses. In each run, most neurons in the network exhibit periodic tuning. Moreover, the network clearly exhibits a few discrete periods. We used grid search over hyperparameters with roughly 1 / 2 an order of magnitude variation for key hyperparameters ( σ x , σ g , λ Sep , λ Inv , λ Cap ) and found that multiple modules robustly emerge across this range. Computational limitations prevented us from sweeping more broadly. Example units from a run are shown in Fig. 4. Below, we characterize different facets of the results.

6.2 Generalization to input trajectory statistics

A drawback in previous (supervised) deep learning grid cell works was poor generalization when the networks are evaluated in arenas larger than their training arenas. Grid cells in our networks

σ

c

1m

1.5m

b

d

e

training speed

distribution

eval speed

distribution

count

speed (m/s)

speed (m/s)

d(g(0),g(x))

cum. probability

0

1

a

2m

spatial separation (m)

temporal separation

(timesteps)

neural space distance

x

σ

g

(m)

0.

0

0

0.

2

Room trajectories

1.5

1.0

0.5

0.0

0.5

1.0

Room trajectories

:

T:

generalize to arenas many times larger than the distances explored during training (Fig. 5c-e) (training trajectories extend < 1.2m from the starting position (Fig. 5a)). Secondly, the training trajectories were drawn i.i.d. from a uniform velocity distribution, Fig. 5b (left), while evaluation trajectories were constructed to be smoother and avoid the walls of a confined arena (Fig. 5b, left). As found in experiments, the periodicities of the emergent grid cells remained the same, regardless of the size of the arena (Fig 5c-e).

6.3 State space analysis of trained RNNs

We turn to a more-detailed characterization of the grid cells in each module and their cell-cell relationships/population states. For a big enough set of cells for comparison with experimental results, we needed sufficiently many co-modular cells and so used the network corresponding to Fig. 7a, in which most of the units formed grids of the same period.

We found that cells with similar periods were tightly clustered to have essentially identical periods and orientations (Fig. 6a). Moreover, this population tiled the space of all phases (the Fourier PSD contains a hexagonal structure with wavevectors k 1 , k 2 , k 3 ; we defined three non-orthogonal phases satisfying ∑ a =1 , 2 , 3 φ a i = 0 mod 2 π for each unit from its ratemap r ( x ) : φ a i = phase [∫ d x e i k a · x r i ( x ) ] ), exhibiting a uniform phase distribution (Fig. 6b).

c

d

mean fourier power

2

high

low

k 1

k 2

k 3

toroidal state space

a

projections

module period

module orientation

number of cells

number of cells

x π/3

phase coverage

b

k 1

k

k 3

In theoretical models and real data, grid cells exhibit low-dimensional dynamics and the states of all cells in a module lie on a 2D torus. We used spectral embedding [47] to find that the emergent grid cell population states also fell on a toroidal manifold (Fig. 6a). Following [55], we projected the activity manifold onto 3 axes defined by each set of phases calculated for each unit. This revealed the existence of 3 rings, Fig. 6d, confirming that the attractor manifold has a 2d twisted torus topology.

Ablation experiments We also ablated each of the three loss terms, the data augmentation, i.e., trajectory permutations and the hyperparameter σ g to reveal when multiple modules of grid cells changes to a single module and then to a place-cell like code (Fig 7a-f).

7 Discussion & Future Directions

In this work, we extracted insights from previous first principles coding theoretic properties of grid cells and mechanistic models that produce grid-like firing. We then proposed a novel set of self-supervised loss functions that with the architectural constraint of a recurrent neural network can produce multiple modules of grid cells when trained on a spatial navigation task. In doing so, we have highlighted the distinction between separation and capacity for a neural representation and shown that optimizing for capacity leads to efficient representations that are better for generalization to larger amounts of test data.

σ

trajectory permutations ablation

path invariance loss ablation

separation loss ablation

capacity loss ablation

capacity loss ablation +

=0.1

g

capacity loss ablation +

=0.2

g

a

b

c

d

e

f

88888

Max:0.208Max:0.002

Min:0.000

Min:0.000

Max:0.223Max:0.109

Min: 0.000

Min: 0.000

Max:0.027Max:0.181

Min:0.000

Min: 0.000

Max:0.229Max:0.177Max:0.043Max:0.324Max:0.347

Min: 0.000

Min: 0.000

Min: 0.000

Min: 0.000

Min: 0.000

Max:0.350

Min: 0.000

Min: 0.000

Max:0.072

Min: 0.000

Max:0.198

Min: 0.000

Max:0.007

Min: 0.000

Max:0.264

Min: 0.000

Max:0.200

Min: 0.000

Max:0.365Max:0.175

Min: 0.000

Min: 0.000

Max:0.421

Min: 0.000

Max: 0.261

Min: 0.000

Max: 0.002

Max: 0.109

Max:0.130

Max: 0.014Max: 0.234Max: 0.046

Max:0.333

Max:0.042

Max:0.075

Max:0.335

Max:0.304

Min: 0.000

Min: 0.000

Min: 0.000

Min: 0.000

Min: 0.000

Min: 0.000

Min: 0.000

Min: 0.000

Min: 0.000

Min: 0.000

Min: 0.000

.T

Max:0.003

Min: 0.000

Max:0.209

Min: 0.000

Max:0.292

Min: 0.000

Max:0.145

Min: 0.000

Min: 0.000

Min: 0.000

Max:0.263

Min: 0.000

Max:0.325

Min: 0.000

Max:0.204

Min: 0.000

Min:0.000

Max:0.219

Min: 0.000

0.000

Min: 0.000

.:.

0.000

Munit.s

Min: 0.000

Min: 0.000

Min: 0.000

Min:

Min

Unit: 94

Jnit: 95

Unit:97

σ

Looking to the future, we see four broad directions, two in neuroscience and two in machine learning. Firstly, we believe significantly more work is necessary to understand these trained recurrent networks and connect them back to grid cells. Key questions include: What components of the data, augmentations, losses and architecture control the number of modules, the distribution of the module periods, the period ratios, the structure (square or hexagonal) of the lattices? What do the state spaces of these networks look like? Can we reverse-engineer the learnt mechanisms, using mechanistic analyses [50, 30, 37]? Another interesting question is what underlying connectivity leads to discrete modular responses [29]. How do the learnt representations change when exposed to experimental manipulations, e.g., environmental deformations or rewards [33, 34, 35, 28, 8, 9, 7]? What new experiments can these networks suggest in biological agents? Secondly, we see many opportunities in neuroscience broadly. The biological brain cares about high capacity representations, and grid cells have been implicated in many diverse cognitive and non-spatial tasks [15, 1, 6, 45, 63, 44]. How might our grid cell recurrent networks, and more generally, SSL principles be applied to drive computational neuroscience forward?

Thirdly, we are excited to push our SIC framework further towards machine learning. We envision empirically exploring whether our SIC framework can improve performance in other popular SSL domains such as vision and audition, as well as mathematically characterizing the relationship between SIC and other SSL frameworks. Fourthly, one puzzling aspect of our capacity loss is that it can be viewed as minimizing the entropy of the neural representations in representation space, which flies against common information theoretic intuition arguing for maximizing the entropy of the representations [66, 43, 62, 36, 53, 52]. In some sense, the capacity loss is akin to a 'dual" minimum description length problem [25]: instead of maximizing compression by having a fixed amount of unique data and trying to minimize the code words' length, here we maximize compression by having fixed length code words and trying to maximize the amount of unique data. How can this apparent contradiction be resolved?

References

Dmitriy Aronov, Rhino Nevers, and David W. Tank. Mapping of a non-spatial dimension by the hippocampal-entorhinal circuit. Nature , 543(7647):719-722, March 2017. Number: 7647 Publisher: Nature Publishing Group.

Philip Bachman, R Devon Hjelm, and William Buchwalter. Learning Representations by Maximizing Mutual Information Across Views. In Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019.

Shahab Bakhtiari, Patrick J. Mineault, Tim Lillicrap, Christopher C. Pack, and Blake Aaron Richards. The functional specialization of visual cortex emerges from training parallel pathways with self-supervised predictive learning. October 2021.

Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, et al. A cookbook of self-supervised learning. arXiv preprint arXiv:2304.12210 , 2023.

Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel, Martin J. Chadwick, Thomas Degris, Joseph Modayil, Greg Wayne, Hubert Soyer, Fabio Viola, Brian Zhang, Ross Goroshin, Neil Rabinowitz, Razvan Pascanu, Charlie Beattie, Stig Petersen, Amir Sadik, Stephen Gaffney, Helen King, Koray Kavukcuoglu, Demis Hassabis, Raia Hadsell, and Dharshan Kumaran. Vector-based navigation using grid-like representations in artificial agents. Nature , 557(7705):429-433, May 2018.

Xiaojun Bao, Eva Gjorgieva, Laura K Shanahan, James D Howard, Thorsten Kahnt, and Jay A Gottfried. Grid-like neural representations support olfactory navigation of a two-dimensional odor space. Neuron , 102(5):1066-1075, 2019.

Jacob LS Bellmund, William De Cothi, Tom A Ruiter, Matthias Nau, Caswell Barry, and Christian F Doeller. Deforming the metric of cognitive maps distorts memory. Nature human behaviour , 4(2):177-188, 2020.

Charlotte N Boccara, Michele Nardin, Federico Stella, Joseph O'Neill, and Jozsef Csicsvari. The entorhinal cognitive map is attracted to goals. Science , 363(6434):1443-1447, 2019.

William N Butler, Kiah Hardcastle, and Lisa M Giocomo. Remembered reward locations restructure entorhinal spatial maps. Science , 363(6434):1447-1452, 2019.

Mathilde Caron, Piotr Bojanowski, Julien Mairal, and Armand Joulin. Unsupervised PreTraining of Image Features on Non-Curated Data. arXiv:1905.01278 [cs] , August 2019. arXiv: 1905.01278.

Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised Learning of Visual Features by Contrasting Cluster Assignments. arXiv:2006.09882 [cs] , January 2021. arXiv: 2006.09882.

Rishidev Chaudhuri and Ila Fiete. Computational principles of memory. Nature neuroscience , 19(3):394-403, 2016.

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Contrastive Learning of Visual Representations. arXiv:2002.05709 [cs, stat] , June 2020. arXiv: 2002.05709.

Xinlei Chen and Kaiming He. Exploring Simple Siamese Representation Learning, November 2020. arXiv:2011.10566 [cs].

Alexandra O. Constantinescu, Jill X. O'Reilly, and Timothy E. J. Behrens. Organizing conceptual knowledge in humans with a gridlike code. Science , 352(6292):1464-1468, June 2016. Publisher: American Association for the Advancement of Science.

Christopher J Cueva and Xue-Xin Wei. Emergence of grid-like representations by training recurrent neural networks to perform spatial localization. International Conference on Learning Representations , page 19, 2018.

Yedidyah Dordek, Daniel Soudry, Ron Meir, and Dori Derdikman. Extracting grid cell characteristics from place cell inputs using non-negative principal component analysis. eLife , 5:e10094, March 2016. Publisher: eLife Sciences Publications, Ltd.

William Dorrell, Peter E Latham, Timothy EJ Behrens, and James CR Whittington. Actionable neural representations: Grid cells from minimal constraints. International Conference on Learning Representations , 2023.

Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, and Nicu Sebe. Whitening for Self-Supervised Representation Learning. page 10, 2021.

William Falcon and The PyTorch Lightning team. PyTorch Lightning, March 2019.

Ila R. Fiete, Yoram Burak, and Ted Brookings. What Grid Cells Convey about Rat Location. Journal of Neuroscience , 28(27):6858-6871, July 2008. Publisher: Society for Neuroscience Section: Articles.

Ruiqi Gao, Jianwen Xie, Xue-Xin Wei, Song-Chun Zhu, and Ying Nian Wu. On path integration of grid cells: Group representation and isotropic scaling. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems , volume 34, pages 28623-28635. Curran Associates, Inc., 2021.

Ruiqi Gao, Jianwen Xie, Song-Chun Zhu, and Ying Nian Wu. LEARNING GRID CELLS AS VECTOR REPRESENTA- TION OF SELF-POSITION COUPLED WITH MATRIX REPRESENTATION OF SELF-MOTION. page 24, 2019.

Richard J. Gardner, Erik Hermansen, Marius Pachitariu, Yoram Burak, Nils A. Baas, Benjamin A. Dunn, May-Britt Moser, and Edvard I. Moser. Toroidal topology of population activity in grid cells. Nature , 602(7895):123-128, February 2022. Number: 7895 Publisher: Nature Publishing Group.

Peter D Grünwald. The minimum description length principle . MIT press, 2007.

Torkel Hafting, Marianne Fyhn, Sturla Molden, May-Britt Moser, and Edvard I. Moser. Microstructure of a spatial map in the entorhinal cortex. Nature , 436(7052):801-806, August 2005. Number: 7052 Publisher: Nature Publishing Group.

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 9726-9735, Seattle, WA, USA, June 2020. IEEE.

Alexandra T Keinath, Russell A Epstein, and Vijay Balasubramanian. Environmental deformations dynamically shift the grid cell spatial metric. Elife , 7:e38169, 2018.

Mikail Khona, Sarthak Chandra, and Ila R. Fiete. From smooth cortical gradients to discrete modules: spontaneous and topologically robust emergence of modularity in grid cells. Technical report, bioRxiv, January 2022. Section: New Results Type: article.

Mikail Khona, Sarthak Chandra, Joy J Ma, and Ila R Fiete. Winning the lottery with neural connectivity constraints: Faster learning across cognitive tasks with spatially constrained sparse rnns. Neural Computation , pages 1-20, 2023.

Mikail Khona and Ila R Fiete. Attractor and integrator networks in the brain. Nature Reviews Neuroscience , pages 1-23, 2022.

Talia Konkle and George A Alvarez. A self-supervised domain-general learning framework for human ventral stream representation. Nature communications , 13(1):1-12, 2022.

Julija Krupic, Marius Bauza, Stephen Burton, Caswell Barry, and John O'Keefe. Grid cell symmetry is shaped by environmental geometry. Nature , 518(7538):232-235, 2015.

Julija Krupic, Marius Bauza, Stephen Burton, and John O'Keefe. Framing the grid: effect of boundaries on grid cells and navigation. The Journal of physiology , 594(22):6489-6499, 2016.

Julija Krupic, Marius Bauza, Stephen Burton, and John O'Keefe. Local transformations of the hippocampal cognitive map. Science , 359(6380):1143-1146, 2018.

Xin Liu, Zhongdao Wang, Ya-Li Li, and Shengjin Wang. Self-supervised learning via maximum entropy coding. Advances in Neural Information Processing Systems , 35:34091-34105, 2022.

Ziming Liu, Mikail Khona, Ila R Fiete, and Max Tegmark. Growing brains: Co-emergence of anatomical and functional modularity in recurrent neural networks. arXiv preprint arXiv:2310.07711 , 2023.

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 , 2017.

Tzuhsuan Ma. Towards a theory for the emergence of grid and place cell codes . PhD thesis, Massachusetts Institute of Technology, 2020.

Alexander Mathis, Andreas V. M. Herz, and Martin Stemmler. Optimal Population Codes for Space: Grid Cells Outperform Place Cells. Neural Computation , 24(9):2280-2317, September 2012.

Patrick Mineault, Shahab Bakhtiari, Blake Richards, and Christopher Pack. Your head is there to move you around: Goal-driven models of the primate dorsal pathway. In Advances in Neural Information Processing Systems , volume 34, pages 28757-28771. Curran Associates, Inc., 2021.

Ida Momennejad. Learning structures: predictive representations, replay, and generalization. Current Opinion in Behavioral Sciences , 32:155-166, 2020.

Serdar Ozsoy, Shadi Hamdan, Sercan Arik, Deniz Yuret, and Alper Erdogan. Self-supervised learning with an information maximization criterion. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 35240-35253. Curran Associates, Inc., 2022.

Seongmin A Park, Douglas S Miller, and Erie D Boorman. Inferences on a multidimensional social hierarchy use a grid-like code. Nature neuroscience , 24(9):1292-1301, 2021.

Seongmin A Park, Douglas S Miller, Hamed Nili, Charan Ranganath, and Erie D Boorman. Map making: constructing, combining, and inferring on abstract cognitive maps. Neuron , 107(6):1226-1238, 2020.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32, 2019.

Lawrence K Saul, Kilian Q Weinberger, Fei Sha, Jihun Ham, and Daniel D Lee. Spectral methods for dimensionality reduction. Semi-supervised learning , 3, 2006.

Rylan Schaeffer, Mikail Khona, and Ila R Fiete. No free lunch from deep learning in neuroscience: A case study through models of the entorhinal-hippocampal circuit. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022.

Rylan Schaeffer, Mikail Khona, and Ila Rani Fiete. No Free Lunch from Deep Learning in Neuroscience: A Case Study through Models of the Entorhinal-Hippocampal Circuit. August 2022. Pages: 2022.08.07.503109 Section: New Results.

Rylan Schaeffer, Mikail Khona, Leenoy Meshulam, Brain Laboratory International, and Ila Fiete. Reverse-engineering recurrent neural network solutions to a hierarchical inference task for mice. Advances in Neural Information Processing Systems , 33:4584-4596, 2020.

Vemund Schøyen, Markus Borud Pettersen, Konstantin Holzhausen, Marianne Fyhn, Anders Malthe-Sørenssen, and Mikkel Elle Lepperød. Coherently remapping toroidal cells but not grid cells are responsible for path integration in virtual agents. bioRxiv , pages 2022-08, 2022.

Ravid Shwartz-Ziv, Randall Balestriero, Kenji Kawaguchi, Tim G. J. Rudner, and Yann LeCun. An information-theoretic perspective on variance-invariance-covariance regularization, 2023.

Ravid Shwartz-Ziv, Randall Balestriero, and Yann LeCun. What do we maximize in selfsupervised learning?, 2022.

Ben Sorscher, Gabriel C Mel, Surya Ganguli, and Samuel A Ocko. A unified theory for the origin of grid cells through the lens of pattern formation. Advances in Neural Information Processing Systems , page 18, 2019.

Ben Sorscher, Gabriel C Mel, Samuel A Ocko, Lisa M Giocomo, and Surya Ganguli. A unified theory for the computational and mechanistic origins of grid cells. Neuron , 111(1):121-137, 2023.

Sameet Sreenivasan and Ila Fiete. Grid cells generate an analog error-correcting code for singularly precise neural computation. Nature Neuroscience , 14(10):1330-1337, October 2011. Number: 10 Publisher: Nature Publishing Group.

Kimberly L Stachenfeld, Matthew M Botvinick, and Samuel J Gershman. The hippocampus as a predictive map. Nature neuroscience , 20(11):1643-1653, 2017.

Martin Stemmler, Alexander Mathis, and Andreas VM Herz. Connecting multiple spatial scales to decode the population activity of grid cells. Science Advances , 1(11):e1500816, 2015.

Hanne Stensola, Tor Stensola, Trygve Solstad, Kristian Frøland, May-Britt Moser, and Edvard I. Moser. The entorhinal grid map is discretized. Nature , 492(7427):72-78, December 2012. Number: 7427 Publisher: Nature Publishing Group.

Joshua B Tenenbaum, Vin de Silva, and John C Langford. A global geometric framework for nonlinear dimensionality reduction. science , 290(5500):2319-2323, 2000.

Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive Representation Distillation. arXiv:1910.10699 [cs, stat] , January 2020. arXiv: 1910.10699.

Yao-Hung Hubert Tsai, Yue Wu, Ruslan Salakhutdinov, and Louis-Philippe Morency. Selfsupervised learning from a multi-view perspective, 2021.

Simone Viganò, Valerio Rubino, Antonio Di Soccio, Marco Buiatti, and Manuela Piazza. Grid-like and distance codes for representing word meaning in the human brain. NeuroImage , 232:117876, 2021.

Xue-Xin Wei, Jason Prentice, and Vijay Balasubramanian. A principle of economy predicts the functional architecture of grid cells. eLife , 4:e08362, September 2015. Publisher: eLife Sciences Publications, Ltd.

James C. R. Whittington, Timothy H. Muller, Shirley Mark, Guifen Chen, Caswell Barry, Neil Burgess, and Timothy E. J. Behrens. The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation. Cell , 183(5):12491263.e23, November 2020.

Mike Wu, Chengxu Zhuang, Milan Mosse, Daniel Yamins, and Noah Goodman. On mutual information in contrastive learning for visual representations, 2020.

Dehong Xu, Ruiqi Gao, Wen-Hao Zhang, Xue-Xin Wei, and Ying Nian Wu. Conformal isometry of lie group representation in recurrent network of grid cells. arXiv preprint arXiv:2210.02684 , 2022.

KiJung Yoon, Michael A Buice, Caswell Barry, Robin Hayman, Neil Burgess, and Ila R Fiete. Specific evidence of low-dimensional continuous attractor dynamics in grid cells. Nature neuroscience , 16(8):1077-1084, 2013.

Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow Twins: SelfSupervised Learning via Redundancy Reduction, June 2021. arXiv:2103.03230 [cs, q-bio].

Chengxu Zhuang, Siming Yan, Aran Nayebi, Martin Schrimpf, Michael C. Frank, James J. DiCarlo, and Daniel L. K. Yamins. Unsupervised neural network models of the ventral visual stream. Proceedings of the National Academy of Sciences , 118(3):e2014196118, January 2021.

A Experimental Details

Architecture and training data + augmentations

Our code was implemented in PyTorch [46] and PyTorch Lightning [20]. Hyperparameters for our experiments are listed in Table 1. Our code will be made publicly available upon publication.

v

v

velocity inputs

self-supervised

SIC loss

Norm-ReLU

x

y

start

end

v

1

v

2

v

3

v

4

v

5

v

6

v

7

v

8

v

9

a

b

Architecture

Data and Augmentations

v

v

velocity inputs

self-supervised

SIC loss

Norm-ReLU

x

y

start

end

v

1

v

2

v

3

v

4

v

5

v

6

v

7

v

8

v

9

a

b

Architecture

Data and Augmentations

a

b

..

80601

88810

B All Ratemaps

Φ

2

Φ

1

R

0 R

I

a

λ

λ

1

2

x

Φ = 0

Φ = π

Φ = 0

Φ = π

b

C Construction of the grid code

To explain the structure of the grid code, we consider idealized tuning curves in 1d.

Each cell i is defined by its periodicity λ α ∈ R and preferred phase φ i ∈ S 1 . All cells in the same module α have the same periodicity and uniformly tile all allowed phases. For position x ∈ R ,

The tuning curves corresponding to this module can be seen in Fig. 10a.

For this module, we can define φ α ( x ) = 2 π λ α x modulo 2 π . Here φ α ∈ S 1 . So the firing rate can now be written as

All information about the current state of the module is encoded in the single variable φ α . Thus the set of phases { φ α } α ∈ S 1 × ... × S 1 uniquely define the coding states of the set of grid modules.

For 2 modules, defined by { φ 1 , φ 2 } , these states can be visualized as being on a torus S 1 × S 1 , Fig.10b.

D Nonlinear Dimensionality Reduction

For Fig. 6, we qualitatively followed the methodology used by seminal experimental papers examining the topology of neural representations [12, 24]: we used principal components analysis to 6 dimensions followed by a non-linear dimensionality (in our case, spectral embedding) reduction to 3 dimensions. Similar results are obtained if one uses Isomap [60].

Peer reviews:
comments: SUMMARY:
Shows how grid modules of grid cells can emerge as solutions to a self-supervised learning framework, implemented as a recurrent neural network.
They take insights from Continuous Attractor models (velocity dependent weights), Dorrell et al (2023) representation theory (path invariance), and ideas about efficient coding (Sreenivasan & Fiete), and put them all within a single SSL framework, based on three loss functions for RNNs - maximising separation of distinct locations, path invariance and high capacity for encoding locations.   
The simulation results show modules of grid cells are formed.

STRENGTHS:
The single theoretical framework for explaining various aspects of the grid code is a strength

WEAKNESSES:
The advance on previous work is not so clear, given that each aspect has been presented previously, with representation theory covered by Dorrell et al., nice analysis of the emergence of grid codes in RNNs in Sorscher et al., and coding efficiency in Sreenivasan and Fiete, and using the basline continuous attractor model of RNNs with velocity dependent weights for path integration.

QUESTIONS:
Given the eventual aim to understand what is necessary for the emergence of grids in RNNs, should Sorscher et al have been cited earlier? Or the aim to understand the effect of environmental manipulations such as rewards, should Nayebi et al (NeurIPS 2021) be cited? 
For the eventual aim of pushing this framework towards machine learning in general domains such as vision, audition etc, what will correspond to velocity, given the reliance on a framework based on path integration?

LIMITATIONS:
yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-03

comments: SUMMARY:
The paper proposes a computational framework for the emergence of grid cells in the mammalian cortex through self-supervised learning. The learning objective is formulated combining requirements of path independence for location code, error-correcting coding, efficient coding. Validity of the approach is demonstrated through numerical experiments on a recurrent neural network. Resulting cells reproduce important properties of grid cells observed in biology: cells organized into modules with common spatial frequency and orientation; modules exist for a range of frequencies; cells in a single module regularly tile the space.

STRENGTHS:
The paper formulates a principled and biologically meaningful optimization problem and arrives at a representation that manifests important properties of grid cells. 

The role of each component of the objective function is studied experimentally.

Rich future directions outlined in Discussion section.

WEAKNESSES:
Authors do not discuss relation of their work to the literature arguing for grid cells role in predictive representation (e.g., Stachenfeld et al, 2017; Momannejad 2020). 

The biological plausibility of proposed learning procedure is also not discussed.

QUESTIONS:
Do all learning batches start from the same implied position x? Otherwise it's hard to imagine coordination of position codes between batches.

Reader needs at least some elaboration on updates of W.

What's a ratemap? Is it a reception field?

I could not decipher figure 5a, left.

line 199-200: do you mean arena is larger in testing than in training? how can the value for the low spatial frequency be found in testing?

LIMITATIONS:
See weaknesses
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work reviews some of the issues with existing models of grid cells (cells in mammalian brains that fire when the animal is located at the vertices of a hexagonal grid) and suggests a new model based on recurrent neural networks (RNNs). The model is self-supervised, eliminating the worry that the structure of the readout in a supervised task could influence the outcome. Besides leading generically to grid-like firing, the model also exhibits multiple grid scales organized in modules such that the scale is the same within each module but the phase varies.\n\nSTRENGTHS:\n*Originality:*\nThe paper suggests a way for grid cells to emerge from a self-supervised learning (SSL) paradigm, in contrast to previous work which works mostly in a supervised regime.\n\n*Quality:*\nThe paper does a good job of surveying prior work and includes a fair amount of simulations.\n\n*Clarity:*\nThe presentation is generally clear.\n\n*Significance:*\nGrid cells are of tremendous interest in neuroscience and there is a considerable volume of work attempting to explain their properties and the reasons behind their existence. This paper provides a novel model for how grid cell may emerge – as a self-supervised means of keeping track of an animal's location in space – and is thus of great interest to neuroscientists.\n\nWEAKNESSES:\n1. The authors present this work as a significant advance over methods based on supervised learning because the latter depend on specific design choices. However, the same seems to be true in the new approach: for instance, while the separation and path-invariance loss are pretty natural, the capacity loss is counter-intuitive, as mentioned even by the authors in the Discussion. Excluding the capacity loss eliminates the multi-scale nature of the solution. Moreover, the emergence of grid cells is sensitive to the parameters used in the loss, as shown in Figure 7. It is thus not immediately obvious that the proposed method requires any less fine tuning to lead to grid cells than prior models.\n2. Ideally the code used to run all the simulations would have been included with the supplementary material. It was promised for after acceptance, but this seems hard to justify, since the code exists already, and it could be useful for a thorough evaluation of the paper.\n3. Most of the figure panels should be significantly larger – Figure 7 is a particularly bad example. I understand that space is a limiting factor, but some progress can be made by including fewer ratemaps (don't see the need for more than 3x3 or 4x2 examples of each kind). Also ensure that font sizes don't go below 7 or 8 – when they do, it may be better to just remove the text because it is very inconvenient to read.\n\nQUESTIONS:\n1. The permutation invariance discussed around eq. (2) assumes Euclidean space, where the order of the steps does not change the outcome of a path. This seems overly restrictive since a typical animal's habitat is unlikely to feature a flat 2d plane, but instead is more likely to contain obstacles, hills, tunnels, etc. Can the authors comment on how their method might adapt to such cases?\n2. The motivation for the capacity loss in eq. (9) seems a bit obscure. This was touched upon briefly in the Discussion, but it would be useful for the authors to indicate how they came up with this particular formulation. Also, were other capacity loss functions attempted, and how did they work?\n\nMinor comments:\n* line 127: $f$ is used here with a different meaning from $f$ in, e.g., eqns. (2) or (3), which is a bit confusing\n* Figure 6, panel c: what non-linear dimensionality reduction technique was used?\n\nLIMITATIONS:\nThe authors have adequately discussed limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper shows that recurrent networks trained with a \"self-supervised\" loss leads to units of the internal representations that organize as grid cells. In particular, the paper defines a loss that promotes separations between neural representations encoding different spatial locations, encourages a representation to be invariant to different possible paths taken leading to it's representation, and maximizing the capacity of the representation, and the authors use paired velocities and neural representations as their dataset.\n\nSTRENGTHS:\n- The paper was very clearly written and organized, with nice visuals that supported the text. Moreover, the paper provided a helpful background of previous research that was relevant to the formulation used in the paper.\n- The paper defines a loss that is nicely linked to existing theories for the emergence of grid cells, and shows that this loss, optimized using gradient descent leads emergence of grid cells in artificial recurrent networks.\n- The authors performed ablations of the hyperparameters in their loss to show the dependence of their results on the different terms\n\nWEAKNESSES:\n- How robust are the results to other hyperparameters, like batch-size, learning rate, etc?\n- While the authors criticize previous work that identified the emergence of grid cells using supervised RNN to specifics of the target function (line 55), the authors do not seem to properly explain how their setup differs, and does not lead to similar implicit assumptions. For example, what is the difference between the velocities being used as a supervised signal (which the authors criticize), versus incoporating implicitly into their dataset and self-supervised loss? Are there similar assumptions with respect to the creation of the dataset (e.g having a sufficient number of examples with overlapping positions)\n- Further, the authors assert that \"SSL mitigates the need for large scale supervised data\", but it is unclear to me how different it is to incorporate the velocities as a paired dataset rather than a target variable for a supervised objective. \n- (Minor) The claim in the discussion \"how might ... SSL principles be applied to drive computational neuroscience forward\" seems too general.\n- (Minor) Difficult to read text in Figure 7\n- (Minor) extra italics on t in line 138\n\nQUESTIONS:\n- Do all the internal units have grid cell properties, or only a fraction?\n- Related, but rather than show example units that look like grid cells, are there any metrics that quantify the extent to which a cell is a grid cell?\n- Are there any predictions that can be made, for example, what might happen for animals that explore 3d space?\n- Can the authors comment on the relationship between the toroid's and the encoding of space to make the paper self-contained (e.g. schematics from Fig. 2)?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}