A Representational Model of Grid Cells' Path Integration Based on Matrix Lie Algebras
Authors: Ruiqi Gao
Jianwen Xie
Xue-Xin Wei
Song-Chun Zhu
Ying Nian Wu
Venue:  (2021)
Keywords: grid cells
path integration
representational model
Lie algebras
error correction

Abstract: The grid cells in the mammalian medial entorhinal cortex exhibit striking hexagon firing patterns when the agent navigates in the open field. It is hypothesized that the grid cells are involved in path integration so that the agent is aware of its self-position by accumulating its self-motion. Assuming the grid cells form a vector representation of self-position, we elucidate a minimally simple recurrent model for grid cells' path integration based on two coupled matrix Lie algebras that underlie two coupled rotation systems that mirror the agent's self-motion: (1) When the agent moves along a certain direction, the vector is rotated by a generator matrix. (2) When the agent changes direction, the generator matrix is rotated by another generator matrix. Our experiments show that our model learns hexagonal grid response patterns that resemble the firing patterns observed from the grid cells in the brain. Furthermore, the learned model is capable of near exact path integration, and it is also capable of error correction. Our model is novel and simple, with explicit geometric and algebraic structures. 

Under review as a conference paper at ICLR 2021
A REPRESENTATIONAL MODEL OF GRID CELLS' PATH INTEGRATION BASED ON MATRIX LIE ALGEBRAS
Anonymous authors Paper under double-blind review
ABSTRACT
The grid cells in the mammalian medial entorhinal cortex exhibit striking hexagon firing patterns when the agent navigates in the open field. It is hypothesized that the grid cells are involved in path integration so that the agent is aware of its self- position by accumulating its self-motion. Assuming the grid cells form a vector representation of self- self-possiion, we elucidate minimally simple recurrent model for grid cells' path integration based on two coupled matrix Lie algebras that underlic two coupled rotation systems that mirror the agent's 's self-motion (1) When the agent moves along a certain direction, the vector is rotated by generator matrix. (2) When the agent changes direction, the generator matrix is rotated by another generator matrix. Our experiments show that our model learns hexagonal grid response patterns that resemble the firing patterns observed from the grid cells in the brain. Furthermore, the learned model is capable of near exact path integration, and it is also capable of error correction. Our model is novel and simple, with explicit geometric and algebraic structures.
INTRODUCTION
Imagine walking in the darkness. Purely based on your sense of self-motion, you can gain sense of self-position by integrating the self movement a process often referred to as path integration (Darwin)
1873, Ettenne & Jeffery, 2004] |Hafting et al. 2005 Fiete et al., 2008 McNaughton et al. (0066)
While the exact neural underpinning of path integration remains unclear, it has been hypothesized
that the grid cells (Hafting et al.,
2005: Fyhn et al.)
2008 Yartsev et al. 2011 Killian et al., 2012
Jacobs et al. 2013 Doeller et al.
201O) in the mammalian medial entorhinal cortex (mFCC may be
involved in this process (Gil et al.
2018 Ridler et al. [019 Horner et al.) (016). The grid cells are so
named because individual neurons exhibit striking firing patterns that form hexagonal grid patterns
when the agent (such as a rat) navigates in 2D open field (Fyhn al. 2004; Halting al. 2005
Fuhs & Touretzky 2006 Burak & Fiete 2009 Sreenivasan & Fiete 2011 Blair et al. 2007 Couey et al. [2013] de Almeida al.] 2009 Pastoll et al. ( & Burak 2020). The grid cells
interact with the place cells in the hippocampus (O' Keefe 1979). Unlike a grid cell that fires at the
vertices of lattice, a place cell often fires single few locations.
The purpose of this paper is to understand how the grid cells may perform path integration (or "path integration calculations"). We propose a representational model in which the self-position is represented by the population activity vector formed by grid cells, and the self-motion is represented by the rotation of this vector. Specifically, our model consists of two coupled systems: (I) When the agent moves along a certain direction, the vector is rotated by a generator matrix of a Lie algebra. (2) When the agent changes movement direction, the generator matrix itself is rotated by yet another generator matrix of a different Lie algebra. Our numerical experiments demonstrate that our model learns hexagon grid patterns which share many properties of the grid cells in the rodent brain. Furthermore, the learned model is capable of near exact path integration, and it is also capable of error correction.
Our model is novel and simple, with explicit geometric and algebraic structures. The population activity vector formed by the grid cells rotates in the "mental" or neural space, monitoring the egocentric self-motion of the agent in the physical space. This model also connects naturally to the basis expansion model that decomposes the response maps of place cells as linear expansions of response maps of grid cells (Dordek et al. 2016 Sorscher et al. 2019). Overall, our model provides

---

Under review as a conference paper at ICLR 2021
new conceptual framework to study the grid cell systems in the brain by considering the structure of the intrinsic symmetry (through Lie algebra) of the task which the path integration system is solving.
REPRESENTATIONAL MODEL FOR PATH INTEGRATION
Consider an agent navigating within a squared domain (theoretically the domain can IR²2). Let (z1, 22) be the self-position of the agent in 2D environment. At self- celf-position æ, if the agent makes a displacement ôr along the direction E [0, 2T then the self-position is changed to a + dæ, model, we polar coordinate system (see figure bb by directly using (8, 5r), while only keeping (óx1, 5x22) implicit. (8, ör) is the biologically plausible egocentric representation of self-motion.
where 5æ (ōx11 6x22) (ör cos 0, ör sin 0). In
We assume that the location x in the 2D environment is encoded by the response pattern of a d-dimensional vector v(x) (vi(a)); 1,.... d) with each element representing the firing rate of one neuron when the animal location æ. From the embedding point of view, essentially we embed the 2D domain in R² as 2D manifold in a higher dimensional space IR Locally we embed the 2D local polar system centered x (see figure b)) into F.d so that it becomes local system around exx) (see figure 1e).
population of neurons (e.g., d 200), which correspond
1600 v(x bo +xx)
80
x + 8x
ôr
B(0)rrx
8r
2 v(x)
(b)
(c)
Figure 1: Illustration of the proposed representational model. (a) 2D local polar system centered at for egocentric self-motion, to he embedded in IRd (b) 2D local displacement & and local change of direction 50. (c) Mirroring relations in (b). is mirrored by v(x)). Local displacement är from along direction 0 is mirrored by (B) ör applied (xx). Local change of direction 80 is mirrored by C50 applied to (0)).
2.1 THE PROPOSED REPRESENTATIONAL MODEL: COUPLING TWO ROTATION SYSTEMS
Assuming 8r to be infinitesimal, we propose the following model
va + 5ι)) = (I ((0) + o(ōr),
which parameterizes recurrent neural network (Hochreiter & Schmidhuber 1997h. where I is the identity matrix, and (0) is d-dimensional matrix depending on the direction 0, which will need to be learned.
Rotation. We assume (00) B()T. i.e., skew-symmetrie, so that B(0)ö rotation or orthogonal matrix, due to that (I + (())rr) + B(0)rr) I +0r2)). Because the upper triangle part of (0) is the negative of the transpose of the lower triangle part (the diagonal elements zeros), in fact only need to learn its lower triangle part. The geometric interpretation is that, the agent moves along the direction 8, the vector vxx) is rotated by the matrix (()), while the l2 norm [[:)))] ||2 remains stable (figure 1D). We may interpret [/= = Σd= (z:) as the total energy grid cells, which stable across different locations. From embedding point of view, the local polar system in figure is embedded into d-dimensional sphere in neural response space.
When the agent makes an infinitesimal change of direction from 0 to + 50, B(0) is changed to B(0 + 50). We assume
B(0 + 60) = (I + C80) + o(68),
(2)
where C is d-dimensional matrix, which is also to be learned. We again assumes C -C so that I + C89 is a rotation matrix. The geometric interpretation is that the agent changes direction, B(0) is rotated C.
Equations I and 2 together define our proposed model for path integration, which couples two rotation systems.
2

---

Under review as a conference paper at ICLR 2021
2.2 A MIRROR OF EGOCENTRIC MOTION: PRESERVING LOCAL GEOMETRIC RELATIONS
As a representational model, equations ① and (2) form a mirror in the d-dimensional "mental" (or neural) space for the egocentric motion in the 2D physical space. Importantly, the embedding preserves the local geometric relations of the local polar system.
Let dgv(x)) v( + dx) v(x) be the displacement of vxx) when the agent moves from by år along direction 0. It follows from equation (1) that (z)(() = (B()))r + o()r)))((x). Ignoring high order terms, we obtain (0+500()))
(I + 6 That is, with ör fixed, the local change of v(x) along different A are rotated versions of each other, mirroring the local polar system at æ. See figure for an illustration. As for the angle between vxx) and v(x + 8x), i.e. how much the vector rotates in the neural space as the agent moves in 2D physical space by ôr, we have
Proposition 1 In the above notation, let da be the angle between v(x) and v(x + 6x), we have da = 38r O( where 8r =16x2||, and 3 B (())v( // is independent of 0.
See Supplementary A.I for a proof. That means, the angle da in the d-dimensional neural space proportional to the Euclidean distance ór in the 2D space, and more importantly the angle da is independent of direction 0, i.c., 3 is isotropic.
2.3 HEXAGON GRID PATTERNS
For the learned model, 33 can be much bigger than 1, so that the vector v(x) will rotate back to itself in short distance, causing the periodic patterns of v(x). Moreover, 8 does not depend on the direction of self self-motion, and this isotropic property appears to underly the emergent hexagonal periodic patterns, as suggested by the following result.
The hexagon grid patterns can be created by linearly mixing three Fourier plane waves whose directions are 2r/3 apart. In the following, state theoretical result adapted from Gao al. (2018) that connects such linearly mixed Fourier waves to the geometric property in Proposition in the previous subsection.
Proposition 2 Let e(x) = (exp(i(a,, a)). 1,2,3) where (aj, 1,2,3) three 2D vectors of equal norm, and the angle between every pair of them is 2nr 2T/33. Let v(x)) = Ue(æ). where U is arbitrary unitary matrix, i.e., U I. Let da be the angle between v(x:) and v(a: + ōx)), we have da = 3ôr + 0(8r2), where dr = ||x and B x [|a is independent of the direction of dæ.
See Supplementary A.1 for proof, which relies on the fact that (a,.j 2, 3) forms a tight frame in 2D. Proposition 2 says that the geometric property that emerges from our model as elucidated by Proposition
is is satisfied by the orthogonal mixing of three Fourier plane waves that creates hexagonal grid patterns. We are currently pursuing more general analysis of our model, i.e., equations and
2.4 JUSTIFICATION AS MINIMALLY SIMPLE RECURRENT MODEL
Now justify equation I as minimally simple recurrent model. To start, the general form of the model is v(x + dx) = F F(v(x). 5r, 0). For infinitesimal ör, a first-order Taylor expansion gives
v(x 8x) vxx) + f 0)8r + o(6r),
where the function F(u, 5,0) satisfies F(v. 0, 8) = v, i.e., the vector representation stays the same there is no self-displacement, and f(v, 36 F(v, 1==0. i... the first derivative at 8 = 0.
a
The function f(v, 0) transforms to another vector of the same dimension and the transformation depends on 0. A minimally simple model a linear transformation that depends on i.c., we can assume (xx: dx)) vxx) + ((0)v((=)) + o(ōr), which leads to equation where the linear transformation is B B()) Equation 2 can be similarly justified.
In this paper, assume linear recurrent model for its simplicity and its explicit geometric meaning as rotation. It is important to emphasize that our arguments are not mutually exclusive with the work based non-linear recurrent neural network model (Burak & Fiete 2009 Couey et al., 2013).
2009e
3

---

Under review as a conference paper at ICLR 2021
In fact, our linear rotation model may serve prototype approximation which may help better understand these nonlinear models direction we did not pursue here.
2.5 MATRIX LIE ALGEBRAS AND GROUPS: FROM INFINITESIMAL TO FINITE
For finite, non-infinitesmmal, self sisf-diipl Ar, divide Ar into N steps, so that or Ar/N - 0 as N + oc, and
v Ax) = +I+ B(0) ( v(z) + exp (x))
The above math underlies the relationship between matrix Lie algebra and matrix Lie group (Taylor] 2002). For fixed 8, the set of Mo(Ar) which is both group and a manifold. The tangent space of Mo(Ar) (Ar) at identity is called matrix Lie algebra. (0) is the basis of this tangent space, and is often referred as the generator matrix.
(((()(AAr) for Ar E R forms matrix Lie group,
Similarly for a finite change of direction A0, we obtaín
B + A0) = exp B())
(5)
The set of R(0) exp(CA0 for all 0 E [0, 2T] (with mod 2n addition arithmetics) forms another matrix Lie group, with C' being the generator matrix of its matrix Lie algebra.
Approximation to exponential map. For finite but small Ar, exp (B((BB))) can be approximated by a second-order Taylor expansion
(xp(B(0)Ar) = II+ B(0) B(0) o
(6)
Similarly, exp(CAA) can be approximated by exp( (C20) + CA0 C2 /
Path integration. Now we can cast path integration in the language of Lie group. Specifically, the input includes the initial position (0),, and the self-motions (8(*), Ar(!)) for = 1.... T. Initializing
()) = ()) the vector is updated recurrently according to
v (t) exp( B (((()))(())))))))))))) (r)))))--)))))))) (tt-1))
That is, the vector (()) is rotated by B (B())) according Ar(") geometrically.
Modules. Experimentally, is well established that grid cells are organized in discrete modules (Barry
et al. 2007 Stensola al. 2012) blocks. We thus partition the vector v(x) into K blocks, v
(v)(( (z) k I..... K). Correspondingly the generator matrices B(0)- = diag (0),,, k 11 1,... K)
and C = diag(Ck,
1, ..., K) are block diagonal. This greatly reduces the number of parameters to be learned. Note that each sub-vector (x) is rotated by a sub-matrix B (0), which is in turn rotated by Ct.
Metric. By the same argument as in Proposition I for module k. let Us be the sub-vector. Let dax be the angle between (z) and (x + 8σ), then the angle daz = Sgör, where ór = || and 8 is independent of 6. That is, if the agent moves by ōr, the vector Un (x) rotates by an angle 0g 3h determines the metric or scale of the response maps of the k-th block of grid cells, i.e., it tells us how
fast the sub-vector rotates as the agent moves.
3 INTEGRATION WITH BASIS EXPANSION MODEL
For each vzz). we need to uniquely decode æ. We thus need to integrate the path integration model with the basis expansion model that connects grid cells to place cells. Each place cell fires when the agent is at a specific position. Let Mar(2) be the response map for the place cell associated with position x'_ It measures the adjacency between and x'. A commonly used form of Agr when the agent navigates in the open field is the Gaussian adjacency kernel Ag = exp(- || aar=a'l (=2((((((02)
3.1 BASIS EXPANSION
A popular model that connects place cells and grid cells is the following basis expansion model (or (CS-d model) (Dordek et al. 2016):
Ax (x) = Σ
u Us (x(x) = (w(x), u((x))
(8)
1=1
4

---

Under review as a conference paper at ICLR 2021
where v(x:) = (v)(),, = I.....d) d)", and u(x') (uq,0e,
Ag (at
1..., d)". Here (u,( 1, ...., d) forms set of d basis functions for expanding Ax (a) for all places x', while u(x') is the read-out weight vector for place cell x', and needs to be learned.
Experimental results have shown that the connections from grid cells to place cells are excitatory (Zhang al. 2013] Rowland et al.] 2018). We thus assume that Vi, 2 0 for all and æ.. We can also make v(x)) be non-negative by adding bias term. Please see Supplementary A.4 for details.
U/ (z)
Illustration of ba- sis expansion model (g)))) = >i "4 (xr), where v;(x) is the response map of i-th grid cell, shown at the hottom, which shows 5 different i. Ag (a)) is the re- sponse map of place cell associ- ated with x', shown at the top. which shows 3 different x'. Wa is the connection weight.
Figure 2:
3.2 DECODING, RE-ENCODING, AND ERROR CORRECTION
For a neural response vector v. such as (t) in equation 7). the response of the place cell centered at location x' is (v, u(a: )). We can decode the position a by examining which place cell has the maximal response, i.c.,
c arg ma u(x')). x²
After decoding æt we can re-encode v t v()), which amounts to projecting onto the 2D manifold formed by v(z) for all æ. The set of v(x) forms a codebook, and the projection via re-encoding enables error correction by removing the possible errors or noises in v (see Supplementary A.2).
3.3 UNITARY REPRESENTATION AND HARMONIC ANALYSIS
Underlying the integration of our proposed path integration model (equation D and (equation 2 and the basis expansion model (i.e., equation is the group representation theory.
Our path integration model leads unitary group representation. Let M& (Ar) (()) with finite nonniiniintessmaa)) Ar, let x =
-- (Ar cos 0, sis 0), and M (xx = M)(r)). For each given x, (x:) is an orthogonal matrix. Collectively, (x) forms unitary representation of E RR, i.e., 2D Euclidean group, where the additive group action in IR2 is represented by matrix multiplication. For each element of the matrix, Ma)(2) is function of æ. According to the fundamental theorems of Schur (Zee (0166) and Peter-Weyl (Taylor 2002). if M is an irreducible representation of a finite group or compact Lie group. then (M)( form a set of orthogonal basis functions of æ. This leads to deep generalization of harmonic analysis or Fourier analysis. Let v(x) M (z)u((0) (where we choose the origin 0 as the reference point). The elements of (x)), i.e., (v,(2)) = 1,..,)), are also basis functions of ar. These basis functions serve to expand (Aar (ar), Vaz) that parametrizes the place cells, and these basis functions are generated by the matrix Lie algebras of our path integration model. Thus group representation provides unifying theoretical framework for the two hypothesized roles of grid cells, namely path integration and basis expansion.
In our work, we do not assume each block matrix M& (x) to be irreducible. Thus each learned i (x)) within a block is a linear mixing of orthogonal basis functions in an irreducible representation, and different U, (2)) within the same block are not necessarily orthogonal. However, different es (x(x)) in different blocks are close to orthogonal in our experiments (see Supplementary A.3 for details).
4 LEARNING
The unknown parameters are (1) (v(2)), Va.) (2) (u(x'), Va'). (3) (B(0), (4) C. To learn these parameters, we define a loss function: L Lo + XiLi A where
Lo Eg [ (x)
(v(2),
(10)
L Eg,sa |02 Ax) - expl (x)))
(11)
L2 Eg B(0 A0) cxp(CC)))(()))
(12)
where || ||2 denotes the sum of squares of the elements of the vector matrix. X and X2 chosen so that the three loss terms are of similar magnitudes. In Lo. Ag (x) are given as Gaussian adjacency kernels, and we aim to learn the basis functions v(x:). Ia and La serve to constrain
5

---

Under review as a conference paper at ICLR 2021
the basis functions v(x) so that path integration can be performed based our proposed model (equation and (equation 2). In L1, Ax = =( (Ar cos 8, Ar sin 0).
As the generator matrices for two coupled rotation systems, (0) and C' are both assumed to be skew-symmetric, so that only the lower triangle parts of the matrices need to be learned. We further assume (0) and C' to be block-diagonal with each block corresponding to module, consistent with the experimental observations (Stensola et al.) 2012). For regularization, we add a penalty on so that the connections from grid cells place cells are
|u(=²)) and further assume u(x') excitatory Zhang et al. 2013 Rowland et al. 2018).
We minimize the loss function by stochastic gradient descent, specifically, Adam optimizer (Kingma & Ba 2014). where the expectations are approximated by Monte Carlo samples of (x, x') and x, Ax) See Supplementary B.1 for details of generating Monte Carlo samples for learning.
Unlike previous work on learning basis expansion model (or PCA-based model (Dordek et al. 2016)). we do not constrain the basis functions (xx) = (v;(2)) 1, .... d) to be orthogonal each other. Instead, we constrain them by our path integration model [ and (2) via the loss terms 11 and Lg. In fact, the learned 0/ (a:) within the same block are not orthogonal, although v; (xr from different blocks tend to be orthogonal (see Supplementary A.3) 1.2 constrains (0) to be rotated versions of each other, and it important for the emergence of hexagon grid patterns according to our experiments. is also important for reducing model complexity by constraining (0) for different 0.
Because Ag (a+) contains a whole range of frequencies the Fourier domain, the learned response maps of the grid cells span range of scales too. It is also worth noting that, consistent with the experiential observations, we assume individual place field Ag' (xx) to exhibit a Gaussian shape, rather than Mexican-hat pattern (with balanced excitatory center and inhibitory surround) as assumed in previous basis ex expansion models (Dordek et al. 2016 Sorscher et al. 2019) of grid cells.
EXPERIMENTS
In the numerical experiments, we use a square environment with size 2m x 2m, which is discretized into 80 x 80 lattice. For direction, discretize the circle [0, 2x] into 144 directions. With the above discretizations, we use nearest neighbor linear interpolations for values in between. We use the second-order Taylor expansion (6) to approximate the exponential maps exp( B (0) and exp(CAA)) The local motion Ar and A6 are constrained to local ranges, i.e., Ar is smaller than 3 grids on the lattice and 20 is smaller than 12.5 degrees. For the adjacency kernel A (a) in Lo in equation 10, we use Gaussian kernel with a = 0.07. v(Φ) is of d = 192 dimensions, which is partitioned into K modules, each of which has cells. Our results are robust to the specific choice of the number of modules or the number of cells (see Supplementary B.2
5.1 HEXAGON GRID PATTERNS
Figure 3: Grid firing patterns emerge in the learned network. Every response map shows the firing pattern of one neuron (i.e, one element of v) in the 2D environment. Every row shows the firing patterns of the neurons within the same block or module. (Zoom in for high quality.)
Figure 3 shows the learned firing pattern v(x)= = (v: (x), 1....,) d) over the 80 x 80 lattice of æ. Every row shows the learned units belonging to the same block or module. Regular hexagon grid patterns emerge for both v(x) and uxx). Within each block module, the scales and orientations are roughly the same, but with different phases or spatial shifts. Notably, the emergence of hexagon patterns does not rely on specific block size nor number of blocks. See Supplementary B.2 for the

---

Under review as a conference paper at ICLR 2021
patterns of learned v(x) with different block sizes, learned ((0) and learned u(x). For the learned (0),, each element shows regular sine/cosine tuning over 0.
We further investigate the characteristies of the learned firing rate patterns (i.c., v(x)) using measures adopted from the grid cell literature. Specifically, the hexagonal regularity, scale and orientation of grid-like patterns quantified using the gridness score, grid scale and grid orientation (Langston et al. 2010 Sargolini et al.] 2006). which are determined by taking a circular sample of the auto- correlogram of the response map. All learned patterns exhibit significant hexagonal periodicity in terms of gridness scores (mean 1.08, range 0.60 to 1.57). Specifically, a unit is considered to be grid-like the gridness score exceeds the 95th percentile of null distribution obtained by applying spatial field shuffles to the response map, following the standard procedure in (Hafting et al. 2005 Barry & Burgess 2017). On average, the 95th percentile is 0.35 for all the units. Figure 4 shows six examples of the autocorrelograms of the response maps and the corresponding gridness scores, each of which is from a different module. The grid scales of learned patterns (mean 0.39, range 0.24 to 0.61), shown in Figure 4p, follows multi-modal distribution. The ratio between neighboring modes are roughly 1.44 and [.51, which closely matches the theoretical predictions (Wei et al. 2015 Stemmler al. 2015) and also the empirical results from rodent grid cells (Stensola al. 2012).
The grid orientations of learned patterns, as shown in Figure 44, are also multi-modal distributed, grid cells (Stensola al., 2012). See Supplementary [B. for the
consistent to the observations
detailed spatial profile of every unit of (xx). Collectively, these results reveal striking, quantitative correspondence between the properties of our model neurons and those of the grid cells in the brain.
0.79
155
0.24
0.35
0.53
c
0.29
0.45
1.44
1.51
15.0
mobabblle density
Probability density
12.5
10.0
1.33
0.60
5.0
2.5
0.0
0.0
0.4 0.5 orientation (r)
scale (m)
Figure 4: Model grid cells exhibit modular structure that is consistent with experimental data.(a) Examples of autocorrelograms of the response maps and the corresponding gridness scores, each of which is from different module. (b) Multi-modal distribution of grid scales. The scale ratios closely match the real data (Stensola 2012). (c) Multi-modal distribution grid orientations.
5.2
PATH INTEGRATION AND ERROR CORRECTION
We then examine the ability of the learned system on performing path integration, by recurrently updating (t)) as shown in equation and decoding (t)) to x()) for t = 1,... ...,, T' using equation 9 Re-encoding e)t) < ()) after decoding is adopted. Figure 5 shows an example trajectory of
accurate path integration for T 80. As shown in figure 5p. with re-encoding, the path integration error remains close to zero over a duration of 500 time steps (< 0.01 cm, averaged over 1,000 episodes), although the model trained by the mongle-time-ster loss in equation I1 Without re- encoding, the eгror goes slight higher but still remains reasonable (ranging from 0.0 to 5.4 cm, mean 3.8 cm). The performance of path integration would be improved as the block size becomes larger, i.e., more units or cells in each module (figure 5b.). When block size is larger than 20, path integration almost exact for the time steps tested.
We further assess the ability of error correction of the learned system. Specifically, along the way of path integration, at every time step t, two types of errors are introduced to ((); (1) Gaussian noise or (2) dropout masks, i.e., certain percentage of units are randomly set to zero. Figure 55 summarizes the path integration performance with different levels of introduced errors for T = 100. For Gaussian noise, we use the average magnitude of units in v(x)) as the reference standard deviation (s), i.e.,
s = / u x l2/d. The results show that re-encoding is crucial for eтror correction. Notably, with re-encoding, the path integration works reasonably well even if Gaussian noise with magnitude of is added or 50% units are dropped out at each step, indicating that the learned system is quite robust to different sources of errors.
7

---

Under review as a conference paper at ICLR 2021
0.05
Sort
Bee
5.00
50 papo
0.04
0.03
10
0:02
5.02
0:22
5.00
0:02
70
100
20
45
Figure Iearned model can perform path integration. (a) Black: example trajectory. The decoded selff-ooitions (red) accurately matches the real path. (b) Path integration error over number of time steps. (c) Path integration enror over different block sizes, for 50 and 100 time steps. For (b) and (c), averaged error and + with introduced emrrs.. Left:
I standard deviation band over 1,000 episodes shown. (d) Path integration Gaussian noise. Right: dropout mask.
5.3 ABLATION STUDY
We conduct systematic ablation study to assess the impact of individual components of the model on learning hexagonal grid patterns (measured by gridness score) and the ability to perform path integration (T = 100 time steps). Tablle[suumnn summarizes the results where the model is trained with certain component removed. Specifically. for the emergence of regular hexagonal patterns, all components appear be important, especially L2 which is crucial. In comparison, u(æ) Z 0 is entirely critical. Another observation is that Lo. L1 and penalty on |u(x)]² are crucial for accurate path integration. See Supplementary BB.4 for details.
Table 1: Ablation study. A certain component of the model is removed and the learned model is evaluated in terms of gridness score and path integration error over T 100 time steps. Skew-symmetry is for ( and C.
Lo
L1
Le
Regularize 22x)) Skew-symmetry u(w) 2 0
Gridness
path integration error (cm)
/
-0.10 + 0.15
44.2 : 15.4
X
/
/
0.23 + 0.22
30.6 1 11.8
/
X
/
/
0.32 1 0.31
0.00 1 0.00
X
V
/
-0.07
26.3 = 10.2
X
/
0.00 = 0.00
/
X
0.89 = 0.27
0.00 = 0.00
/
V
1.08 + 0.31
0.00 + 0.00
DISCUSSION AND CONCLUSION
6.1 RELATED WORK ON MODELS OF GRID CELLS
Our work related to several lines of previous research. First, RNN models have been used to model grid cells and path integration. The traditional approach uses simulationbbasee models with hand-crafted connectivity (Zhang) 1996 Burak & Fiete 2009 [oueyy et al. 2013 Pastoll et al.
2013 Agmon Burak 2020). More recently, two pioneering papers (Cueva & Weil 2018 Banino RNN approach to learn the path integration model and discovered that grid-like response pattern could emerge in the optimized network. These results are
et al. 2018) developed an optimization-
further substantiated in following-up research (Sorscher et al.) 2019 Cueva et al., 2020). Compared these studies, our path integration model more explicit, coupling two rotation systems neural space mirror the egocentric motion in physical space. In doing so, results reveals new insights regarding why lattice and in particular hexagonal response patterns may emerge in neural networks trained to perform path integration.
Second, as discussed in Sec our model is naturally connected to the basis expansion models of grid cells. A key ingredient of this line of work (Dordek al. 2016; Sorscher et al.) 2019 Stachenfeld jet al. 2017) is that, under certain conditions, the principal components of the place cell activities exhibit grid patterns. Importantly, work differs from these models in that, unlike PCA, make
8

---

Under review as a conference paper at ICLR 2021
assumption about the orthogonality between the basis functions, and the basis expansion formulation is obtained via group representation from our path integration model. Group representation unifies path integration and basis expansion, which are two roles hypothesized for grid cells. Furthermore, in previous basis expansion models (Dordek et al. 2016 Sorscher et al. 2019), place fields with Mexican-hal patterns (with balanced excitatory center and inhibitory surround) were assumed in order to obtain hexagonal grid firing patterns. However, experimentally measured place fields were instead well characterized by Gaussian functions. Crucially, in our model, hexagonal grids emerge after learning with Gaussian place fields, and there is no need to assume any additional surround mechanisms.
In another related paper, Gao et al. (0188) proposed vector representation of 2D position and matrix representation of 2D displacement. Our work goes beyond that work in revealing two coupled Lie groups and Lie algebra structure, as well as further integrating the path integration model with the basis expansion model.
6.2 CONCLUSION
This paper entertains representation model of grid cells' path integration, which mirrors the egocentric self-motion by coupling two rotation systems. The proposed model can be justified as a minimally simple recurrent model, and has explicit geometric and algebraic structures. As we have shown, this simple model framework leads to system that captures many of the experimentally observed properties of the grid cell system in the brain.
Our path integration model is linear in the vector v(x). but it is non-linear in the displacement Ax. Since the basis expansion model linear in v(x), it may be preferable to have the path integration model linear in v(x) too. The rotation of v(x) is capable of path integration, and the way v(x) rotates can explain the hexagon periodic patterns and the metric of each module. The connection between the two models is based on the group representation theory.
As to modularity, in the context of our path integration model, it means that each sub- vector ia (x) is rotated by a separate generator sub-matrix, or driven by separate recurrent network, so that the dynamics of the sub-vectors are disentangled. This appears to be biologically plausible. We believe modularity is part of the design of the network.
In terms of representation learning, it is common to represent the physical state by a vector, i.c., embedding the physical state into higher dimensional neural space. However, the problem of representing continuous motion in the physical space, or continuous transformation of the physical state, or continuous relation between the states has not received an in-depth treatment. The continuous motion in the physical space often has a native Lie algebra and Lie group structure. Our work provides an explicit representation of the continuous motion and its algebraic structure by matrix Lie algebra and matrix Lie group that act in the neural space. We believe our method can be applied to modeling head direction system well as modeling the motor cortex for representing the continuous motions of arm, hand, pose, etc.
Last but not least, the proposed model can also be used for path planning. See Supplementary Cor the preliminary results).
REFERENCES
Haggai Agmon and Yoram Burak. A theory of joint attractor dynamics in the hippocampus and the entorhinal cortex accounts for artificial remapping and grid cell field-to-fild variability. eLife, e56894, 2020.
Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel, Martin Chadwick, Thomas Degris, Joseph Modayil, et al. Vector-based navigation using grid-like representations in artificial agents. Nature, 557(7705):429, 2018.
Caswell Barry and Neil Burgess. To be grid cell: Shufiling procedures for determining' gridness'. BioRxiv, pp. 230250, 2017.
Caswell Barry, Robin Hayman, Neil Burgess, and Kathryn J Jeffery. Experience-dependenn rescaling of entorhinal grids. Nature neuroscience, 1066):682-644, 2007.

---

Under review as a conference paper at ICLR 2021
Hugh T Blair, Adam C Welday, and Kechen Zhang. Scale-invariane memory representations emerge from moire interference between grid fields that produce theta oscillations: à computational model. Journal of Neuroscience, 27(12):3211-322 2007.
Yoram Burak and Ila R Fiete. Accurate path integration in continuous attractor network models of grid cells. PLoS computational biology, 1000291, 2009.
Jonathan Couey, Aree Witoelar, Sheng-Jia Zhang, Kang Zheng, Jing Ye, Benjamin Dunn, Rafal Czajkowski, May-Britt Moser, Edvard I Moser, Yasser Roudi, al. Recurrent inhibitory circuitry as a mechanism for grid formation. Nature neuroscience, 16(3):318-324, 2013.
Christopher J Cueva and Xue-Xin Wei, Emergence of grid-like representations by training recurrent neural networks perform spatial localization. arXiv preprint arXiv:1803.0777 2018
Christopher J Cueva, Peter Wang. Matthew Chin, and Xue-Xin Wei. Emergence of functional and structural properties of the head direction system by optimization of recurrent neural networks. International Conferences on Learning Representations (ICIR), 2020.
Charles Darwin, Origin of certain instincts, 1873.
Licurgo de Almeida, Marco Idiart, and John E Lisman. The onput-output transformation of the hippocampal granule cells: from grid cells to place fields. Journal of Neuroscience, 29(23): 7504-7512, 2009.
Christian F Doeller, Caswell Barry, and Neil Burgess. Evidence for grid cells human memory network. Nature, (53(7281))6577 2010.
Yedidyah Dordek, Daniel Soudry, Ron Meir, and Dori Derdikman. Extracting grid cell characteristies from place cell inputs using non-negative principal component analysis. Elife, 5:e10094, 2016.
Ariane S Etienne and Kathryn J Jeffery. Path integration in mammals. Hippocampus, 14(2):180-192. 2004.
lla R Fiete, Yoram Burak, and Ted Brookings. What grid cells convey about location. Journal of Neuroscience, 28(27):6858-63 -6771, 2008
Mark C Fuhs and David S Touretzky. A spin glass model of path integration in rat medial entorhinal cortex. Journal of Neuroscience, 26( :266664446 2006.
Marianne Fyhn, Sturla Molden, Menno Witter, Edvard Moser, and May-Britt Moser. Spatial representation in the entorhinal cortex. Science, (0555688):2558-111 2004.
Marianne Fyhn, Torkel Hafting, Menno P Witter, Edvard Moser, and May-Britt Moser. Grid cells in mice. Hippocampus, 18(12): 1230-12233--38 2008
Ruiqi Gao, Jianwen Xie. Song-Chin Chun Zhu, and Ying Nian Wu. Learning grid cells as vector rep- resentation of self-position coupled with matrix representation of self-motion. arXiv preprint urXiv:/810.055 2018
Mariana Gil, Mihai Ancau, Magdalene I Schlesiger, Angela Neitz, Kevin Allen, Rodrigo J De Mareo, and Hannah Monyer. Impaired path integration in mice with disrupted grid cell firing. Nature neuroscience, 21(1):81-91, 2018.
Torkel Hafting, Marianne Fyhn, Sturla Molden, May-Britt Moser, and Edvard Moser. Microstructure of spatial map in the entorhinal cortex. Nature, 436(7052):801, 2005.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Aidan Horner, James A Bisby, Ewa Zotow, Daniel Bush, and Neil Burgess. Grid-like processing of imagined navigation. Current Biology, (6(6):842-8477 2016.
Joshua Jacobs, Christoph T Weidemann, Jonathan F Miller, Alec Solway, John F Burke, Xue-Xin Wei, Nanthia Suthana, Michael R Sperling, Ashwini D Sharan, Itzhak Fried, et al. Direct recordings of grid-like neuronal activity in human spatial navigation. Nature neuroscience, 16(9): 188, 2013.
10

---

Under review as a conference paper at ICLR 2021
Nathaniel J Killian, Michael J Jutras, and Elizabeth A Buffalo. A map of visual space in the primate entorhinal cortex. Nature, 491(7426):761, 2012.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69800 2014.
Rosamund Langston, James A A Jonathan J Couey, Cathrin B Canto, Tale I. Bjerknes, Menno P Witter, Edvard Moser, and May-Britt Moser. Development of the spatial representation system in the Science, 328(5985)
2010.
Bruce L McNaughton, Francesco P Battaglia, Ole Jensen, Edvard Moser, and May-Britt Moser. Path integration and the neural basis of the cognitive map'. Nature Reviews Neuroscience, 7(8): 663, 2006.
John O' O'Keefe. review the hippocampal place cells. Progress in neurobiology, 13(4):419-44 439, 1979.
Hugh Pastoll, Lukas Solanka, Mark CW van Rossum, and Matthew F Nolan. Feedback inhibition enables theta-nested gamma oscillations and grid firing fields. Neuron, 77(1):141-155 2013.
Thomas Ridler, Jonathan Witton, Keith Phillips, Andrew D Randall, and Jonathan T Brown. Impaired speed encoding is associated with reduced grid cell periodicity in mouse model of tauopathy, bioRxiv, pp. 595652, 2019.
David C Rowland, Horst Obenhaus, Emilie R Skytøen, Qiangwei Zhang, Cliff G Kentros, Edvard Moser, and May-Britt Moser. Functional properties of stellate cells in medial entorhinal cortex layer ii Elife, 7:e36664, 2018.
Francesca Sargolini, Marianne Fyhn, Torkel Hafting, Bruce L McNaughton, Menno Witter, May- Britt Moser, and Edvard Moser. Conjunctive representation of position, direction, and velocity in entorhinal cortex. Science, 312 762,, 2006.
Ben Sorscher, Gabriel Mel, Surya Ganguli, and Samuel Ocko. A unified theory for the origin of grid cells through the lens of pattern formation. In Advances in Neural Information Processing Systems, pp. 10003-10013, 2019.
Sameet Sreenivasan and Ila Fiete. Grid cells generate an analog error-correcting code for singularly precise neural computation. Nature neuroscience, 14(10):1330, 2011.
Kimberly L Stachenfeld, Matthew M Botvinick, and Samuel Gershman. The hippocampus as predictive map. Nature neuroscience, 20(1 1):1643, 2017.
Martin Stemmler, Alexander Mathis, and Andreas VM Herz. Connecting multiple spatial scales to decode the population activity of grid cells. Science Advances, 1(11):e1500816, 2015.
Hanne Stensola, Tor Stensola, Trygve Solstad, Kristian Frøland, May-Brit Moser, and Edvard I Moser. The entorhinal grid map is discretized. Nature, 492((427):72, 2012.
Michael Taylor. Lectures on lie groups. Lecture Notes, available at http://www. unc. edduhathh/Facullymeellieg html, 2002.
Sathamangalam Srinivasa Varadhan. On the behavior of the fundamental solution of the heat equation with variable coefficients. Communications on Pure and Applied Mathematics, 20(2): 431-455, 1967.
Xue-Xin Wei, Jason Prentice, and Vijay Balasubramanian. A principle of economy predicts the functional architecture of grid cells. Elife, 4:c08362, 2015.
Michael M Yartsev, Menno P Witter, and Nachum Ulanovsky. Grid cells without theta oscillations in the entorhinal cortex bats. Nature, 479(7371):103, 2011.
Anthony Zee. Group theory in nutshell for physicists. Princeton University Press, 2016.
Kechen Zhang. Representation spatial orientation by the intrinsic dynamics of the derdddirection cell ensemble: theory. Journal Neuroscience, 16(6):2112-2126 1996.
11

---

Under review as a conference paper at ICLR 2021
Sheng-Jia Zhang, Jing Ye, Chenglin Miao, Albert Tsao, Ignas Ceriiauskss, Debora Ledergerber, May- Britt Moser, and Edvard Moser. Optogenetic dissection of entorhinal snoothinalhippocaappe functional connectivity. Science, 340(6128), 2013. 2013.
12

---

Under review as a conference paper at ICLR 2021
A THEORETICAL ANALYSIS
A. 1 PROOF OF PROPOSITIONS
Proof of Proposition I For an infinitesimal bæ ór(cos f, sin 0), v(x)) is changed to v(x + dæ:)
Let da be the angle between v(x) and v(2 + dæ)). It can be obtained from
(3) v( + 62:))
v(x)) v (x)(((((())))))(
(13)
v(œ) + B(0) B + o(rr))
(14)
[B0))(t)]](0r222+ 2 o(ór)"
(15)
where T (()))(()) because (0) -B))) Let
f B [B(0) (2) (f]0)o(())1/|l((())]
(16)
It is independent of because for any 20, ( 20)e R(20) B(O)e) and R R(A0) orthogonal.
thus ( + A0) = || [0))w) |2 for any A0. Note ||=) = || + 5x)) because (ör)
orthogonal,
cos(
1- 8a 8- 822 2 o(ba²)
(17)
u() dz:)) 20x))
(18)
1- (3ór) o(8r2).
(19)
Thus the angle &a = Bór. m
Proof of Proposition 2 Suppose v(x) is defined as in Proposition 2 That is, v(x) -Ue(x), where U is arbitrary unitary matrix. e(x) (exp(i(a,,
= 1, 2) where (a,. j 1,2, 3) are
three 2D vectors equal norm, and the angle between every pair of them is 2 2T/33. Then we have [x)) = 3, and the angle between v(z) and v( + dx)) denoted da, is
(v(2)), v(x dx)) [|
cos( RE
(20)
8a))
1
(E(((((() u(x+ )x)))
(21)
3
(EE(((())UUUeee 8x))
(22)
3
RE
Σ 1=1
exp(ila,, 5zx))
(23)
3
3
cosl (aa,, 5ax))
(24)
3
j=1
1 3
1
Σ
(a,, 8x))2 + 0(622)
(25)
3
2
j=l
32
1
+ o(rr2)
(26)
2
cos (35r) o(rr²).
(27)
Thus we have da = 38r + 0(6r3). Here the key is that (a,,j = 1,2,3) forms a tight frame in the 2D space, in that for any 2D vector dx, ΣE (6x, a² x ||a ||| Thus , oc ||ayll O
A.2 ERROR CORRECTION
Suppose v(x) + is noisy version of v(z), can we still decode æ accurately from v? Here assume e ~ NO(0, (2)((()(() where is the dimensionality of v. and 7² measures the variance of noise relative to /=(==)]2 2/dd i.e., the average of (v,(2))2 1, 1,.., d).
13

---

Under review as a conference paper at ICLR 2021
The heat map
h(x')) (1, u(x')) (m(=)) u(x²)) + (c, u(x') = A(x, x²) e(ax),
(28)
where e(x')
(0))) (1((2')))))))))) 1 //d) For A = exp(- x' x2 = (v(z), u(x')) )) , if o² is small, A(x, x') decreases 0 quickly, i.e. if la' x > 8, then A(r, æ²) < exp( -82 (2o²)), and the chance for the maximum of ((x²)) to be achieved at an a' so that x
can be very small.
The above analysis also provides justification for regularizing |u(=')) in learning.
For error correction, we want to be big, and we want o² be small. But for path planning, we also need big a², That is, we need A(æ, æ') at multiple scales.
A.3 ORTHOGONALITY RELATIONS
For (x) that form group, matrix representation M is equivalent to another representation ((x) if there exists matrix P such that (xx (m) for each A matrix representation is reducible if is equivalent a block diagonal matrix representation, i.e., we can find a matrix P, such that PM (x) P-1 is block diagonal for every Suppose the group is finite group group, and M is a unitary representation. If M is block-diagonal. M diag( MM, with nonneeuivalennt blocks, and each block Mg cannot be further reduced, then the matrix elements (May( (z)) are orthogonal basis functions of æ. Such orthogonality relations are proved by Schur (Zee 2016) for finite group, and by Peter-Weyl for compact Lie group (Tayoor] 2002). For our case, theoretically the group of displacements in the 2D domain is R22 but we learn our model within a finite range, and further discretize the range into lattice. Thus the above orthogonal relations relevant.
compact Lie L..,, K),
In our model, we also assume block diagonal M, and we call each block a module. However, do not assume each module is irreducible, i.e., each module itself may be further diagonalized into block diagonal matrix of irreducible blocks. Thus the elements within the same module ea (x) may be linear mixing of orthogonal basis functions. and they themselves may not be orthogonal. However, different modules may linear mixings of different sets of irreducible blocks, and thus different modules can be orthogonal to each other. Figure 6 visualizes the correlations between each pair of the learned (x) and vy (x), i.j I,....dd d. For v, (x) and vy (x) from different modules, the correlations are close to zero; ie., v.(x) and (x) from different blocks approximately orthogonal to each other. But u, (xx) and vy (x) from the same block are not orthogonal to each other.
0.
02
000
-02
-04
Figure 6: Correlation heatmap for each pair of the learned (x(x:) and (z(x). The correlations computed over 80 x 80 lattice æ.
A.4 ADDING BIAS TERM
Since ||m())] is fixed under rotation, we can add a bias to the vector v(x) so that the elements become non-negative. Specifically, let ((x²) v(x)) b. where b is d-dimensional vector with equal
14

---

Under review as a conference paper at ICLR 2021
elements, then v(x) ((x) b. The recurrent model for v(z + 6x) (I (r)) can then be translated into 0(x a + 5x) = (I + B(0)or) còr, where -B The basis expansion model A₂ (x) (v(x)), u ad)) can be translated into (r (i(xz), u(x')) - b(x'). where b(x=)= = (b, u((²)). (xx) rotates around b.
B EXPERIMENTS
B.I MONTE CARLO SAMPLING
For learning, the expectations in loss terms are approximated by Monte Carlo samples. Here we detail the generation Monte Carlo samples. For (x, x') used in Lo = Ez x Ax x') (v(x), uxx')) is first sampled uniformly within the entire domain, and then the displacement dæ between and x'' is sampled from a normal distribution (0, I²I2), where a = 0.48. This to ensure that nearby samples are given more emphasis. We let xx
da,, and those pairs (ær, within the range of domain (i.e., 2m x 2m, 80 x 80 lattice) are kept as valid data. For (æ, Aœ) used in L1 Ae (o( + Ax) exp( (()) Ar) (z) Ax is sampled uniformly within a circular domain with radius equal to grids and (0, 0) the center. Specifically, Ar?, the squared length of Ax, is sampled uniformly from [0, 3] grids, and & is sampled uniformly from [0, 2 We take the square (Ar cos 0, Ar sin 0). Then
root of the sampled 2r as Ar and let Ax from the region such that both æ and æ + Aa are within the range of domain. For (0, A0) used in L2 Eg, A8 (0 + 20) exp(CA0) (0) ( we enumerate all the pairs of discretized (i.e., 144 directions discretized for circle [0, 2x) and A6 (i.e., 5 directions within range [0, 12.5] degrees) as samples.
uniformly sampled
B.2 LEARNED PATTERNS
Figure shows the learned patterns of u(x) with blocks of size 32 cells in each block. Figure shows the learned patterns of v (xx) and u (z) with blocks of size 16. Figure Sauure/ssowe shows the learned patterns of v(x) with 16 blocks if size 12. Regular hexagon patterns emerge in all these settings.
Figure 7: Learned patterns of (x)) with 6 blocks of size cells in each block. Every row shows the leamed patterns within the same block.
We further evaluate the spatial profile of the learned patterns with blocks of size 16 using the same measures as the main text. All learned patterns exhibit significant hexagonal periodicity in terms of gridness scores (mean 1.06, std 0.27, range 0.58 to 1.48), which exceed the 95 percentile of null distributions obtained by applying spatial field shuffle each response map. The grid scales of learned patterns (mean 0.38, range 0.27 to 0.56), as shown in figure T0al follows multi-modal distribution. The ratio between neighbouring modes are roughly 1.37 and I.38. As shown in figure 10b the grid orientations of learned patterns are also multi-modal distributed.
Figure shows the learned patterns of a block of (0) over from 0 to 2v. Regular sine/cosine tunings emerge. (O)) can be isometric to (0), in the sense that for each column i, the angle between (+) and (, + A0) is A
B.3 SPATIAL PROFILE OF LEARNED HEXAGON GRID PATTERNS
Figure 12 shows the spatial profile of the patterns of ((x)) over the 80 lattice.
15

---

Under review as a conference paper at ICLR 2021
0
C
5
0
1000
00000000
0002
8
830
5
0 000
0000
060
0000
0000
0000
000
2000
0
000
000
100
000
000
600
0 1.0
000
200
0000
001
2000
0000
00000
1.00
100
0 100
100
000
1000
000
0
P
0
c
0
p
(a) Learned patterns of v())
o
0
(b) Learned patterns of u(x))
Figure 8: Learned patterns with 6 blocks of size 16. Top: v(x). Bottom: u(x). Every row shows the learned patterns within the same block.
B.4 LEARNED PATTERNS IN ABLATION STUDY
Figure 13 shows the learned patterns of vxx) where the model is trained with certain component removed.
C
PATH PLANNING
Define and model the adjacency according to Stachenfeld et al. (2017).
20
Ax(xxx')= E
E
y 1xx = 2')|20 = x) = (u(x), u.,
(29)
where IE is with respect to a random walk exploration policy, and y is the discount factor that controls the temporal and spatial scales. We can discretize into a finite list of scales. The above model, ie., (x:) and u. (x'), can be learned by temporal difference learning. The basis expansion model with d & N2 enables efficient learning from small amount of explorations, so that we can fill in unexplored (x, x') based on the learned v(x) and ux,())).
For random walk diffusion in open field, Ay (2x x') x exp(-læ - x212/20?), where o² depends on 7.
For random walk in field with obstacles or non-Euclidean geometry, using Varadhan's 's for- mula (Varadhan] 1967), A, (xx x²) can still be approximated by Gaussian kernel except [ x'1 is replaced by geodesic distance.
After learning v (xx), u, (æ:), we propose to use the following method for path planning. Let à be the target. Let (4) be the current position, encoded by v(x(")), we propose to plan the next displacement by
(1+11) arg max M (M(A (Az) u. (i)).
(30)
Ax
16

---

Under review as a conference paper at ICLR 2021
2a3 0.0.
0
0
00
00
o
Figure 9: Learned patterns with blocks of size 12. Left: v(x).. Right: u(x). Every row shows
learned patterns within the same block.
0.2/
0.37
0.51
0.03
0.22
0.47
14
6
1.37
1.38
Probability density
12.
Probability density
5-
10.
4
3
4
2
2
1
0.3
0.4
0.5
0.0
0.1
0.2
0.3
0.4
0.5
Grid scale
Grid orientation
(a)
(b)
Figure 10: (a) Multi-modal distribution of grid scales. (b) Multi-moda distribution of grid orienta- tions.
and let (1+1) (() + (rr(++)), encoded by ((x((++1) M ((Ar((())))(((z())) In the above maximization, Ax is chosen from all the allowed displacements for single step, and we also need to select an optimal y that most sensitive to the change of A. An example of scale selection scheme (.)) > 0.2. When (t)
that we choose the smallest that satisfies maxa= (Az) far from é the selected o² is big. When (+) is close to át, the selected o small. The above method enables automatic selection of scale. We shall explore other schemes of selecting ? in future work.
We test path planning in open field using the learned model. Specifically, the model is first learned using a single scale (x, x'). where σ,, 0.07. Then we assume list of three scales of A, x'), create
ie., J., [0.07. 0.14. 0.28]. and learn three corresponding sets of (xx)., For planning, a pool of allowed displacements from which A is chosen: the length of A can be or 2 grids, and the direction be chosen from 200 discretized angles over [0, 2nr). Figure 14 depicts several
17

---

Under review as a conference paper at ICLR 2021
MAAA AAVE AAAVE
MAVL IV
A MANAA VE WEAAAWAAE
WEWW
A.
E
Figure 11: Learned patterns of block of B(Ø). Each curve shows the patterns of one element of ((0) over E [0, 2T)] Since (()) is skew-symmetric, the diagonal values are zeros and the value of (0) is the same the value of (0) Regular sine/cosine tunings emerge.
examples of planned paths. As the examples show, when (!) is far from the target, kernel with large a, is chosen, and as (x))) approaches the target, kernel with smaller a,, is chosen. A planning episode is treated as successful if the distance between (t) and target is smaller than 0.5 grid within 40 time
steps. In the cases where the distance between the starting point of the agent and the target is smaller
than 20 grids, the successful rate is 100% (test for 10, 000 episodes).
We shall explore this method in irregular fields with obstacles in future work.
18

---

Under review as a conference paper at ICLR 2021
$72,05L 9OS
0.TH 051.00Z
077,05L BCC
G.7. 6
0 952, 952,0C5 AES5
DEE, 0.3, GaT
0.77, 9
845,052 345
0.5 652 9aI
B&A 052,0.2
636,051 0.3 SA1
0.7, 812 6.2
0.%, 0.65
586,55 645
1,55, 0.35 547
150, 61..0.500 am
1SA 13,
10,0M SA
1.5 1.5.7% MM..75.000 GM
135, 531
135,0 6A1
155, DTA AM
1.55, B.3 DAT
155, DJ, AD
1SA 13
1AL SA
1SS A36,0M DM
1.2,433, A3E
13502 CA
155 DJS, OAD
1.85, 8.7. 0.39
84t. 656,02 028
BA 055 656
CH, D. C38
1.2 856 028
B.t, 055
a.. 550,08 028
00
O0
GO
000
DOO
0O0
600
GOO
099
900
0.0
1.05, 0.0 5.29
1 029
11L 621
1,0,0 050 527
0.03 055 6.29
0E6 552,
646, 057 6.29
1 057.027
OO
00L
900
000
OOO
OOO
O
pOG
99
1,18,022 0445
138 642,045
LI5 531.045
1.5, CAS
L1 032,0 645
L17 543 845
037.022 5A5
1.5 632.222 0
D32 CAS
117 645
13 6.5
1.5 D2D SAB
123 032
136 23.
1:60 SA
LI 632, 045
1.LAA4 DAS
LXX 525
LAL CAE
LA 874,0
1.M. 1M,424A0 AES
131,20 BAE
1.M. DOS AM
13L BZA DAS
131 DJI, ABS
125 B25
LEL CAE
133 874,0
1.3, AES
131,02 BAS
T.L DZA, OIS
S41, B38 SAT
8GI B3T DM
OTL 337
633, SAT
06) D3T, AM
DES, C37, DO
843,0% 030 SA1
0.1, 078, SM
544 836 5A3
8.7 63T.0. aM
63. CAT
6TH 63T, GM
AO 537 AM
541, 037 6A3
6R 676 0
Figure 12: Spatial profile of the patterns v(x) over the 80 x 80 lattice. For each unit, the autocorrelogram visualized. Gridness score, scale and orientation are listed sequentially top of the autocorrelogram.
19

---

Under review as a conference paper at ICLR 2021
(a) Learned patterns of v(x) with Lu = E [A(x, x' (v())) u(xx)) > removed.
(b) Learned patterns of v(x) with L Em /n// Ar)
exp( B (0) (r) removed.
(c) Learned patterns v(x)) with L2 Eg ..
B
A
exp CA# (0)
removed.
(d) Learned patterns of v(x) with the regularization of (( removed.
(e) Learned patterns of v(r) with the assumption of skew-symmetry for B(0) and C removed.
(1) Learned patterns of v(w) with the assumption of removed.
Figure 13: Learned patterns of (xz) in ablation study, where the model is trained with a certain component removed.
20

---

Under review as a conference paper at ICLR 2021
Figure 14: Examples of path planning in open field. Red denotes the target. Three scales of
A,(x, (x. x') are used (a,, = [0.07, 0.14, 0.28)). Different colors denote the kernel chosen at each step.
21


Reviews:
[{"review_id": "zl6V-JNUkeL", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: The authors propose a simple recurrent network as a model of spatial navigation in the MEC/Hippocampal network. This model assumes that grid cells only regularly receive egocentric movement information, an important aspect for understanding the origin of these functional cell types in-vitro. Overall, this article should be of interest for any ICLR members interested in biological  spatial navigation. \n\nStrong Points: \nAs stated above, the model is an intuitive explanation of how allocentric-egocentric transformations might be performed. While the authors use a back propagation approach to training, the separated loss functions (eq 10-12) for each layer of the network mean that learning could be performed by predictive contrastive coding, as recent research suggests biological networks may be doing.\nThe learned receptive fields show many of the more nuanced aspects of grid cells found in experimental studies, such as discretized angle relative to the environment. The investigation of error accumulation as a function of time steps since encoding allows additional comparisons to the literature. These give additional confidence that the model is biologically plausible, at some level of abstraction.\n\nWeak Points: As a non-mathematician, it's unclear to me what the implications of the lie algebra presented in the beginning of section 2 are. My understanding is that this makes grid activities the product of two separable matrices (displacement, and rotation). While this is biologically plausible, perhaps the authors could explain any additional reasons why this is of importance.\n\nAdditional Comments: A possibly interesting future experiment would be investigating effects on error correction by only having place cells at a handful of spatial scales (A.2), especially if there is a \"block\" structure in u.", "extra_scores": {}, "extra_text": {}}, {"review_id": "uh2i1TYzpO", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors develop a model for learning the observed responses of grid cells (GC) in the entorhinal cortex from the animal movement vectors. Their key assumption is that the GC activity vector rotates with the movement magnitude according to the Lie group formalism and the corresponding Lie group generator is also rotated by the change in movement orientation. Their claim is supported by a numerical optimization of the objective function reflecting these assumptions as well as the projection onto the place cell representation in the hippocampus.\n\nI find the paper original, interesting and clearly written. The numerical simulations support the claim. However, given the current state of the field, I would like to see an analytical demonstration of this claim like in the work of Sorscher et al which the authors cite. Specifically, would it be possible to demonstrate that dropping the orthogonality constraint used in Sorscher et al and introducing the rotation of the Lie generator would still result in realistic GC responses as the authors claim? Such analytical demonstration would provide a much needed insight into the operation of the system. \n\nMinor comments:\n\nPage 3, second line after Eq(4): the closing parenthesis is missing after B(\\theta\n\nSorscher et al reference is listed twice\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nAdded after author response. My enthusiasm for the paper has diminished because it seems to be more of an incremental step over Gao et al 2019 and the authors did not provide additional analytical insight into their new results. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "HAmW11WbxOs", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a model of the grid cell system based on computing an embedding of position and head direction that allows matrix Lie algebras to translate and rotate the coordinate frame.  The results show that individual elements of the embedding have spatial response profiles resembling grid cells in entorhinal cortex.\n\nEquations 1 and 2 are a nice mathematical approach.  the justification for using Lie algebra is clear and its very elegant, and seems like a good idea.   The results are interesting too - very neat to see the hexagonal grid arrangement emerge.  But why?  Here I'm not sure we learn anything.  In fact the authors seem to state that certain parameter settings such as beta are required to bring about this result, but there's little insight provided as to why.  And how does it affect the result?  Also, the modularity - one of the most interesting facets of the grid cells system - is rigged in advance, rather than an emergent property.  So it leaves you wondering, what's the point here?  For example, what is the advantage conferred by embedding 2D position into a higher dimensional space?  why is this a good thing for the brain to do?  and why a grid system?  the answer seems to be \"that's what emerges from our matrix Lie algebra model.\"  ok, but that's not very illuminating.  Surely there should be some way to reason about this result and why its useful for brains to have this type of representation.  That would make the paper a lot more interesting in my view.\n\nThe authors do a nice job investigating the influence of different terms.  Ok, but it also seems like just hunting and pecking phenomenology, try this try that.\n\nAlso this sentence in the intro I find rather unsatisfying:  \"It is worth noting that our work is mainly concerned with representation learning. We do not seek to pursue biologically realistic modeling of neural dynamics..\" - so you are modeling an aspect of biology, but you aren't concerned with realistic modeling of neural dynamics?  I can understand perhaps leaving out spiking neurons and ion channels and all that, but it would seem to me that taking into account what is actually neuronally feasible is important.  For example, presumably having a local connectivity structure in B is important, and that could well affect the results.  Its not just about representation learning, but implementation and biophysical constraints are also important.\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "-3OwLJY1xYz", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a simple recurrent model of how grid cells may perform path integration, which also recapitulates the well-known finding that grid cells exhibit hexagonal firing patterns. Their model consists of two primary components where self-position is represented by a population activity vector (which is rotated by a generator matrix of a Lie algebra whenever the agent moves in a given direction), and where self-motion is represented by the rotation of this vector (whereby when the agent changes its direction, this generator matrix is itself rotated by another generator matrix).\n\nStrengths:\n+ I think the model is a mathematically elegant formalism.\n+It is more explicit than the approach taken by task-optimized (nonlinear) RNNs.\n\nWeaknesses:\n-\tIt is unclear to me what scientific insight we get from this model and formalism over the prior task-optimized approaches. For instance, this model (as formulated in Section 2.3) is not shown to be a prototype approximation to these non-linear RNN models that exhibit emergent behavior. So it is not clear that your work provides any further “explanation” as to how these nonlinear models attain such solutions purely through optimization on a task.\n-\tFurthermore, I am not really sure how “emergent” the hexagonal grid patterns really are in this model. Given partitioning of the generator matrices into blocks in Section 2.5, it almost seems by construction we would get hexagonal grid patterns and it would be very hard for the model to learn anything different.\n\nWhile the ideas of this paper are mathematically elegant, I do not see the added utility these models provide over prior approaches nor how they provide a deeper explanation of the surprising emergent grid firing patterns observed in task-optimized nonlinear RNNs. For these reasons, I recommend rejection.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a recurrent model of self-position, with a handcrafted expression of the rotational structure in terms of a matrix Lie group. As noted by the reviewers, this work strongly builds upon Gao et al (ICLR 2019). This really is mentioned too late and not prominently enough in the manuscript, and furthermore, the difference to this work is not clearly explored in the paper (there are just two sentences immediately prior to the conclusion and no experimental comparison). The reviewers pointed out that the phenomena observed here are handcrafted into the structure of the model, rather than being emergent. The reviewers raised concerns that it is not clear what conclusion to draw from this work.
For these reasons, I recommend rejection this stage.
Decision: Reject