Global Distortions from Local Rewards: Neural Coding Strategies in Path-Integrating Neural Systems
Authors: Francisco Acosta
Fatih Dinc
William T Redman
Manu Madhav
David Klindt
Nina Miolane
Venue: NeurIPS 2024 poster (2024)
Keywords: neuroscience
recurrent neural networks
grid cells
geometry
Primary area: neuroscience_and_cognitive_science

Abstract: Grid cells in the mammalian brain are fundamental to spatial navigation, and therefore crucial to how animals perceive and interact with their environment. Traditionally, grid cells are thought support path integration through highly symmetric hexagonal lattice firing patterns. However, recent findings show that their firing patterns become distorted in the presence of significant spatial landmarks such as rewarded locations. This introduces a novel perspective of dynamic, subjective, and action-relevant interactions between spatial representations and environmental cues. Here, we propose a practical and theoretical framework to quantify and explain these interactions. To this end, we train path-integrating recurrent neural networks (piRNNs) on a spatial navigation task, whose goal is to predict the agent's position with a special focus on rewarded locations. Grid-like neurons naturally emerge from the training of piRNNs, which allows us to investigate how the two aspects of the task, space and reward, are integrated in their firing patterns. We find that geometry, but not topology, of the grid cell population code becomes distorted. Surprisingly, these distortions are global in the firing patterns of the grid cells despite local changes in the reward. Our results indicate that after training with location-specific reward information, the preserved representational topology supports successful path integration, whereas the emergent heterogeneity in individual responses due to global distortions may encode dynamically changing environmental cues. By bridging the gap between computational models and the biological reality of spatial navigation under reward information, we offer new insights into how neural systems prioritize environmental landmarks in their spatial navigation code.

Global Distortions from Local Rewards: Neural
Coding Strategies in Path-Integrating Neural Systems
Francisco Acosta Department of Physics UC Santa Barbara facostaducsb. edu
Fatih Dine
William T. Redman
CNC Program
Applied Physics Laboratory
Stanford University
Johns Hopkins University
Manu Madhav Biomedical Engineering University of British Columbia
David Klindt
Nina Miolane
NeuroAI CSHL
Electrical & Computer Engineering
UC Santa Barbara
Abstract
Grid cells in the mammalian brain are fundamental to spatial navigation, and therefore crucial to how animals perceive and interact with their environment. Tra- ditionally, grid cells are thought support path integration through highly symmetric hexagonal lattice firing patterns. However, recent findings show that their firing patterns become distorted in the presence of significant spatial landmarks such rewarded locations. This introduces a novel perspective of dynamic, subjective, and action-relevant interactions between spatial representations and environmental cues. Here, we propose a practical and theoretical framework to quantify and explain these interactions. To this end, we train path-integrating recurrent neural networks (piRNNs) on a spatial navigation task, whose goal is to predict the agent's s position with special focus rewarded locations. Grid-like neurons naturally emerge from the training of piRNNs, which allows us to investigate how the two aspects of the task, space and reward, are integrated in their firing patterns. We find that geometry, but not topology, of the grid cell population code becomes distorted. Surprisingly, these distortions cells despite local changes in the reward. Our results indicate that after training with location-specific reward information, the preserved representational topol- ogy supports successful path integration, whereas the emergent heterogeneity in individual responses due to global distortions may encode dynamically changing environmental cues. By bridging the gap between computational models and the biological reality of spatial navigation under reward information, we offer new insights into how neural systems prioritize environmental landmarks in their spatial navigation code.
global in the firing patterns of the grid
Introduction
Motivation Neuroscientists describe computations in the brain by jointly considering the firing activity of a population of neurons in neural state space RR. this space, each basis vector corresponds one neuron, one point summarizes the activity of all neurons at a given instant, and a trajectory in neural state space describes the time evolution of the firing of all neurons. Across many brain regions and modalities, these trajectories have been observed be mostly constrained to lower dimensional manifold within the state space: the neural manifold (see Figure I right). Such manifolds have been observed in the motor cortex
2) the hippocampus cortex [5] 6), head direction circuit [7] 8], and grid cells in the entorhinal cortex [9]. Characterizing the geometric properties of neural manifolds including their dimension, topology, and curvature enables wholistic understanding of multi-neuron activity through time, and quantified approach to
visual
describe computations the brain 10
12
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Grid cells;
Place cells
Roal Futh inforeed rroh
Firing field of one grid cell
Neural manifold grid cell module
Et
L
Ut
N
it+1
End
Inferred Position
1.0 m
1..0
Figure I: Path integrating Neural Systems. Left: an artificial agent explores its 2D environment traveling along the path 2, shown in black. Middle: the agent's velocity U2 is given as input to neurons in a path- integrating recurrent neural network (piRNN) grid cells in the mammalian brain [3 The agent maintains representation of its movement across multiple grid cell modules. This representation linearly decoded onto place cells, providing an estimate the agent's 's new position 2 Right: Each grid has firing field that a hexagonal lattice. Together, grid cells' activity within a grid cell module forms a 2D torus 9 20
Grid cells are neurons in the medial entorhinal cortex (MEC) which exhibit activity patterns character- ized by hexagonal lattices across space [13] (see firing field on Figure [ Grid cells can grouped into distinct subpopulations called modules, specified by the spatial period of their hexagonal lattice [14] The neural manifold corresponding to neurons in a grid cell module has been shown to form 2D torus in the high high-dimensional state space [9. The traditional view in neuroscience is that grid cells support path integration the process used by an animal to keep track of its position by integrating its past position and current speed -by providing global metric for physical space. However, recent experiments suggest that grid cell lattices become distorted in the presence of relevant landmarks, such as rewards [15 16 17. In other words: grid cell activity might not reflect true physical distances but instead underlies a metric for subjective action-relevane space [18]. While the distortion of grid cell activity has been reported qualitatively, it remains to be described quantitatively.
Contributions We propose a practical and theoretical framework to quantify and explain the distor- tion of grid cell neural manifolds under task-relevant reward information. We apply this framework to compare computational and biological neural systems performing spatial navigation.
Our technical contributions are:
We adapt the training of biologically-inspired RNNs performing path integration (piRNNs) 19 to account for spatial rewards via new saliency loss term.
2. We introduce methodology to quantify the deformation of grid cells' firing fields and neural manifolds two scenarios aabsence and presence of rewards and describe the topological changes in the grid cell module tori after training with reward.
3. We illustrate the implicit regularization achieved by fixing the readout weights, which retains the existing toroidal topology of the neural manifolds during reward training.
Our conceptual contributions are:
1. We quantify the introduction of reward in spatial navigation tasks by linking the geometric distortions of grid cells' responses and corresponding distortions of their neural manifolds.
2. In piRNNs, we link the emergent distortions of the individual hexagonal responses to geometric deformations of the neural manifold.
3. We provide evidence suggesting that grid cells may be involved in continual learning.
After reward training, the toroidal topology continue support successful path integration, while geometric deformations of the neural corresponding to global distortions in the inidividual re- sponses -may represent local, dynamic environmental cues, such reward information. This dual representation allows the grid cell modules maintain their foundational navigation capabilities while adapting to new environmental variables.

---

2 Background & Related Works
2.1 Neuroscience of Spatial Navigation
Path integration is the process by which animals use self- self-motion cues including information from the vestibular system and propriaception to integrate their past movements and positions, allowing them to maintain estimate of their position. Theoretical models have explored mechanisms by which path integration can be performed by neural networks in the mammalian brain [21], most notably through networks of place cells and grid cells [22)
Place cells are functional class of neurons found in the hippocampus 23 implicated in memory and spatial navigation. The neural activity of place cells is believed to represent the animal's 's current physical location in space, thereby creating a "cognitive map" 24
region of the brain
Grid cells are a functional class of neurons found in the medial entorhinal cortex 13 whose spatial firing pattern form a hexagonal lattice covering their 2D environment in open field settings. Many works [25] 26 27 show that grid cells play an important role path integration, by integrating
self-motion information sending neuronal signals to their downstream place cells.
The grid cell firing field can be described by three parameters: (1) spacing, distance between lattice points, (2) orientation of the lattice relative to a reference direction, and (3) phase, or the lattice offset relative to reference point. A grid cell module consists of grid cells with the same spacing and orientation but varying phases [4]. Each module is known to form a neural manifold with the topology of the 2D torus T [9]
2.2 RNNs for Spatial Navigation
Biologically-insee inspiredd, computational models of place cells and grid cells have emerged in recent years. Specifically, recurrent neural networks trained to perform path integration (piRNNs) of artificial agents have been shown to exhibit artificial neurons that behave like biological grid cells [28 229 30 19 31 32). This paradigm allows for systematic exploration of phenomena observed
neural circuits involved in spatial navigation, such as hippocampal remapping (33) Below, introduce the piRNN framework of spatial navigation from [19]
Let x E I222 denote the agent's position, Az E IR2 the displacement in unit time, e the velocity, r(x) RN be the activity of N grid cells, ((x) E IR"" be the activity of P place cells; r; (2() and (x(x) being the activity of the 5th grid cell and nh place cell, respectively -see Figure | (left). The grid cell activity r(z) for each module forms toroidal neural manifold embedded in the neural state
space N torus on Figure (right).
Place cells in PiRNNs: As done in the literature 30
we model place cell activity & as linear readout of the activities, r, of the piRNN neurons. The position estimate is decoded from as:
8 Qr", & = aar max o3. PEP
where Q E PxN is the place cell readout matrix. The architecture is illustrated in Figure Previous work has shown that the piRNN neurons, T, become grid cells after training [30].
The piRNN is trained to perform path- auth-integration and correctly infer the spatial position. Its goal is to learn the place cell readout matrix Q. the neural representation (r(x), Yr), its recurrent weight matrix W E R X and its input weight matrix U E mathbbR2x via minimization of the loss function L. The loss used by Xu et al. [9 is:
Path integration by PiRNNs:
L Lemor + Lxemel + Leuntormal + Gregulurization
(2)
The error the prediction of the spatial position is written Leror E Ar [Terroe] where:
T
I LagTrr
Σ [1 + Am +.... Art) Qr(x + Ar, + +... Ar
We refer to Sec. S3 for details the other terms: Lvemel, Loonformal- Crogularization.
3

---

A
Rate maps
Uniform salioncy
module 1
module A
module
M
module
module
Rate maps
Varying saliency
2'2
Figure 2: Geometry of grid cell module torí is changed by presence of salient features in the environment. An agent (for us, a piRNNN is trained perform path integration saliency. Canonical grid cells develop hexagonal lattice responses (rate maps) across M modules. The population activity of single grid cell module forms torus in neural state space. B. The same agent undergoes a second phase of training. with its environment now containing rewards (areas of high importance, or saliency). We model this saliency by modifying the loss of our piRNN to prioritize accurate position decoding near rewards. Its grid cells adjust their individual responses, which we geometric deformations of the neural tori.
environment with uniform spatial
3 Theory of Deformations: From Firing Fields to Neural Manifolds
Recent experimental evidence in animals and our own experiments piRNNs reveal that grid cell firing fields get distorted after introducing a reward in the agent's spatial environment. Here, we propose theory to relate deformations of the firing fields (see Figure middle) to deformations of the corresponding neural manifolds (see Figure right) during path-integration with spatially localized rewards. We first present the general theory of deformations. Then, illustrate with two scenarios observed in practice: (i) smoothing of firing fields (diffused units) observed in our piRNN experiments described in Section 4 and (ii) attraction of firing fields (attracted units) observed in [15]
3.1 General Theory
Deformation of firing fields Consider one grid cell with firing field + rr(z)) over the environ- ment R² (Figure middle). In line with experimental observations, we assume that the introduction of reward, modeled by the saliency map s(r), deforms the firing field of the grid cell (see Figure 2) We model this deformation as diffeomorphism
IR2 R22 of the environment.
We further assume that every grid cell i has a constant firing energy budget, in that: feege Ta (x) )d is constant. Putting this together, introducing a reward the environment yields a new firing field for grid cell written
(z) det 4' (x)).rr o (xx for all x E R.2
where Ø' is the Jacobian l. and "." denotes scalar multiplication. This formulation deforms the shapes of the firing fields, and modifies their amplitudes in order to keep the firing energy budget constant. This statement made precise in the proposition below.
4. det (z)) represents a change volume induced by the deformation
The deformation of the grid cell firing fields ri by & given by r + r defines right group action of the group of diffeomorphisms over the space of firing fields. The firing energy budget is an invariant of this group action.
Proposition
The definition of group action and the proof are given in Appendix SI Here, we do not specify a formula for Instead, we suppose that & can be estimated from experiments with piRNNs or animals performing path-integration tasks without and with rewards.

---

Deformation of neural manifold The deformation of each firing field via O induces deformation of the associated neural manifold: the torus associated given grid cell module changes shape in neural state space R.V (see rightt)) Before deformation, the torus neural manifold can be where the periodicity of the firing fields means that the function R is not necessarily injective: one point on the manifold surface may correspond to several locations x's s.s. Introducing a reward in the environment yields a new neural manifold written
represented by the parameterized surface x + R(x) = Iri
R (x) = det 2 (t) ((x) E R for all x E
(5)
In what follows, we consider that is injective, or we restrict its domain of injectivity. this context, we can predict some geometric properties of the deformed neural manifold, as given in the proposition below.
The deformation of the neural manifold R by given by R R defines right group action of the group of diffeomorphisms over the space of neural manifolds. The barycenter and the topology of the manifold are invariants of this group action, provided that the firing fields are regular.
Proposition 2.
Original
Diffused
Attractive
Band
Firing Field
exp.var.
50% vxp.
exp.ar.
15-
PCA
0-
-15
MDS 0-
H1
TDA
H2
Filtration Value 20
Figure Relating firing fields neural manifolds across deformations in synthetic grid cells. We investigate how deformations away from perfect hexagonal symmetry the firing fields of synthetic grid cells affects the geometry of the toroidal neural manifold. From left to right. Original units. The original hexagonal grid cells show clear signatures of toroidal topology, as indicated by the presence of 2 loops in the first homology group (HI) and void in the second homology group (H2). We show 2D projections using principal components analysis (PCA) and multidimensional scaling (MDS) serve as baselines against which to compare the manifolds of deformed grid cells. PCA projections are consistent with "flat" torus geometry. Diffused units. Diffused units created from convolution of the original grid cells with a Gaussian kernel maintain toroidal topology, hut PCA and MDS show that the size of the neural manifold is reduced predicted by theory. Attracted units. Inspired by experimental evidence from 15 we created attracted units from synthetic grid cells by applying diffeomorphism to the 2D environment. While manifold size is unchanged, PCA projections suggest the torus becomes more curved in neural state space. Toroidal topology is preserved. Band units. We created a synthetic module with 17 17% of original grid units replaced with band units of same spatial scale, with uniformly distributed orientations. The geometry and topology of the resulting manifold are largely unchanged.
5

---

fore the proof. Next, we consider deformations of the firing fields that have appeared See Appendix for empirically, either in our piRNN experiments or in animal experiments.
3.2 Diffused Units:
Smoothing of Firing Fields
We show how smoothing of the firing fields of the grid cells, observed empirically in Section impacts neural manifolds in neural state space.
Conjecture 1 (Reduction hypothesis). The smoothing of the firing fields by Gaussian filter reduces the size S of the neural manifold R, where the size is defined as: = max [[(x)]]-
The size of the neural manifold becomes, after introduction of reward and deformation by 4;
/ MERA
S max
G(x u))((())))
(6)
where G represents the Gaussian filter used for smoothing. When a function [|[]] is convolved with a Gaussian filter, the resulting function is smoother than |/ The convolution effectively averages the values of ||] over the neighborhood defined by the Gaussian, which reduces the peaks of |R]]. This because the convolution essentially a weighted average. Consequently, via such a deformation of the firing fields, the resulting manifold will be smaller.
Testing the Reduction Hypothesis In practice in our experiments, we perform principal component analysis and other dimension reduction techniques to compare the sizes of the neural manifolds before and after the deformation produced by the introduction of reward.
3.3 Attracted Units: Displacements of Firing Fields' Centers
We show how the attraction of the grid cells' firing field centers to reward locations, observed in experiments in [55 and simulated with exaggeration in Figure 3 impacts the neural manifold in state space.
Conjecture 2. Attracting the centers of the firing fields region of the environment expands the corresponding region of the neural manifold, provided that the firing rates are regular enough.
Indeed, the diffcormorphism & that represents this deformation displaces volume elements of the environment, placing more volume in the center. Therefore, the term det &' (2(x)) that measures volume changes will be high next to the center of the environment. The region of the neural manifold corresponding to the center of the environment will therefore be magnified.
We include a more detailed discussion on the link between rate map distortions and neural manifold geometry in Section S2
We illustrated several synthetic grid cell modules to demonstrate how various types of common grid cell firing field deformations affect the geometry of the toroidal neural manifold, summarized in Fig. 3 Global deformations such as the emergence of diffusive units with less crisp hexagonal firing patterns, attracted units with distance- based scaling, and the addition of band-like cells to group of grid cells did not change the toroidal topology of the modules. Moreover, as predicted by Conjecture [ the diffused units reduced the size of the manifold. Overall, global distortions can be added to the grid modules without destroying the existing toroidal topology of the neural manifold subserving the spatial navigation. As we show in the next section, such patterns emerge as result of the dynamic integration of environmental information during the piRNN training. This process enables the networks to simultancously solve path integration and represent environmental cues effectively.
Empirical Results
In this section, we test the theoretical global distortions introduced in the previous section by training piRNNs perform path integration while enforcing higher accuracy in positional decoding near salient locations to model the presence of local rewards. We first introduce the multi- training procedure with non-uniform saliency. Then we discuss the effects of saliency training on the toroidal topology of the neural manifold, and examine the emergent distortions in the grid cell firing fields. Finally, we confirm our theoretical prediction from the previous section regarding how diffused units lead to reduction in manifold size.

---

A
Module 7 (before)
B Module 7 (after) Frozen read-out
C Module 7 (after) Trained read-out
Homology 2 Homology
0.00
0.02
0.00
0.02
0.00
0.02
Filtration Value
Filtration Value
Filtration Value
Figure 4: Toroidal topology is preserved after saliency training when place cell read-out is frozen. For grid cell module 7, project the neural activity 6 dimensions using PCA, and perform persistent homology. We show the 20 most persistent features for homology dimension (loops) and homology dimension (voids). Gray bars are 20 most persistent features from 10 random shuffles of the data. A. The population activity of module 7 of the pretrained RNN (before saliency training) has Betti numbers (1,2,1). consistent with toroidal topology. B. The population activity of module of the RNN after saliency training retains toroidal topology. if place cell read-out frozen (only grid representations destroyed after saliency training place cell read-out is allowed change.
allowed change). C. The toroidal topology
4.1 Training piRNNs with Non-Uniform Spatial Saliency
We investigate how the representations learned by piRNNS change through multiple phases of training incorporating spatial saliency into the objective. We begin by pretraining the piRNN model introduced in [19. which learned representations with a large fraction of units (> 70%) classified as grid cells, based on the widely-used gridness score [34]. These representations exhibit high degree of hexagonal periodicity at different scales, correspondong to distinct grid cell modules. We employed procedure inspired by [9] to identify the grid cell modules (see Fig. S3).
After this initial pretraining phase, we modified the loss to imitate the non-uniform saliency across the environment that is characteristic of animals exploring their environments in more naturalistic settings [8] that may contain rewards. This stands in contrast to the highly uniform "open field" settings typically used in experiments, where neurons with a high degree of hexagonal periodicity are commonly found, as shown in Fig. 2 We add kernel (1 + s(x)) multiplying the loss term Eq, that differentially weights the penalty of incorrect self-position estimation across different locations in the environment, thus quantifying the notion of spatial "saliency". The new loss L which
modifies Eq. 3 reads
C Er,d. [(1 + s(x)) Lemor))
In our experiments, we parameterize s(x) with a Gaussian of width a, centered at x,:
s(x) so exp (-11x zz ==112/2o?))
/ 2nT??
(8)
and we set x, 0 and 0. 0.05L, where L is the length of the square environment.
4.2 Saliency during the training may respect or destroy the toroidal topology
We performed the second phase of training with the saliency loss in two ways. First, we allowed all connections, including the readouts from the piRNN units to the place cells, to be learned. In this case, consistently observed change in the topology of the grid modules, where the toroidal structure underlying the grid cells was destroyed (Fig. 4A,C), though the firing rates showed qualitatively similar patterns the grid cells (Fig. S4p.
As a next experiment, during the second phase of training we froze the readout weights from the RNN to the place cells and observed that the grid cell module toroidal topology was preserved (Fig. 4A,B). Therefore, freezing the readout may be seen as an implicit regularization that enabled continual learning in the grid cell population. By preserving the toroidal topology of the neural manifold,
7

---

the system leverages its previously learned representational structure subserving successful spatial navigation while encoding a dynamic environmental variable, i.e., the saliency. Therefore, for the rest of our experiments, we fine-tuned the piRNNS with frozen readout weights.
4.3 Saliency-tuned piRNNs develop diverse set of distorted tuning features
In our experiments, we found that following the saliency training phase (with the modified loss term L Cerror), ), the piRNNs developed neurons with more heterogeneous set of spatial tuning properties, as illustrated in Fig. 5 Specifically, we observed the emergence of diffused and hand units, as predicted by our theoretical framework in Section Interestingly, we did not observe the attractive distortions predicted in Fig.
An important aspect of these distortions is that they are global. whereas the reward was localized at the origin. Given that place cells average the input from multiple grid cells through their readouts, it is possible that global distortions in the grid cells facilitate local changes in the final output. For example, the ring units may arise because the reward information is rotation- ally symmetric, eliminating the need to retain angle information and creating the secondary ring structures shown in Fig. 5A. Additionally, if the center of the en- vironment needs to be represented preferentially over the rest of the space, it makes sense for the represen- tation of places outside the origin to become less crisp to conserve, e.g., firing rate budget. This could be achieved by a convolution in the firing rates, leading to the diffused units observed in Fig. 5B. Finally, grid cells can be described as a summation of a few band cells in different orientations. If these bands cover all angles, the result can approximate a place cell in the origin, making the band units, observed in Fig. 5JC. a mutually synergistic solution for both representing spatial navigation and the reward information (located at the origin).
Original (pre-trained)
After saliency training
0.08
21% espuar.
25%
PCA
80.0
0.10
MDS
0.10
H1
TDA
H2
Filtration Value
As the final test, we focused on modules that showed increased levels of diffused units and tested the re- duction hypothesis presented in Conjecture[] [ which suggests that the emergence of diffused units should decrease the overall size of the manifold. As expected, the diffusing units did not change the topology of the module, and resulted in a geometrical reduction in size (Fig. 6)
Figure 6: Diffused units lead to reduction in the manifold size. When we analyzed a module with many emergent diffused units after the saliency training, we observed that the topology of the module preserved. However, the size of the manifold was reduced in line with our Conjecture
A Ring units
B Diffused units
C
Band units
From moduleslls111 1,2.8 Beefore From modules 1,2,18 Jefler)
From modules 2. 5, Tocforel From modules 2, 5.7 lafter
From mudule 1 (awlone)
From mudule (elber)
Figure 5: Saliency-tuned PiRNNS develop diverse set of tuning features observed in neuroscience. A. Across multiple modules, we observe the emergence of many units characterized by rings of high activity. B. We also observe the emergence of diffused units which are active almost everywhere. C. Finally, we find many units develop band structures.
8

---

Overall, our results suggest that the global distortion mechanisms allow the grid cells to retain their spatial navigation capabilities while adapting to the localized reward signals. This balance can enable the piRNNs to maintain navigation functionality and incorporate new environmental cues effectively.
5 Discussion
Local reward leads to global changes in the firing rates: When we trained the piRNNs with local rewards, encouraging the network to represent specific places more accurately, we observed distinct global changes in the firing profiles of grid cells. In our experiments with piRNNs, we observed diffused and banded patterns, as well as units with secondary local ring-like structures similar to those observed in human studies (35], but not the attracted firing patterns.
To our initial surprise, we did not observe the kind of local "magnification" or deformations suggested by the interepretations of previous rodent experiments from [1] in any of our experiments. Instead, we observed global changes in the firing fields. Therefore, we present in silico evidence that challenges the conventional interpretation that reward modulation results in local magnification of the grid cell lattice as too simplistic an explanation. Indeed, because grid cells provide global code for space, we propose that reward modulation should result in global deformations of spatial responses in MEC, as observed in our piRNN experiments.
No attracted neurons emerged in reward-modulated PiRNNs:
The appearance of diffused units suggests that multiplying the loss function by a local reward signal in the navigation space causes convolution in grid cell firing patterns, indicating a duality between grid cell firing patterns and changes in the reward function, akin to the Fourier Transform. The emergence of banded units aligns with representations of the origin, as the Fourier transform of square wave periodic function with most of its mass centered at the origin. The ring units' emergence is also consistent with the reward being centered, leading to a trade-off in representing angles for better distance representation to the reward. Notably, local changes in the reward function caused global changes in grid cell firing patterns, reinforcing the concept of this duality.
Dynamical interactions lead to distorted units: Previous studies have shown that units with banded firing rates emerge during reorganization 36). Additionally, research has demonstrated that banded units and grid cells can coexist, with the same cells behaving as either depending on the environment or the task (37). A common finding across these studies, which is consistent with our results here, is that grid cells transform into banded or other unit types with different symmetries, likely influenced by the complexity of the action rather than the properties of the navigated space.
Distorted grid cells emerge with frozen readouts, otherwise complex reorganization ensues: The fact that the toroidal topological structure is destroyed when readouts are allowed to change supports our hypothesis that the distortions observed in grid cell firing fields are related to the network's s efforts to adapt to a changing environment, such as the introduction of a reward, without completely reorganizing itself. Complete reorganization might minimize loss function associated with the reward's ' location but would fail to generalize to changing environmental conditions, such as the introduction of subsequent phases of training with different saliency distributions.
We conjecture that the distortions observed in our experiments, and regularly reported in neural data 37 36 35 15), signal continual learning. These distortions expected vary from environment to environment and task to task. This suggests that grid cells can exhibit significant flexibility, dynamically adjusting new tasks and conditions while maintaining overall functionality. Hence, although their toroidal neural manifold topology supports navigation, global distortions may facilitate the dynamic encoding of localized environmental cues.
6 Conclusion
Prior work with grid cells has primarily focused on their contribution to spatial navigation. Recent studies have also considered local changes in grid cell firing patterns, such as attractions to reward positions [15]. In this work, by analyzing how piRNNs handle these tasks simultaneously, we tested whether global distortions emerge on grid cell firing patterns and how they affect their functionality and their adaptability in representing complex environmental cues. Overall, our theoretical and experimental findings suggested nuanced view: the hexagonal grid structures facilitate the decoding of spatial locations, while global, not necessarily local, distortions in the grid cell geometry can

---

dynamically encode environmental cues relevant to specific actions, highlighting the flexibility of grid cells in adapting various environmental contexts and tasks (37.
Though grid cells were not traditionally considered in the context of continual learning, our results implicate their involvement in these paradigms. Additionally, studies of spatially structured, periodic, and global rewards should illuminate the duality in the dynamic representation of environmental cues by grid cells, which may allow building towards theoretical understanding of the duality between local rewards and global distortions we observed in this work.
Acknowledgments and Disclosure of Funding
We thank Simon Mataigne, Adele Myers, and members of the Geometric Intelligence lab for helpful discussions and feedback on this manuscript. F. A. and N. M. acknowledge funding from the NSF grant 2313150. WTR acknowledges funding by an Internal Research and Development grant from JHU/APL. FD receives funding from Stanford University's Mind, Brain, Computation and Technology program, which is supported by the Stanford Wu Tsai Neuroscience Institute.
References
[1] Juan A. Gallego, Matthew G. Perich, Raeed H. Chowdhury, Sara A. Solla, and Lee E, Miller Long-term stability of cortical population dynamics underlying consistent behavior. Nature Neuroscience, 2602),260-770, February 2020.
[2] Mark M. Churchland, John P. Cunningham, Matthew T. Kaufman, Justin D. Foster, Paul Nuyujukian, Stephen I. Ryu, and Krishna V. Shenoy. Neural population dynamics during reaching. Nature, 487(7405):51-566, July 2012.
[3] Edward H. Nich, Manuel Schottdorf, Nicolas W. Freeman, Ryan J. Low, Sam Lewallen, Sue Ann Koay, Lucas Pinto, Jeffrey L. Gauthier, Carlos D. Brody, and David W. Tank. Geometry of abstract learned knowledge in the hippocampus. Nature, 595(7865):80-84 July 2021.
[4] Wei Guo, Jie J. Zhang, Jonathan Newman, and Matthew A. Wilson. Latent learning drives sleep-dependent plasticity in distinet CA1 subpopulations. Preprint, Neuroscience, February 2020.
[5] Gurjeet Singh, Facundo Memoli, Tigran Ishkhanov, Guillermo Sapiro, Gunnar Carlsson, and Dario L. Ringach. Topological analysis of population activity in visual cortex. Journal of Vision, 8(8):11 1, June 2008.
[6] Dean A. Pospisil and Jonathan W. Pillow. Revisiting the high-dimensional geometry of popula- tion responses in visual cortex, February 2024.
[7] Adrien Peyrache, Marie M. Lacroix, Peter C. Petersen, and György Buzsáki. Internally organized mechanisms of the head direction sense. Nature Neuroscience, 18(4):569-575, April 2015.
[8] Rishidev Chaudhuri, Berk Gerçek, Biraj Pandey, Adrien Peyrache, and Ila Fiete. The intrinsic attractor manifold and population dynamics of canonical cognitive circuit across waking and sleep. Nature Neuroscience, 22(9):1512- 1522- September 2019.
[9] Richard J. Gardner, Erik Hermansen, Marius Pachitariu, Yoram Burak, Nils A. Baas, Ben- jamin A. Dunn, May-Britt Moser, and Edvard Moser. Toroidal topology of population activity in grid cells. Nature, (1277889):1233111 February 2022.
[10] Mehrdad Jazayeri and Srdjan Ostojic. Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity. Current Opinion Neurobiology, 70:113-120. October 2021.
[11) Peiran Gao, Eric Trautmann, Byron Yu, Gopal Santhanam, Stephen Ryu, Krishna Shenoy, and Surya Ganguli. theory of multineuronal dimensionality, dynamics and measurement. Preprint, Neuroscience, November 2017.
[12] Yeon Chung and L. F. Abbott. Neural population geometry: An approach for understanding biological and artificial neural networks. Current Opinion in Neurobiology, 70:137-144, October 2021.
10

---

[13] Torkel Hafting, Marianne Fyhn, Sturla Molden, May-Britt Moser, and Edvard I. Moser. Mi- crostructure spatial map in the entorhinal cortex. Nature, 436(7052):0018806 August 2005.
[14] Hanne Stensola, Tor Stensola, Trygve Solstad, Kristian Frøland, May-Britt Moser, and Edvard I. Moser. The entorhinal grid map is discretized. Nature, 492(7747)::22-75, December 2012.
[15] Charlotte N. Boccara, Michele Nardin, Federico Stella, Joseph O' Neill, and Jozsef Csicsvari. The entorhinal cognitive map is attracted to goals. Science, 363(64334:11443-444, March 2019.
[16] William N. Butler, Kiah Hardcastle, and Lisa M. Giocomo. Remembered Reward Locations Restructure Entorhinal Spatial Maps. Science (New York, N.Y.), 36666644444444)44444222 March 2019.
[17] Julija Krupic, Marius Bauza, Stephen Burton, and John OeKeefe. Local transformations of the hippocampal cognitive map. Science, 359(6380): 14306668))1113-1144444 143-11466 March 2018.
[18] Gily Ginosar, Johnatan Aljadeff, Liora Las, Dori Derdikman, and Nachum Ulanovsky. Are grid cells used for navigation? On local metrics, subjective spaces, and black holes. Neuron, pages 60896-6273323)002225
April 2023.
[19] Dehong Xu, Ruiqi Gao, Wen-Hao Zhang, Xue-Xin Wei, and Ying Nian Wu. Conformal Isometry of Lie Group Representation in Recurrent Network of Grid Cells, November 2022.
[20] Erik Hermansen, David A. Klindt, and Benjamin A. Dunn. Uncovering 2-D toroidal representa- tions in grid cell ensemble activity during 1-D behavior, November 2022.
[21] Alexei Samsonovich and Bruce L. McNaughton. Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model. Journal of Neuroscience, 17(55):5900-5920, August 1997.
[22] Yoram Burak and lla R. Fiete. Accurate Path Integration in Continuous Attractor Network Models of Grid Cells. PLoS Computational Biology, 5:e 100291, February 2009.
[23] O' O'Keefe and J. Dostrovsky. The hippocampus a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Research, 34(1):171-173 November 1971.
[24] John O' Keefe and Lynn Nadel. The Hippocampus as Cognitive Map. Oxford University Press, 1978.
[25] Marianne Fyhn, Torkel Hafting, Alessandro Treves, May-Britt Moser, and Edvard Moser. Hippocampal remapping and grid realignment in entorhinal cortex. Nature, (46(7132):190-11) 194, 2007.
[26] KiJung Yoon, Michael A Buice, Caswell Barry, Robin Hayman, Neil Burgess, and Ila R Fiete. Specific evidence of low-dimensional continuous attractor dynamics in grid cells. Nature neuroscience, 16(8):1077-1085 2013.
[27] Mariana Gil, Mihai Ancau, Magdalene I Schlesiger, Angela Neitz, Kevin Allen, Rodrigo J De Marco, and Hannah Monyer. Impaired path integration in mice with disrupted grid cell firing. Nature neuroscience, 21(1):81-91, 2018.
[28]
Christopher J. Cueva and Xue-Xin Wei. Emergence of grid-like representations by training recurrent neural networks to perform spatial localization, March 2018.
[29] Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel, Martin J. Chadwick, Thomas Degris, Joseph Modayil, Greg Wayne, Hubert Soyer, Fabio Viola, Brian Zhang, Ross Goroshin, Neil Rabinowitz, Razvan Pascanu, Charlie Beattic, Stig Petersen, Amir Sadik. Stephen Gaffney, Helen King. Koray Kavukcuoglu, Demis Hassabis, Raia Hadsell, and Dharshan Kumaran. Vector-basee navigation using grid-like like representations artificial agents. Nature, 557(7705))4229 433, May 2018.
[30] Ruiqi Gao, Jianwen Xie, Xue-Xin Wei, Song-Chun Zhu, and Ying Nian Wu. On Path Integration of Grid Cells: Group Representation and Isotropic Scaling, November 2021.
11

---

[31] Ben Sorscher, Gabriel C. Mel, Samuel A. Ocko, Lisa M. Giocomo, and Surya Ganguli. A unified theory for the computational and mechanistic origins of grid cells. Neuron, 111(():21-137..13. January 2023.
[32] Rylan Schaeffer, Mikail Khona, Tzuhsuan Ma, Cristóbal Eyzaguirre, Sanmi Koyejo, and Ila Rani Fiete. Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells, November 2023.
[33] Markus Borud Pettersen, Vemund Sigmundson Schøyen, Anders Malthe-Söreesseee and Mikkel Elle Lepperød. Decoding the Cognitive map: Learning place cells and remapping, March 2024.
[34] Rosamund F. Langston, James Ainge, Jonathan J. Couey, Cathrin B. Canto, Tale L. Bjerk- nes, Menno P. Witter, Edvard I. Moser, and May-Britt Moser. Development of the Spatial Representation System in the Rat. Science, 328(598)) 1580, June 2010.
[35] Joshua Jacobs, Christoph T Weidemann, Jonathan F Miller, Alec Solway, John Burke, Xue- Xin Wei, Nanthia Suthana, Michael R Sperling, Ashwini D Sharan, Itzhak Fried, et al. Direct recordings of grid-like neuronal activity in human spatial navigation. Nature neuroscience, (89):118--199 2013.
[36] Vemund Schøyen, Markus Borud Pettersen, Konstantin Holzhausen, Anders Malthe-Sorenssee and Mikkel Elle Lepperød. Coherently Remapping Toroidal Cells But Not Grid Cells are Responsible for Path Integration in Virtual Agents, August 2022.
[37] Julija Krupic, Neil Burgess, and John O' Keefe. Neural representations of location composed of spatially periodic bands. Science (New York, N.Y.), 337(6096) August 2012.
[38] Leland Mclnnes, John Healy, and James Melville. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction, September 2020.
[39] Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD'96, 96,9, pages 226-231, Portland, Oregon, August 1996. AAAI Press.
[40] Francisco Acosta, Sophia Sanborn, Khanh Dao Duc, Manu Madhav, and Nina Miolane. Quanti- fying Extrinsic Curvature in Neural Manifolds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 610-619, 2023.
[41] Chad Giusti, Eva Pastalkova, Carina Curto, and Vladimir Itskov. Clique topology reveals intrinsic geometric structure in neural correlations. Proceedings of the National Academy of Sciences, 112(44): 1455-13460, November 2015.
[42] Gard Spreemann, Benjamin Dunn, Magnus Bakke Botnan, and Nils A. Baas. Using persistent homology to reveal hidden information in neural data, October 2015.
[43] Julián Burella Pérez, Sydney Hauke, Umberto Lupo, Matteo Caorsi, and Alberto Dassatti. Giotto-ph: A Python Library for High-Performance Computation of Persistent Homology of Vietoris-Rips Filtrations, August 2021.
12

---

S1 Proofs of Results from the Theory Section
Here we provide more detailed discussion and derivations of the theory introduced in Sec- tion B
S1.1 Diffusion of rate maps
We start by proving the following proposition, about a single grid cell i. This proposition refers to the definition of T from Eq. 4
Proposition 1. The deformation the grid cell firing fields ri by D given by + 0= -r defines right group action of the group of diffeonorphisms over the space of firing fields. The firing energy budget is invariant of this group action.
Proof. We first prove that defines a valid right group action, by proving that it verifies the group
action's 's axioms: identity and compatibility.
The identity diffeomorphism + = maps every point to itself ((x) z, has Jacobian equal to the identity matrix with determinant Thus, (z) det id' (x)]|; id(x) [1]r;(x) = r.(2))
This shows that the scalar field remains unchanged under the identity diffeomorphism, satisfying the identity axiom of group action.
The composition of two diffeomorphisms t and acts on the firing fields as:
Woo (x) = det V o 2))(x) ir (v o (x)
By the chain rule for determinants of Jacobians, we get:
fo (x) = det ('((())) det ('(x)r; o ( (x).
This expression matches the successive application of the transformations:
(rs v) ((x) det & (x) ())()))
det Φ (xxx)) det ((((x))) ((((2))
rs (V (x)
This shows the compatibility property for right group action. Further, the firing energy budget an invariant of the group action. Indeed, by definition of multivariate change of variables, we have
/ Jeeee
r (x)dr = JEER
det s ('(x))r; ((x)dc
r;(x)dd
((22)
re(x)dx
JZER
for every grid cell i.
Next, we prove the result regarding the deformation of the neural manifold.
The deformation of the neural manifold R by & given by t R T defines right group action of the group of diffeomorphisms over the space of neural manifolds. The barycenter and the topology of the manifold are invariants of this group action, provided that the firing fields are regular enough.
Proposition 2.
Proof. We first prove that defines valid right group action, by proving that it verifies the group action's axioms: identity and right compatibility. These axioms are verified, using the same arguments the ones used in the proof of group action over the firing fields above.
Next, we prove that the barycenter of the neural manifold is unchanged by the deformation. The barycenter before deformation is:
f R(xda,
(SI)
13

---

and, after deformation:
J
/ | det D' (2) (())dx = f R(æ)dz,
JZERA
R
(z)dx
=
JZERS
(S2)
by definition of the multivariate change of variables. Thus, the barycenter is unchanged.
Next, show that the topology of the manifold M defined by R is the same the topology of the manifold M defined by R.
Two manifolds are topologically equivalent (homcomorphic) if there exists an homeomorphism V : M + M between them, i.e., a map v that is continuous, bijective, with continuous inverse.
Consider the map V M - M defined for p R(x)) E M, where the unique pre-image of p by injectivity, as:
(p) , (x) del (x))rr ((x) det 4 (R (P))
We show that Vi is continuous. The diffeomorphism smooth on TR2 by definition. The function R is also smooth since every firing field is smooth. Next, R-1 1 is also considered smooth, that o Rt1 is smooth. Thus, the map det ((((z)) is smooth as it is the determinant of a smooth function. Thus, v smooth and therefore continuous.
Further, V is bijective: & and R-1 are bijective by definition of diffcomorphism and because we restricted R to its domain of injectivity. The determinant is non for the same reason. Thus, V/ is bijective. We assume that R is regular enough so that the inverse of Y is continuous as well. Thus, W an homeomorphism and the two manifolds are topologically equivalent.
S2
Refinement on theory following rebuttal
S2.1
Diffused Units: Isotropic Smoothing of Firing Fields
We show how diffused units, observed practice in Section 4 lead smaller neural manifolds in neural state space.
To this aim, we represent the firing field one grid cell as a mixture of K Gaussians, centered at locations Pi...., PK that are regularly distributed along a grid. That is, for E R::
r(x) Σ G(p))o))xx), where: G(uR, (z)
1
exp
| Mall3)
(S3)
2no²
20
Next, we describe the transformation of the grid cell units into diffused units as a convolution with a 2D isotropic Gaussian filter G(O, o²) for some variance o². Consequently, instead of writing this transformation as the action diffcomorphism 4 we write is as the action E R..
r (x) (G(0, (x), Vo R..
(S4)
We now prove our main result diffused units.
Proposition 3. When the original grid cell units become diffused units after integration of reward: (i) the energy budget is unchanged, and (ūi) the size of the original neural manifold decreases, where the size is defined as S = max,era [R(z))) with R(x) (x),....,x(())) for N units.
Proof. (i). We first prove that the energy budget is unchanged.
14

---

The energy budget before the introduction of reward, for original unit, is:
E = J JEERR r(x)dx
1
Σ G(pu, (z)
K
JEER?
K
1
G(us, (2))
K
AERR
1
ldr
K
1.
To compute the energy budget after the introduction of reward, for diffused unit, we first provide an alternative formula for (x)). We have;
r (x) (G(O, o²) xr) +)) (x)
K
(((),0)) Σ G(u), (x)
K k=1
(G(0, ((1R;,0)) (x)
K Σ Giu 0² + (z),,,
K
where have used the fact that the convolution is linear operation, and the fact that the convolution of two Gaussians a Gaussian with added means and variances.
The energy budget after the introduction of reward, for a diffused unit, is therefore:
E
S T (x)ddr
1
Σ G(uu, (x)
JOER2
K
Therefore, the energy budget conserved: F
(ii). Next, we prove that the size of the neural manifold corresponding to the diffused units is smaller than the size of the neural manifolds of the original units.
Consider a neural manifold corresponding to N units. The size of the manifold is the maximum value taken by any of its N units, across the domain R².
S max [[(x)) = max max maxr, (I)) ZER2
(S5)
2E3" TEN
Since all units have the same profile, their maxima are equal. Hence, we compute the maximum of one original unit and of one diffused unit. Additionally, each unit is mixture of K Gaussians. Assuming that the Gaussians are disjoints enough, which is the case in practice, we compute the maximum of unit the maximum of one of its Gaussians.
Accordingly, the size of the neural manifold corresponding to the original units is:
1
max r r(x) max G(u1: K 2To?
=
BER²
Next, the size of the neural manifold corresponding to the diffused units is:
1
S = max r(x) = max G(m,, o² + +z K 2n ( + o
=ER²
K
xER²
Thus, have S < S, which proves that the neural manifold has smaller size.
15

---

S2.2 Band Units: Anisotropic Smoothing of Firing Fields
We show how band units, observed in practice in Section 4 lead to a torus manifold that collapses into a circle manifold. As in the previous subsection, we represent the firing field of unit mixture of K Gaussians as in Eq. S3
Next, we describe the transformation of units into band units as a convolution with a 2D anisotropic
Gaussian filter G(O, 2) for Σ E Sym(2) with eigenvalues (0, o²) with o² >> 1.
Consequently, instead of writing this transformation as the action of a diffeomorphism Φ, we write is the action of Σ E Sym(2). (x) - (r) Ve E R??
(S6)
We now prove main result band units.
Proposition 4. When the original units become band units after integration of a reward: (i) the energy budget unchanged, and (ii) the original neural manifold (a torus) collapses into a circle.
Proof. 1. We first prove that the energy budget is unchanged.
We recall from the previous subsection that the energy budget before the introduction of reward, for original unit, is:
F
r(æ)da =1
ZEB
To compute the energy budget after the introduction of reward, for band unit, we first provide an alternative formula for r"(r)) (I)) We have:
xr) r (z) (G(O, =)) (z)
K
1
G(O,
Σ
G(px: of) (x)
K
k=1
K
(G(O, G(uus of)) (x)
K
E
K
where have used the fact that the convolution is linear operation, and the fact that the convolution of two Gaussians a Gaussian with added means and variances.
The energy budget after the introduction reward, for diffused unit, is:
E
r" (x)dx
K Σ k=1
G(pn. E E
Jeere K
Therefore, the energy budget conserved: E
2. Next, prove that neural manifold evolves from torus circle.
We do this by proving that the application of a Gaussian filter is continuous operation in the variable a², For this, we use the spectral theorem, which states that every real, symmetric matrix is diagonalizable in an orthonormal basis, to write Σ as:
Σ = Rdiag(O, R²)R
(S7)
16

---

where is a 2D rotation matrix. Accordingly, we see that a impacts the firing field r continuously, as follows:
K G(uk. Rdiag o² + (z))
1 K
(x)
Thus, the evolution of the neural manifold as a increases is continuous. At the limit, where o² + oo we have the band units observed in practice.
Next, we prove using group theory that band units generate a circle manifold. (TODO: Add theory written by Fran).
Consequently, the progression from the original units to the band units is progression from torus manifold to a circle manifold.
S2.3 Ring Units: Angular Smoothing of Firing Fields
We show how ring units, observed in practice in Section can be described via an angular smoothing of firing fields. As in the previous subsection, we represent the firing field of one unit as a mixture of K Gaussians as in Eq. S3
Next, describe the transformation of units into ring units as convolution with 2D anisotropic variable Gaussian filter. In this convolution, the kernel varies depending on where it is being applied, i.e. its position which will express in polar coordinates with (p, 0).
In other words, at (p, ()) fixed, the function - Gvar(u; p. 0) a Gaussian kernel with mean and variance that may depend on p and 0. In our case, given reference radius pa > O, the dependency of the mean and variable on (p, 0) has the form:
-- ( m)) Vu R2 (S8)
exp where is the Kronecker symbol and Eg = Rg. di..gaa (0, R.g"
Gvar (w; p. S (w) 6p=po-
2T / det Mg
The mean of the Gaussian kernel always O, but the covariance matrix depends 6. The Gaussian kernel itself is 0 if p different than the reference radius Po.
This variable kernel can be used to perform an anisotropic smoothing in the direction @ for a ring of radius Pa. via convolution. For simplicity, we consider that the center of the ring is also the center of the polar coordinate system. We write the convolution with this variable kernel as:
/
[
p0 ( (Grar r) (v),
Gru) w;))ddu
r(u) u, 0)du, (S9)
JUER
JUERA
where write in polar coordinates (p,
In Figure ST we verify empirically that this formulation creates the rings that we observe in practice.
Ring Units with ring radius = 10
Ring Units with ring radius = 20
Standard deviation = 3 Standard deviation = 5
Standard deviation = Standard deviation = 5
Figure S1: Convolution with a variable Gaussian kernel creates the ring units observed in practice. From left to right, we vary the radius Po and the standard deviation of the variable Gaussian kernel.
17

---

S3 Network details & hyperparameters
All code used for this paper is publicly available at: https://githe tig........ com/ wwaeeerriiinnteeee haaptttrriiitteeleeeeeee/eeeeeee
Grid cells in piRNNS Gao al. [30] and Xu et al. [9] introduce a framework where trans-
formations in the environment are related to transformations in neural state space. That is, for
displacement Az, the neural representations r transform as:
r(x + Ar) = Fa=(r(x)),
(S10)
where F is assumed to respect conformal isomelry-a small displacement vector Ar at x in physical space conformally mapped to small displacement vector r(x + Ax) in neural state space. This can be understood on x, but is independent of the direction of Ar. Xu et al. [19] approximate the transformation FA (r))) using an RNN:
"magnification" of the displacement by constant factor s(z) that may depend
(S11) where W E NNN is the recurrent weight matrix and U E RNx2 is the input weight matrix with Ax called the input. The neural representations r(x) will become grid cells after training. To incorporate the fact that biological grid cells are organized into discrete modules [4] they assume that the representations r(z) are divided into sub-vecton corresponding to modules, and therefore they assume that W is Block-diagons matrix and U is divided into sub-blocks by row.
((++Ar) = ReLU(Wr((z) + UAr),
Next, we provide expressions for the remaining loss terms from Eq. The kernel term is:
Lememl = Ez [[l - Qr(z)1
(S12)
Since Φ is fixed, this term induces the linear read-out of grid cell representations r with matrix Q to match the Gaussian kernel.
The conformal isometry term is:
Loonformal = En [S (2) (x)r
(S13)
where $AZ (ar) ([]r(x)) r(x + Ar) |r]l) This term enforces that the magnitude of displace- ments in neural state space induced by displacement in the real environment independent of the direction of the displacement at each point. Thus, the grid cell representation taken as a map from R²² to IR is conformal (angles are preserved locally).
The regularization term is:
Legularizattin = 1
(S14)
This term can interpreted as inducing sparsity in the place cell activity.
S3.1 Pre-training Details
We pre-trained the model from 19 for 25,000 epochs with the task, architecture, and training hyperparameters originally used in their work, specified in Table [ Every 500 epochs, we extracted the network activities and averaged over 100 trajectories.
Once the network was trained and learned units with a high degree of hexagonal periodicity across multiple modules, we modified the loss term (see Eq. 7 and initiated a new phase of training, saliency training. We ran 667 runs spanning different experimental conditions (see Table 2 and hyperparamater search (see Table
In our analysis in the main text we use Gaussian saliency kernel located at the center of the environment, with a. = 0.05, shown in Fig. S2
S4 Identification of neuron subpopulations
The activity of each unit can be visualized as (res X res) rate map, with res 40. Building upon the methodologies in (9. we attempt to discover subpopulations in the network by clustering RNN units based on their spatial autocorrelations. We begin by computing the spatial autocorrelations of all 1800 RNN units, where each spatial autocorrelation is a matrix of size (res - 1) x (res - 1). We proceed by flattening the matrix, yielding 1800 points in a IR (res- (res=1) and reduce the dimension to 2D using UMAP [38]. Finally, we use DBSCAN 39 to cluster the points the 2D UMAP projection, resulting in our final unit clusters. These clusters correspond to the usual notions of grid cell module. The 2D UMAP projection colored according to OBSCAN-obtaing clusters is shown in Fig. S3
18

---

Architecture
N Ngrid
1800
RNN step
10
place cell
0.07
Number of modules
150
Resolution (res)
40
Training
Number of epochs
25,000
Learning rate
0.006
Optimizer
adam
Batch size
10,000
kernel weight
1.05
error weight
0.1
conformal weight
0.005
Q reg. weight
0.2
Path length
30
Table I: Pre-trained model parameters.
Experiments
Ngrid
1800
RNN step
[10,20,40,6]
error weight
(0.1.0.3.0.500.9222
[1,10, TO00]
F.
((0.5.0.5) (0.05.0.10015.002200....
0+
Table 2: Different experimental conditions.
S5 Geometric Methods in Neuroscience
There exist many approaches for revealing manifold structure neural population activity which rely dimensionality reduction techniques like PCA, UMAP, MDS, Isomap. LLE, and ISNE. These techniques can reveal the existence of lower-dimensional structure in neural population activity; however, these methods may misrepresent manifolds with non-trivial topological structure, like circles, spheres, and tori. Topological data analysis (TDA) methods like persistent homology hrevealed [8 9 However, these methods do not reveal many geometric properties of neural manifolds including notions of distance, angles, or curvature. Recent methods (40) quantify the extrinsic curvature neural manifolds leveraging tools from differential geometry.
S5.1 Topological Data Analysis
Topological Data Analysis (TDA) has been used extensively study the structure neural population
41 42. In this work leverage the parallelized CPU computations provided by giotto-ph [43] to compute persistent homology based on Vietoris-Rips filtrations. We begin with projection of the module activities to 6 the dimensional space defined by PCA. We then compute persistent homology setting the prime field p = 2, and Euclidean distance as the metric.
activity [5| 8 9
S6 Limitations
We introduce saliency-trained piRNN framework building off the model from [19 and 30) Many other works propose their own versions piRNNs with different specifications, which we have not fully explored. In particular the recent work of 32 introduce novel self-supervised piRNN
Experiments
Learning rate
[3e-4, 6e-4
Path length
[50.75.100]
Table 3: Hyperparameter search for saliency training.
19

---

31
30
64 or
Figure S2: Saliency kernel for analysis in Sec. 4.1
A
2D UMAP Embeddings of
B
Exemplary unit rate maps
Spatial Autocorrelations
Cluster 7
15
UMAP2 2
IDs
Cluster
Cluster 1
Cluster
Cluster
Cluster
Cluster
Cluster
-10
Cluster
15
UMAP
Figure S3: Identification of network subpopulations by clustering of unit rate maps.. A 2D UMAP embeddings of spatial autocorrelations of rate maps of all RNN units, colored according to cluster membership determined with DBSCAN clustering on embeddings. B Rate maps typical units in cluster (top) and cluster (bottom).
framework which considerably differs from the rest, and future work should explore the effects of saliency training on this novel type of piRNN. Furthermore, while we intend bridge the gap between path-integrating artificial neural systems and observational evidence from experiments in navigational neuroscience, the present work does not directly analyze neural data from real animals.
S7 Reproducibility
In training of our piRNNs and conducting our analyses, we made use of 10 NVIDIA A100 GPUs and 20 NVIDIA A100 GPUs spread over clusters.
20

---

A
B
SOHOO 0000.
00g0
190
900
2000
Figure S4: A. Rate maps for module 7 of piRNN with frozen linear read-out during saliency training. We observe the stereotypical deformations from the pure hexagonal grids described in Sec. 4 and the toroida topology preserved. B. Rate maps for module piRNN with trainable linear read-our during saliency training. The deformations are more extreme, and toroidal topology of the module destroyed.
21

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer (Yss], , [No] or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No] ", it is perfectly acceptable answer "[No] provided a proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] "r "[NA] is not grounds for rejection. While the questions are phrased in binary way, acknowlede that the true answer is often
nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper the supplemental material, provided in appendix. If you answer [Yes] a question, in the justification please point to the section(s) where related material for the question can be found.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract describes the results presented in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please see Section S6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
22

---

The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification:
We provide the assumptions, conjectures, and proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: code used for the paper is publicly available on Github as specified.
Guidelines:
23

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
All our code used in this paper is publicly available on GitHub.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc)
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https htaipps.cc/pullllgggiddd/////ooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
24

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, we included these details in the supplementary.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We include the error bars for the TDA Figs.
and
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of emrrrs is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
25

---

For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We described our compute resources in Section S7
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http::/ppe.aaapp...ccc///////cca
Answer: [Yes]
Justification:
We confirm that this is true.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
This is not applicable.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation. generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
26

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We properly cited all work whose codebase we used.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwpppswiptttaeeeeaa www.wwtthhc....mm com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
27

---

13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/cooeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This is not applicable for our work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According the NeurlPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
28

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "wbDG9XKIEU", "reviewer": "Reviewer_MTEF", "summary": "This paper provides a mathematical theory of how spatially local reward distortions can lead to global representational distortions in grid cells.", "questions": "- Could the authors please clarify equations (1) and (3)? I am confused by them. In particular, (1) seems to suggest that $\\phi(x) = Qr(x)$. But equation (3) suggests that they are not equal (otherwise $L_{error}$ would always be zero). Perhaps relatedly, where does the position estimate $\\hat{x}$ enter into the loss?", "limitations": "Yes.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The paper is extremely well-written, self-contained, nicely motivated from biology, and mathematically elegant. As someone who does not know the grid cell theory literature that well, I really appreciated learning from this paper.", "weaknesses": "The paper seems to be somewhat similar to previous work in the literature (e.g., Xu et al, Conformal Isometry of Lie Group Representation in Recurrent Network of Grid Cells), impacting its novelty. Similarly, a look at the literature shows that there are hundreds of theory papers on grid cells in the literature (which makes sense, they are a beautiful and simple phenomenon), so the paper is not particularly original in this regard.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the thoughtful review, and address your comment and questions below. We look forward to hearing from you in case you have any additional comments!\n\n**Answer to weakness on Originality/Novelty:** We agree that the originality and novelty of our approach could have been made clearer in the original paper, which did not emphasize these points enough. In the final version, we propose to add a subsection 2.3 in Section 2 on Background & Related Works to explain the following:\n\nIn this work, we use the supervised piRNN approach as a framework to study the deformations in grid cell firing patterns reported in the experimental neuroscience literature. We do so by modifying the piRNN’s supervised loss in an interpretable way to account for varying saliency of space. Particularly, our saliency training allows for systematic study of two highly entangled behavioral variables: i) where the rewards are located and ii) which locations the agents have the tendency to visit the most. To our knowledge, this approach is completely novel in the literature. \n\nIn fact, in our new Fig. N4, we show that the grid cells contributing to the coding of the origin (the most visited location), but not the saliency map (the reward location), are robust to global distortions during the saliency training. In other words, as a secondary objective is introduced, grid cells with projections to the place fields at the edges are more likely to become distorted. To our knowledge, we are the first to make this prediction, which can be tested experimentally, for instance, by introducing both reward and landmark locations. Furthermore, we introduce a mathematical, geometric framework to explain and quantify the firing patterns’ deformations observed in our experiments; this is also novel and original. Finally, we show “that global distortion (geometry) can be added while maintaining the toroidal topology for path integration” (citing reviewer yMbD), which is true once the read-out weights to place fields are frozen, but not otherwise!\n\nOne related work is Nayebi, et. al. “Explaining heterogeneity in medial entorhinal cortex with task-driven neural networks” (NeurIPS 2021). The authors were also motivated by results from experimental neuroscience on reward-modulated neural responses by grid cells. While not the central question of their work, Nayebi, et al. propose a modification of training piRNNs to account for reward modulation: they modify the way training paths are generated in the presence of reward. Specifically, the authors modified the training paths ge to be either (1) direct movement toward a reward location, called “pure exploitation”, (2) random walk, as is standard in the piRNN literature, called “pure exploration”, or (3) an intermediate policy consisting of a mix of movement toward reward and random walk exploration. Their results are consistent with the existence of nontrivial reward-modulated response changes in piRNNs. However, they did not seek to characterize the specific deformations to the neural responses, as we do in our work.\n\n**Answer to question on Equations (1) and (3):** Thank you for pointing this out. On the one hand, there is $\\phi(x_t)$, which is the vector of place cell activity corresponding to position $x_t$. This is an “encoding” of 2D position into a place cell code. \n\nOn the other hand we have $\\hat{\\phi}_t = Qr_t$, which is a linear readout of the recurrent grid cell activity $r_t$. We apologize for the typo in Equation (1), which thus should have been: $\\hat\\phi = Qr$ and $\\hat x = \\text{argmax} \\hat \\phi.$\n\nSuccessful path integration here means that $\\hat \\phi_t \\simeq \\phi(x_t)$, captured in the $L_{error}$ term. To further highlight this point, we can rewrite Equation (3) as\n\n$$L_{error} = \\sum_{t=1}^T ||\\phi(x + …) - \\hat \\phi (x + …)||^2 = \\sum_{t=1}^T ||\\phi(x + …) -Qr (x + …)||^2.$$\n\nWe thank you for catching this.\n\n**Conclusion**\n\nWe once again thank you for your time and attention. We hope that we were able to resolve your concern. If not, we look forward to discussing these topics further! If you believe we were able to adequately address your concerns, we hope that you will consider supporting our work with a strong acceptance!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S2cXWqGzr1", "reviewer": "Reviewer_yMbD", "summary": "The paper investigates the phenomenon of grid cell distortion in rewarded environments, a topic of interest in neuroscience. The authors propose a theoretical framework to understand how the 2D firing fields of grid cells deform while preserving their topological structure in high-dimensional neural space. By providing a position dependent weighted loss term, spatial saliency loss, the experiments showed that global distortion (ring, diffused and band patterns) appeared in the firing patterns of grid cells. The authors also provided detailed analyses of piRNNs under various conditions, including frozen and trainable place cells.", "questions": "1. In the experiment, s(x) is defined as a Gaussian and the mean is at the center of the map, which gives more weight on the transformation at the center area. I may expect to see firing patterns that magnify towards the center. Do you have any insights on why the distortion seems to be uniformly appeared on the whole map? \n\n2. For s(x), have the authors tried to change the scale or the position of the distribution to see how the patterns changed? \n\n3. I'm curious if the authors tried different number of grid cells or modules in the experiments. Does the distortion need a larger number of cells to form?\n\n4. In Figure 5, grid cells in the same module may emerge different distortion patterns. For example, in module 2, there are diffused and ring units. Are there any insights why this happens?", "limitations": "The limitation of this work is well illustrated in the appendix.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well written with a comprehensive review of related works and makes a good point on a interesting topic. \n\nThe authors propose a theory which offers a potential explanation of deformation. It shows that global distortion (geometry) can be added while maintain the toroidal topology for path integration. This finding could inspire future research in the field. \n\nI really like the simplicity of this work. The introduction of the spatial saliency loss (Eq. 7) is elegantly simple, building upon previous literature in a clear and interpretable way. The author leverages the flexibility of piRNN and specify an explicit scale at different locations while preserving the topological properties.", "weaknesses": "The work is all based on simulated results and evaluate on mostly on the change of the firing patterns. It would be better for the authors to show some comparison with the \"real\" data from rodent brain, i.e. the diffused or band unit is also found in biology experiments. If the simulation results can match with the experiment in some metrics, it will greatly enhance the confidence of this work. \n\nMore experiment can be provided with the change of the settings, such as different numbers of grid cells or change of s(x), to show the generality of the theory and method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you very much for the thorough review and for your time. We address your comments and questions below.\n\n**On real neuroscience experiments:**\n\nYou make the very important point that linking the theory with “real” data from rodent brains is important to increase the confidence of this work. Thank you for bringing it up. We think that this point deserves additional attention and we discuss it below.\n\nThe existing experimental neuroscience data on such reward-modulated deformations of grid cells firing patterns is limited in number and in the strength of the conclusions. We are aware of the results from [Boccara et al. 2019, Butler et al. 2019, Krupic et al. 2018, and Wang et al. 2021]. Boccara et al. 2019 in particular motivated our approach and we explained and discussed its results with our theoretical framework. To summarize: Boccara et al. observed an attraction of grid cells firing fields towards the reward – a phenomenon called “attracted units” in our paper. Furthermore, band units are well described in the literature – see Krupic et al. 2012. Are you aware of any other recent public datasets that may be relevant? If so, we would be happy to discuss them as well.\n\nDue to the current limited experimental data, we chose to study in silico ANN-based experiments inspired by the existing, real neuroscience experiments. Therefore, the scope of our work is: we investigate how the representations learned by artificial path-integrating RNNs (piRNNs) change to account for “saliency” in the environment, in a way that is (1) interpretable, and (2) inspired by neuroscience experiments. We observe that after training with saliency conditions, our piRNN learns a more diverse set of neural representations (ring neurons, diffused neurons, band neurons) compared to observed in Xu et al. and Gao et al. earlier – while still maintaining toroidal topology. We then build a theoretical framework to explain how different deformations in the individual rate maps affect the geometry of the neural representation manifold, which is novel in the literature.\n\nUnfortunately, being a theory lab, we cannot perform additional experiments on real rodents ourselves. Yet, we believe that our geometric study of piRNNs already provides interesting results for the study of RNNs, which is an important field of research in AI/ML. Additionally, we hope that our work will inspire experimental neuroscientists to perform corresponding experiments on rodents. In that perspective, it might even be preferable that theory and experiments come from two separate research teams, as its final validation on real data would even be stronger. \n\n**On additional piRNN experiments:** Thank you for proposing this more comprehensive set of experiments. We have run them; please see the global response on experiments. We believe that these additional results strengthen our paper and we appreciate your suggestion.\n\n**Answer to question 1.): why such global distortions?**\n\nYou asked why the distortions of grid cells firing patterns appear uniformly, while the reward (saliency) is local and localized at the center of the space. This is a great question! We also expected to see some local distortions of the grid cell firing patterns, given the claims from neuroscience experiments in Boccara et al. However, to our initial surprise, we did not observe this kind of local “magnification” or deformations in any of our experiments. Instead, we observed more global changes in the rate maps – which inspired the title of the work, “Global Distortions from Local Rewards”. Thus we challenge the conventional interpretation that reward modulation results in local magnification of the grid cell lattice as too simplistic an explanation. Indeed, because grid cells provide a global code for space, we propose that reward modulation should result in global deformations of spatial responses in MEC, as observed in our piRNN experiments.\n\n**Answer to question 2. on piRNN experiments that vary the saliency map**\n\nThank you for proposing additional experiments with variations to the saliency map s(x)! Please see our results in the experiments section of the global response. \n\n**Answer to question 3. on the link between distortion and number of grid cells/modules**\n\nWhile we have not run experiments varying the number of grid cells, this is an interesting modification we will investigate in the near future. We would like to emphasize that the number of modules, as we have defined them, is an emergent property of the trained network and thus we do not have direct control over it. How the number of modules that emerge depend on various experimental parameters is a very interesting question that needs to be further explored, but is out of scope here.\n\n**Answer to question 4. on why different distortion patterns appear?**\n\nThank you for the great thought-provoking question. We had not considered why different distortion patterns indeed jointly appear, i.e., why we observe both diffused and ring units after introduction of the reward. Your question also prompts the follow-up question: what role can diffused units, band units, and ring units play? To bring some insights to this question, in our new Fig. N4, we show that the grid cells contributing to the coding of the origin (the most visited location), and not the center of the saliency map (the reward location), are robust to global distortions during the saliency training. In other words, as a secondary objective is introduced, grid cells with projections to the place fields at the edges are more likely to become distorted.\n\n**Conclusion**\n\nWe hope that we were able to address your comments and resolve your questions. These discussions have been beneficial for us, and we will gladly incorporate them into the final version of the paper. If you have any remaining questions, we'd be happy to discuss further!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5v6GoGdEQA", "reviewer": "Reviewer_STz6", "summary": "The overarching goal of this paper is to provide a theoretical framework to understand the experimentally observed effects of rewards on grid cells, which have historically been known to encode spatial information. To do so, they focus on path-integrating RNNs whose units behave like grid cells post training. They show that modeling the influence of rewards as a diffeomorphism of the environment (with constant firing rate budget per neuron), results in changing fire rates of grid cells such that the underlying manifold (toroid) remains unchanged in topology, however its geometry get distorted. That is, grid cells are able to preserve spatial information as before, while accounting for rewards through a global distortion. They further fine-tune piRNNs with a modified loss that captures rewards, and show that under the right training scheme the resultant grid cell firing patterns change in a way that only changes the geometry and not the topology of the neural manifold. In particular, they observe ring like structures, diffused firing grids and band like structures in the presence of rewards in the environment.", "questions": "1. With respect to continual learning, would this scheme be capable of training a salience-trained piRNN on a new changed reward function? I am curious as to whether a salience-remapping can be observed in such a setting, while preserving the same spatial encoding. (To be clear, I am not suggesting this experiment be run during the rebuttal phase, but curious to hear the authors thoughts on this).\n\n2. While the authors claim global distortion in the presence of rewards, in my understanding references 16,17,18 which experimentally observe firing rate distortions report them as local changes as opposed to global distortions How do the authors reconcile their findings with these experimental works?\n\n3. Is there a link to be made between the reward-based loss function and the diffeomorphism theory presented earlier? I.e. does this loss function encourage such a mapping?", "limitations": "The main limitation of this paper seem to be its disconnect with existing experimental work that observed reward-based firing rate distortions in grid cells. Since this is a paper attempting to explain a scientific phenomenon, it is important to address this. I am relatively torn as I think the theory itself is a novel contribution and one that deserves to see the light of the day, I would appreciate if the authors could answer my questions in the weakness and questions section.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. Firstly, the paper is well-written and very clear.\n2. The motivation is very strong: understand what grid cells encode is indeed an important scientific question with recent discoveries showing changes in grid cell firing patterns in the presence of environment cues.\n3. The theoretical framework describing the effect of rewards as a diffeomorphism and the consequent effect on the neural manifold is solid, and a strong conjecture re coding in grid cells.\n4. The addition to the loss function, and fine-tuning schemes that preserved the toroid topology are indeed unique contributions.", "weaknesses": "1. My main concern here is that while the theoretical ideas underlying this work make sense, in practice we do not see firing patterns that have been observed empirically. I understand that the diffeomorphism theory can in principle explain attractive firing patterns that have been experimentally observed, however, the path-integrating models trained by the authors assuming a circular reward function do not show such patterns. As a result, while intuitively the theory makes sense, it seems to me that it is crucial to address the disconnect in the observed results compared to experimental findings. I am curious to hear the authors thoughts on this. \n\n2. I also wish that the experiments were more thorough, for example investigating a range of salience functions potentially uncovering distinct firing rate patterns. Perhaps a different salience map would lead to attraction firing patterns?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your thorough review. We appreciate your attention and address your comments and questions below.\n\n**Main concern: link between theory, and firing patterns observed in piRNN experiments versus real experiments**\n\nThank you for raising the point that distortions of firing patterns in real rodent experiments differ from those observed in piRNN experiments. We think that this is an important point, and is actually one of our most interesting findings, which is answered below in response to your question #2. \n\n**On piRNN experiments**\n\nWe agree that more thorough experiments would be beneficial to this work. This concern was shared across several reviewers. Consequently, we have run additional experiments. These experiments include a wider range of saliency functions as per your suggestion. Please see our global answer where we present and discuss their results.\n\n**Answer to question 1. on continual learning**\n\nYou asked whether this scheme would be capable of training a salience-trained piRNN on a new changed reward function. This is a very interesting question! In the same way that we pretrained a network with uniform saliency (no reward) and then fine tuned the network with non-uniform saliency s(x), a good starting point would be to introduce additional phases of fine-tuning with different saliency maps, and analyze the resulting deformations after each phase. We expect that the neural responses will adapt by deforming in the ways we have described (developing a mixture of band units, diffused units, and ring units), while phases of training without rewards would see a convergence back to the typical hexagonal grid cells.\n\n**Answer to question 2. on global distortions**\n\nYou noted that previous works reporting experimental observations of firing rate distortions interpret them as local changes as opposed to global distortions. This is what we also expected to see a priori, given the claims from neuroscience experiments in Boccara et al. However, to our initial surprise, we did not observe this kind of local “magnification” or deformations in any of our experiments. Instead, we observed more global changes in the rate maps – which inspired the title of the work, “Global Distortions from Local Rewards”. Thus we challenge the conventional interpretation that reward modulation results in local magnification of the grid cell lattice as too simplistic an explanation. Indeed, because grid cells provide a global code for space, we propose that reward modulation should result in global deformations of spatial responses in MEC, as observed in our piRNN experiments.\n\n**Answer to question 3. on link between loss function and diffeomorphism theory**\n\nYou raise a thought-provoking question about how a grid cell deformation, modeled as a diffeomorphism of the 2D environment, depends on the chosen loss function. Dorrell & Latham et al., 2023 showed that hexagonal grid cell representations result from minimizing a particular loss function subject to biological and representational constraints. Their loss function encourages the separation of neural representations of different positions in space, and includes a term that weighs the importance of separating any pair of positions x and x’. This term is directly analogous to the saliency map we introduce in the loss of our piRNN. Because Dorrell & Latham et al. do not study grid cell deformations, we believe that the framework we introduce in our paper for studying grid cell deformations opens a new route to make progress on this question, complementing existing work in the literature. We note that this question is of utmost importance to neuroscience and AI: given some task and objective function, can we predict and explain the representations learned by neural systems?\n\n**Conclusion**\n\nWe thank you for your thorough review and specifically for prompting us to discuss the disconnect between the results from the piRNN experiments and the real experiments, as well as its link with our theoretical framework. We hope that we have addressed your concerns and we look forward to discussing these topics further.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jx6tIXSz3F", "reviewer": "Reviewer_BFFn", "summary": "This paper develops a model meant to explain recent experimental work showing that the orderly receptive fields of grid cells in the rodent brain can be distorted by spatial landmarks. Here, the model is a biologically-inspired ANN which is trained to path-integrate, with non-uniform weighting across the environment to induce distortion. The resulting distortions due to reward are then quantified, which offers some concrete hypotheses about the corresponding behavior expected from grid cells in the brain.", "questions": "* In Fig 3, why does the diffused firing field have different periodicity from the original?\n* Line 228 states that \"we observed the emergence of diffused and band units, as predicted by our theoretical framework\". However, I think Section 3 only examines the effect of diffused and attracted units on the manifold; as far as I can tell, it doesn't make any claims about when units with these tunings are expected to appear.\n* It makes sense to me that attracted units -- which would increase the resolution for highly salient regions of space (correct me if I'm wrong) -- are a relevant distortion. However, they don't seem to be found in Section 4! This is surprising to me.\n* The text of the introduction strongly suggests that reward will be incorporated into the loss function. However, the actual method simply allows for weighting the prediction error for different locations, which is more generic (this is an advantage, in my opinion) and should be better reflected in the introduction.\n* It may be helpful to quantify distortion with some other measures (e.g. distribution distance).", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 1, "strengths": "* Grid cell distortion is a strong empirical finding that calls into question some key assumptions in systems neuroscience, and developing explanations and models for it will be a major contribution to theoretical neuroscience.\n* The model the authors consider here is eminently simple and easy to explain, but captures the problem quite well.\n* The writing is polished, and the paper is clear and easy to read.", "weaknesses": "* The theory in Section 3 is not very deep. As I understand it, it basically lists a few plausible distortions (diffused, attracted, banded) and considers what impact the first two would have on the network. However, only diffused units are studied in the experiments in Section 4, even though banded units are also found, and an additional type (ring units) identified in the experiments is not accounted for. Overall, I feel that the theory doesn't have much explanation to offer of the empirical phenomena.\n* The experiments are not particularly in-depth. Some distortions are identified, but only diffusion is carefully examined. How particular changes to the loss correspond to particular distortions is not studied. This is a passable study of how the manifold geometry is warped (which could still benefit from more thorough experiments), but a suitable contribution for this venue would have to involve a more comprehensive analysis of distortion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read and review our paper. Your feedback was invaluable, as it highlighted that we had not properly emphasized several crucial aspects of our work. In particular, we had not sufficiently explained the generality of our theoretical framework and the fact that our experiments were made possible only when freezing the place cell read-out weights. We address your comments and questions in detail below.\n\n**Theoretical contributions**\n\nWe thank you for this important remark. Motivated by your feedback, we provide additional elements to our theory, as discussed in the global answer to all reviewers.\n\nWe now explicitly model the emergence of the four deformations: diffused units, band units, ring units, and attracted units. Interestingly, this only required a simple modification of the theory. We look forward to hearing your thoughts about it.\n\n**Experimental contributions**\n\nYou noted that we did not sufficiently study how particular changes in the loss function correspond to particular changes in the rate map distortions. Indeed, the earlier experiments were simple in the sense that differences in reward location, magnitude, or spread were not well explored. We now performed these experiments, please see our global response for details. \n\nAs you mentioned, the most relevant modification in our work is the differential weighing of the loss function to incorporate the reward location. Here, we wish to emphasize another of our unique contributions that was missing from your summary. When we first performed the saliency training, we observed that the topology of the neural manifold was not preserved (Fig. 7B). This was a surprising result, one we had not anticipated. Only after we froze the pre-trained output weights to the place cells, we were able to observe both the global distortions and the preservation of the topology (Fig. 7C). \n\nSo, when referring to saliency training, we do not mean a simple modification of the loss function, but also this non-trivial aspect of weight freezing. We believe setting up this framework to study the effects of local rewards is an important technical contribution of our work, which the future work can build on.\n\n**Signal periodicity**\n\nIn Fig 3, showing results on synthetic neural responses, the diffused units have the same spatial periodicity as the original units – as the diffused units were created by applying a Gaussian convolution to the original units. That is, the locations of the signal peaks are the same. \n\n**Wording on diffused and band units**\n\nThis is a very good point, thank you for bringing it up. As you correctly point out, our theory does not explain why some distortions appear and some others don’t. Instead, we provide a mathematical formulation of their distortions, and explain what is the effect of this distortion on the neural manifold. We apologize for the confusing wording, which we propose to correct as follows: \"we observed the emergence of diffused, band and ring units, which fall into the category of distortion via isotropic and anisotropic deformations from our theoretical framework\".\n\n**Attracted units**\n\nYou mentioned that the lack of attracted units in the trained piRNN was surprising. We also expected to see some local distortions of the grid cell firing patterns, given the standard interpretation of neuroscience experiments like Boccara et al. However, to our surprise, we did not observe this kind of local “magnification” or deformations in any of our experiments. Instead, we observed more global changes in the rate maps – which inspired the title of the work, “Global Distortions from Local Rewards”. \n\nWe believe this is a strength of our work, not a weakness. Our results challenge the conventional interpretation that reward modulation results in local magnification of the grid cell lattice, which may be a simplistic explanation. Indeed, in hindsight, the original findings might  have been inconsistent with the conventional wisdom that grid cells provide a global code for space. Thus, our results provide the first evidence that reward modulation may result in global deformations of spatial responses in MEC, and the local distortions may arise due to interactions with other experimental variables not controlled in Boccara et al., 2019, or a reasonable but incorrect interpretation of the data. \n\nIt may also be true that future changes to the saliency training may recapitulate the experimental findings. Our work is just the first step (which sets up the framework) to study this problem theoretically and in detail.\n\n**Rephrasing introduction.**\n\nWe agree with the suggestion of rephrasing the introduction so that it no longer suggests that “reward will be incorporated into the loss function”. Indeed, as you mention, the actual method is simpler. We plan to rephrase as follows: “we modify the loss function to differentially weigh position estimation error based on a saliency map of space.” Thank you for the suggestion!\n\n**Quantifying distortion**\n\nThis is an excellent suggestion! To address this, we performed statistical tests with structural similarity index and Wasserstein distances. Specifically, we computed these distances between images before and after distortion for each class of distortions, and now defined the distortion units as a result of this statistical analysis.\n\nOverall, we believe your feedback was very helpful for increasing our impact. To address your concerns, we have performed additional analyses (leading to new results, see our general response!) and expanded on our theory. If you believe we have adequately addressed your concerns, would you consider increasing your score to support the acceptance of our work?", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the four reviewers for their time and attention. For this rebuttal, we ran additional experiments to draw a more complete picture of the phenomena of firing patterns distortions, and expanded on our theory to address reviewers’ concerns. Along the way, we also found some new exciting results! 

**Additional experiments**

Following suggestions from all 4 reviewers, we performed additional experiments by varying:
1. the location of the reward [$x_*$ in Eq. 8] $x_*$in [(0.5,0.5), (0.8,0.8), (0.5,0.8)] 
2. the overall magnitude of the reward [$s_0$ in Eq. 8] in [1,10,100]
3. the spread of the reward [$\sigma_*$ in Eq. 8] in [0.05,0.1,0.2,0.5]

These experiments, coupled with a new statistical analysis (explained in Figs. N1-N3), confirmed the consistency of our finding that about one-third of the toroidal cells become distorted during the saliency training regardless of the hyperparameters. Moreover, we also realized that the distortions did not emerge in random populations of cells, rather in those that did not preferentially send read-out weights to the place field at the origin (Fig. N4). Notably, even when the saliency map was placed in a location other than the origin (reward location), the global distortion likelihood still decreased towards the origin (most visited location); providing an interesting contrast to the prior work that has observed local distortions towards reward locations.

To date, prior experimental work had not distinguished between the tendency of the animals to visit familiar locations from the reward valence. Thus, our synthetic training allowed us to disentangle these aspects and bring forth new biological predictions, i.e., that global (but not necessarily local) distortions to grid cells may be signs of continual learning towards secondary objectives. Future explicit tests of these ideas can be performed experimentally by introducing both reward and landmark locations to the path-integrating RNNs and/or freely behaving animals!

Overall, after saliency training across several hyperparameters, we observed that
- the toroidal topology of the neural manifolds is preserved (data not shown), 
- diffused, band, and ring neurons consistently emerged as the leading distortions (Figs N1-N3 for _the new quantification analysis_), and
- the distortions did not emerge in random populations of cells, rather those that were coding locations furthest away from the origin (Fig. N4, _new result_).

Finally, we want to note that we ran slightly more than 650 experimental configurations (saliency training), which took on average around 3 hours per experiment. Overall, we used 2400 GPU hours, not counting the additional analyses we performed on top of the saliency training. We will make all of these models and our analyses publicly available, which we believe is an important step forward for the field. We kindly ask that the reviewers consider this as a contribution!

**Link between theory, results on piRNN experiments versus results on real experiments**

In addition to our experimental efforts, we have also performed additional theoretical investigations and expanded on our geometrical framework (to address primarily the concerns by Reviewers STz6 and BFFn). Notably, we complemented our current theory with a framework of global distortions based on diffusion processes. This represents a simple modification, and mathematical refinement, of section 3.2 “Diffused Units: Smoothing of Firing Fields”. Yet, and despite its simplicity, it now characterizes the three phenomena: diffused units, band units, and ring units. Below, we provide the intuition behind the new framework that can model ring, band, and diffused deformations of the grid cell lattice:

- *Diffused units*: In this work, we show that diffused units, observed in practice in our piRNN experiments, lead to smaller neural manifolds in neural state space and leave the energy budget unchanged. Diffused units can be explained by 2D isotropic smoothing of the original grid lattice.
- *Band units*: These units emerge as anisotropic smoothing of the firing fields, which can be useful for example to incorporate direction information. We show how band units, if emerge in large fractions, can lead to a collapse of the toroidal manifold into a circle. In our experiments, we observe only a small fraction of these, and thus the toroidal manifold structure is preserved.
- *Ring units*: These units emerge after angular smoothing of firing fields, which can be useful for example to encode angle-invariant distance information. Similar to before, the energy budget is unchanged. Notably, we observed ring units even when the saliency map was not at the origin, hinting that these units may be more relevant with coding frequently visited space information (here origin), which is also angle-independent.

**Conclusion**

To conclude, our theory does not explain which firing patterns should or should not emerge (which we now explicitly discuss in the paper). Instead, we introduce a rich mathematical framework that (1) quantifies all four firing field deformations, observed across piRNN and real experiments, with group actions, and (2) relate the firing field deformations with a neural manifold deformations. However, we recognize that one part of the theory (group action by diffeomorphisms) explains the results observed in real experimental data, while another part (group action by diffusion) explains the results observed in our piRNN experiments. Accordingly, results from real and piRNN experiments do not agree: the former finds locally attracted units, while the latter finds global distortions.

While we were shocked by this disconnect at first, we now believe that it represents one of our most interesting findings in this work. All in all, we believe that truly uncovering these mysteries will require a collaborative effort. Since all our datasets will be made public, we are also looking forward to the inputs from the full community!

Meta-review: This paper uses recurrent neural networks trained on path integration (piRNNs) to model and understand the effect of local rewards on spatial representation in the brain, which is a well-established question in neuroscience and has been explored in empirical animal studies. As noted by all the reviewers, the theoretical model is both novel and intriguing, and attempts to provide a framework to explain previous empirical observations. The reviewers unanimously agree that the paper is appropriate for publication at NeurIPS.

One issue raised by the reviewers is that certain aspects of the model’s predictions appear to contradict some existing empirical findings in neuroscience. However, this could be seen as an interesting interaction between theory and experiment, which could be further explored in future work. I concur with the reviewers that the novelty of the theoretical contribution justifies the paper's acceptance at NeurIPS.
Decision: Accept (poster)