Explaining heterogeneity in medial entorhinal cortex with task-driven neural networks
Authors: Aran Nayebi
Alexander Attinger
Malcolm G. Campbell
Kiah Hardcastle
Isabel I. C. Low
Caitlin S. Mallory
Gabriel Carreira Mel
Ben Sorscher
Alex H Williams
Surya Ganguli
Lisa Giocomo
Daniel LK Yamins
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: neural coding
medial entorhinal cortex
grid cells
biologically-inspired navigation
path integration
recurrent neural networks

Abstract: Medial entorhinal cortex (MEC) supports a wide range of navigational and memory related behaviors.
Well-known experimental results have revealed specialized cell types in MEC --- e.g. grid, border, and head-direction cells --- whose highly stereotypical response profiles are suggestive of the role they might play in supporting MEC functionality. 
However, the majority of MEC neurons do not exhibit stereotypical firing patterns.
How should the response profiles of these more "heterogeneous" cells be described, and how do they contribute to behavior?
In this work, we took a computational approach to addressing these questions.
We first performed a statistical analysis that shows that heterogeneous MEC cells are just as reliable in their response patterns as the more stereotypical cell types, suggesting that they have a coherent functional role.
Next, we evaluated a spectrum of candidate models in terms of their ability to describe the response profiles of both stereotypical and heterogeneous MEC cells.
We found that recently developed task-optimized neural network models are substantially better than traditional grid cell-centric models at matching most MEC neuronal response profiles --- including those of grid cells themselves --- despite not being explicitly trained for this purpose.
Specific choices of network architecture (such as gated nonlinearities and an explicit intermediate place cell representation) have an important effect on the ability of the model to generalize to novel scenarios, with the best of these models closely approaching the noise ceiling of the data itself.
We then performed in silico experiments on this model to address questions involving the relative functional relevance of various cell types, finding that heterogeneous cells are likely to be just as involved in downstream functional outcomes (such as path integration) as grid and border cells.
Finally, inspired by recent data showing that, going beyond their spatial response selectivity, MEC cells are also responsive to non-spatial rewards, we introduce a new MEC model that performs reward-modulated path integration.
We find that this unified model matches neural recordings across all variable-reward conditions.
Taken together, our results point toward a conceptually principled goal-driven modeling approach for moving future experimental and computational efforts beyond overly-simplistic single-cell stereotypes.

Explaining heterogeneity in medial entorhinal cortex with task-driven neural networks
Aran Nayebi'." Alexander Attinger", Malcolm G. Campbell', Kiah Hardcastle", Isabel LCC. L27 Caitlin S. Mallory', Gabriel C. Mel', Ben Sorscher", Alex H. Williams Surya Ganguli Lisa M. Giocomo 27 and Daniel L.K. Yamins 3,5,7
Neurosciences Ph.D. Program, Stanford University DDepartment of Neurobiology, Stanford University Department Computer Science, Stanford University Department of Applied Physics, Stanford University 'Department of Psychology. Stanford University
"Department Statistics, Stanford University
7 Wu Tsai Neurosciences Institute, Stanford University "Facebook AI Research, Facebook, Inc. CCorrespondence: anayebi @stanf waanoodd.eee edu
Abstract
Medial entorhinal cortex (MEC) supports wide range of navigational and memory related behaviors. Well-known experimental results have revealed specialized cell types in MEC - e.g. grid, border, and head-direction cells - whose highly stereotypical response profiles are suggestive of the role they might play in sup- porting .EC functionality. However, the majority of MEC neurons do not exhibit stereotypical firing patterns. How should the response profiles of these more "het- erogeneous" cells be described, and how do they contribute to behavior? In this work, we took computational approach to addressing these questions. We first performed a statistical analysis that shows that heterogeneous MEC cells are just as reliable in their response patterns as the more stereotypical cell types, suggest- ing that they have coherent functional role. Next, we evaluated spectrum of candidate models terms of their ability to describe the response profiles of both stereotypical and heterogeneous MEC cells. We found that recently developed task-optimized neural network models are substantially better than traditional grid cell-centric models at matching most MEC neuronal response profiles including those of grid cells themselves - despite not being explicitly trained for this pur- pose. Specific choices of network architecture (such as gated nonlinearities and an explicit intermediate place cell representation) have an important effect on the ability of the model to generalize novel scenarios, with the best of these models closely approaching the noise ceiling of the data itself. We then performed in silico experiments this model to address questions involving the relative functional relevance of various cell types, finding that heterogeneous cells are likely to be just as involved in downstream functional outcomes (such path integration) grid and border cells. Finally, inspired by recent data showing that, going beyond their spatial response selectivity, MEC cells are also responsive non-spatial rewards, we introduce a new MEC model that performs htwaardmodulated path integration. We find that this unified model matches neural recordings across all variable-reward conditions. Taken together, our results point toward conceptually principled goal-driven modeling approach for moving future experimental and computational efforts beyond overly-simllistic single-cell stereotypes.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Introduction
From exploring new areas, planning shortcuts, and returning to remembered locations, the ability to self-localize within an environment subserves a range of navigational behaviors that are essential for survival. The hippocampus (HPC) and medial entorhinal cortex (MEC) are known to contain cells that encode the position of an animal by displaying firing fields with strikingly regular response patterns, influenced by self-motion [O'Keefe and Dostrovsky 1971 Hafting et al. 2005 Kropfi et al. 2013 Solstad et al. 2008 Sargolini et al. |006]. For example, MEC grid cells possess characteristically symmetric and periodie tuning curves, yielding hexagonal arrays of neural activity over physical space. Additionally, border cells that fire maximally near environmental boundaries and head direction cells that fire only when an animal faces a particular direction are among the other MEC cell types with interpretable tuning curves.
However, a large fraction of the MEC population have unconventional and heterogeneous tuning to navigational variables and are less obviously well-described in terms of simple, stereotypical tuning patterns [Hinman et al. 2016 Hardcastle et al., 2017]. How can we characterize what these other, more heterogeneous, populations of cells do? Are they critical to the abilities of the animal in real-world memory tasks? And so, how is their role distinct from the more stereotypical grid-like cells? Here, we take a quantitative computational modeling approach to answering these questions.
To start with, we address the question of whether there is a phenomenon to model in the first place. and how one would measure model accuracy quantitatively. To this end, we identify the similarity transform between neural populations different animals - that is, mapping which takes neuronal population vectors in "source" animal and maps it to corresponding neuronal population vectors in a "target" animal. To the extent that robust similarity transforms can be found that match up MEC population responses across multiple animals measured in multiple experimental conditions, then a reliable pattern of neural response behaviors (across conditions) has been identified. This identification strategy is well-defined even when there is no known priori taxonomy of functional response types. We find here that with the right mapping class (mostly Ridge-like linear regression), the inter-animal consistency of MEC neuronal responses is in absolute terms very high (>> 0.8), and in relative terms just as high as for heterogeneous cells as for more stereotypical grid cells.
Using this same similarity transform, we then evaluate the ability of each of multiple computational models to explain response variance of MEC neurons, treating each candidate model potential "source animal" and measuring how well it maps to each target real animal. We look to the recent literature to identify potential candidate models. Over the past several decades, collaborations between experimental and computational neuroscientists have led to the formulation of dynamical- systems models of grid cell formation, and helped illustrate possible functional roles for grid cells in supporting path integration-based hippocampal place cells (Skaggs et al. 1992 Zhang 1996] Fuhs and Touretzky 2006 Burak and Fiete| 2009). These powerful models make a number of non- obvious predictions about MEC neural properties, some of which have been confirmed in subsequent experimental work [Ocko et al., 2018 Campbell et al. 2018). Despite their success, such models are limited in their explanatory scope, hand-designed to capture the properties of one stereotypical cell-type class (e.g. grid or border cells) at a time, or combinations of several cell types via multiple dedicated type-specific modules [Couey et al. 2013 Yoon al., 2013).
potential solution to this problem arises out of recent work creating learned neural networks that achieve path integration [Cueva and Wei 2018 Banino et al. 2018). Intriguingly, these models have been found to contain internal units that resemble grid cells, suggesting that such stereotypical cells embody computational solution to path integration that naturally arises from satisfying an end-to- end functional constraint. Recent work has demonstrated that the underlying mathematical reason for this fact is due to pattern forming dynamics under nonnegativity constraint [Sorscher et al. 2019 2020). Intriguingly, in addition to having units that resemble stereotypical grid or border cells, these learned neural networks also naturally possess wide variety of other less easily described unit types, raising the possibility that these artificial "heterogeneous" units might be somehow resemble the actual heterogeneous cells making up the majority of real MEC populations.
Motivated by these ideas, we generate a wide variety of candidate model networks by varying archi- tectural structure and end-to-end optimization objectives, each expressing a different hypothesis for MEC eircuit structure and function. Architecturally we formulate variants based on using different types of nonlinearities and different types of local recurrent memory circuits (e.g. RNNs (Elman

---

[
UGRNNs [Collins et al. 2017]. GRUs (Cho et al. 2014], and LSTMs [Hochrettrr and Schmid huber 1997). From a task point of view, we test both simple dimensionality reduction [Stachenfeld et al.
2014 Dordek et al. 2016) as well as place cell mediated vector path-integration Banino et al.
2018
and direct position estimation [Cueva and Wei 2018].
Our core result is that there is substantial variation in the models' abilities to explain MEC responses, especially those of the heterogeneous cells, as a function of model architecture and task. Some models, such as the classic "Grid Cell" model based on low-rank decomposition of place cell fields, do a reasonable job explaining grid cell responses but are quite poor at explaining most other neurons. Task oottmiopt learned models typically do better, especially those optimized for place vector -nt-innterratio. The best model with memory -gatd rectified nonlinearities essentially solves the neurons, capturing nearly 100% of the noise ceiling of the data. This is of substantial interest, given that the nonlinear components of this model are not directly optimized to match neural data (just to solve the task), and given that a series of strong control models capture much less of the MEC neural variability. This same model also best generalizes to a variety of novel experimental conditions, and has the best match to the empirical data on grid score distribution metric.
With this predictive model in hand, we then begin to address our second core question: what the functional role of heterogeneous neurons? We generate several results suggesting that heterogeneous neurons are important for path integration, including cell-type specific virtual knockout experiments. in which we compare performance degradation when deleting grid and/or border cells as compared to heterogeneous units. Overall, we find that models are quite robust knockouts, and differences between heterogeneous and stereotypical cell knockouts are very small, suggesting that stereotypical cell-types may not be especially more functionally important than units with less easily-characterized response profiles.
Building on the above results, we extend models to encompass MEC cell responses as a function of reward as well as spatial position, introducing a simple modeling paradigm that performs reward- modulated foraging in the context of the path-integration task. We find that this unified task-optimized model matches neural responses across all reward and spatial conditions, and that a reward-response mechanism at an intermediate point on the exploi- continuum best explains neural responses. Taken together, our results suggest how specific processes of biological performance optimization may have directly shaped the neural mechanisms in MEC as a whole, and provides a path for enlarging the study of MEC beyond veeriy-estricttvve response stereotypes.
2 Reliability of heterogeneous cell response profiles
What firing patterns of MEC cell populations are common across multiple animals, and thus worthy of computational explanation? This question is comparatively straightforward for stereotypical MEC cells. because the very presence of these stereotypical features (c.g. hexagonal grids of given orientation and spatial frequency) allows the definition and measurement of observables that arise reliably across trials and animals (c.g. the grid score distribution). But how can this be done generally for populations in which one does not have a prior characterization of what each cell encodes? We take inspiration from methods that have proven useful in modeling visual, auditory, and motor cortex Yamins and DiCarlo 2016 Kell et a 2018 Michaels et al., 2020]. Specifically, we aim to identify the narrowest class of similarity transforms needed to map the firing patterns of one animal's MEC population to that of another (Fig. [b). As with other cortical areas, this transform class likely cannot be so strict as to require fixed ceuron-to-neurg mappings between MEC cells, since even within the same animal at different times, MEC and HPC populations can undergo remapping that shift cell responses across the population [Faroood et al. 2020 Low et al. 2020). However, the transform class for MEC also cannot be loose as to allow a completely unconstrained linear mapping, since the highly structured response patterns of stereotypical MEC cell types such as grid cells may not be guaranteed to be preserved under arbitrary linear transforms.
To identify this transform class, we utilize data collected from electrophysiology in 12 awake behaving mice (n = 620 cells) performing open foraging for randomly scattered crushed cereal in 100cm 2D arena [Mallory et al., 2021). J. We explore a variety of mapping transform classes between the population rate maps of these neurons (Fig. [b). As a baseline we evaluate strict one-to-one mapping transform, which each target unit mapped to the single most correlated unit in the source animal. We also evaluate more powerful linear transforms, including Lasso, Ridge, and ElasticNet regression,
3

---

Inter asimal Consistency
(Meillan acroc unins)
Ridge CV d (1ag
Fap.
Kalgs & 120
IN sompa IES
Lasso CV O Ray Soel
LP
Janenm Conamnyy
Gra Cals
Blentgeen
Cell
Tlnengeenco
Gra
Grod
Coad Sootd
Figure 1: Heterogeneous cells are just as reliable as grid cells across 2D environments and task conditions. (A) We establish inter-animal consistency levels by mapping animals to each, using inter-amimal transforms L of Al-most-lineal functional complexity. Computational models are then mapped to real animal's MEC data using the same transform class. (B) Inter-animal consistency levels assessed with mapping transform classes different complexity. Median and s.c.m. across 620 cells. (C) The alpha value per neuron as chosen by Ridge CV plotted against the alpha value chosen by Lasso CV, on a log scale, Median and s.c.m. across ten train-test splits. The unity line is in yellow. (D) Per unit sparsity penalty under ElasticNet CV (grey bar in (B)) function of grid score. Median and S.c.m. across ten train-tess teittteetteee splits. (E) The fraction of souree units assigned nonzero weight per target cell plotted against its grid score (median and s.e.m. across train-test splits). Red denotes grid cells with grid score > 0.3 and blue denotes heterogenenus cells with grid score S 0.3. The har plot on the right is the mean and s.c.m. of this median quantity for the identified grid and heterogencous cell populations. The p-value is obtained from an independent (-test across neurons between populations. (F) Inter-animal consistency of each unit plotted against grid score. The bar plot the right is the mean and s.e.m. this quantity for identified grid and heterogeneous cell populations. The p-value obtained from an independent t-test across neurons between populations.
as well as population-Ievel Partial Least Squares (PLS) regression. For all methods with fittable parameters, mapping fit is performed on a random 20% of spatial position bins, and evaluated on the remaining 80% of position bins (see supplement for more details).
The strict one-to-one mapping yielded low inter-animal consistency among the maps considered, capturing around 50% of the target neural response variability. Linear regression with strong sparseness priors, such as Lasso (L1 penalty) and balanced Lasso-Ridge (equal L1 and L2 penalty) regression, also proved be strict when evaluated with fixed regularization level hardly any improvement over the one-to-one mapping. However, pure Ridge (1.2 penalty) regression at this regularization level was highly effective, recovering nearly all target variability for most neurons.
1, yielding
the LI regularization constant on per-cell basis improved the fits, at the cost of requiring substantially looser regularization than for L2 (Fig (.). Under an ElasticNet mapping in which both sparsity penalty and regularization constants were chosen with cross validation, most cells were generally still best explained by an essentially Ridge-like transform with no sparsity penalty (Fig. It However, different target cells required different numbers of source units to achieve effective mapping (Fig. [p). As expected, cells with more stereotypically grid-liks response patterns on average chose higher sparsity penalty (slight positive slope in Fig. I and required fewer source cells to capture (slight negative slope in Fig. [T) as compared to heterogeneous cells, though this effect is weak. Critically, heterogeneous cells did not have lower inter-animal consistency than grid cells (Fig. m.).
These results show that the heterogeneous non-stereotyped cell populations are reasonably similar across animals least up to (mostly Ridge-regularized) linear transform establishing that there is a reliable target pattern to study in the first place.
3 Task-optimized models of MEC spatial response variation
Evaluating a spectrum of candidate models. We evaluated models of several basic types. First, we considered learnable neural networks, of which accept a stream of two-dimensional velocity input vectors, have single layer of hidden neurons identified the putative MEC population, and

---

Nord Phalienity Comiltee
BDON
3008
DNN
RXS
S0000
600
6B5
G30
600
153M
LSMH
ESTM GonNN
GoloN DORNN DORNS
54c.
Signaal
Tecacde 23 Arata
Norkda tniteel
Nora Bodiesiity
N007
DNN
INN
TNNN
RNC
LENN
SENDS
MINN
CRC
GRC
LTTH
ESTM
LETH
Elnaar
Think
Tub.
BUELI
Sproil
Tex.
Sigwal
BLEE!
Rigronal
Figure 2: Task-optimized navigational models best predict the entire MEC population. (A) Neural predictivity of the model MEC units to the real MEC responses of 12 animals in 100em' 2D open field under the ElasticNet CV regression transform (grey in [b). Median and s.c.m. across 620 total units. (B) Same as (A), but now the models are evaluated against responses of MEC units while the animal is traversing 400cma ID track. "ID Track" refers to models trained to path integrate on this same ID track. "2D Arena" refers to models trained to path integrate on the 2.2m² arena and evaluated on the ID track. Median and s.e.m. across 2861 units from different animals.
which are optimized perform some form of path integration readout on simulated motion paths (Erdem and Hasselmo 2012]) in a fixed-sized arena. The networks varied according to the nature of their local recurrent cell structure (ungated RNN, UGRNN, GRU, or LSTM), activation functions (Linear, Tanh, Sigmoid, or ReLU), and output objective function (explicitly constructing set of simulated place field neurons [Banino et al.. 2018 or directly performing two-dimensional position integration [Cueva and Wei 2018). Following Banino et al. [2018). all networks consisted of three nonlinear layers, though for the ungated RNN we included its original two layer version (following Cueva and Weil (2018) and a three layer version ("SRNN"). We ensured that comparisons were fair by equalizing the size of the hidden layer across models, and included an architecture-only control with untrained filter weights.
We also implemented class of models describing MEC activations as Non-negative Matrix Factor- ization (NMF) operating on simulated place cells. The lowest-rank version acts a "Grid Cell" model, inspired by recent work positing MEC low-dimensional embedding of hippocampal place fields 2016), and further validated by the mathematical theory to explain the emergence of grid cells. Higher-rank NMF models titrate between the low-rank Grid Cell model and a full-rank control that measures how well MEC cells can be explained as linear projection of their putative place cell outputs.
(Stachenfeld et al., 2014 Dordek et al.
developed in Sorscher et al. 2019 2020
For evaluation, we map each model's proposed MEC activations to empirically measured firing patterns of the real neurons, using the same type of cross-validated ElasticNet regression transform used to measure inter-animal similarity as in Section 2 The predicted neural responses under the mapping are then compared real target neural responses on a neuron-by-neure basis, and the median of accuracy of these predictions taken over target neurons. In addition to evaluating model- data match when constructed on arenas of the same size in which the neural data were collected (100cm²), we also evaluated models trained on larger arenas (2.2m²) but tested on the 100cm² arena.
Neural prediction results. The results of our model evaluation, shown in Fig. support several inferences:
Different models are substantially different in their ability to predict neural responses. MEC electrophysiology data collected during 2D open-field foraging thus strong model target that effectively separates candidate models from each other.
5

---

Task-optimized models are reliably better than their untrained controls, across all architecture types and objective functions.
Rectification is substantially better than Linear, Tanh, or Sigmoid activation, especially for promoting generalization to new arena sizes.
Under the rectification nonlinearity, gated circuit architectures (UGRNN, GRU, LSTM) improve model fits compared to the simple ungated alternatives (RNN and SRNN).
The explicit place cell construction task leads to substantially better fits than the direct two- dimensional path integration task ("Position Loss"), even though both were reported in the literature to create grid cell-like units. Enforcing a place field representation at the output of the network thus appears to be an important constraint in order to recapitulate responses in MEC.
The Grid Cell (low-rank NMF) model is a poor fit to the MEC population overall.
Full-rank NMF and "Place Cell Linear MEC" output-based controls capture approximately half the explainable variance of MEC neurons, significantly more than the simple velocity linear input control, but substantially less than any of the input-driven ReLU networks.
The best model is the UGRNN ReLU trained with the place cell loss ("UGRNN-ReL.U P-ccaee Cell"). This model captures nearly 100% of the explainable variance of the MEC neural population response when trained for place cell construction on the arenas of the same size as that on which neural data was collected.
Generalization to novel experimental conditions. If a model is truly correct, then once trained, it should capture neural responses in any new tested condition. We tested generalization in two ways. First, as shown in the second from the right bars of Fig. 2 for each model architecture class, we performed a 2D arena size generalization test by constructing models on one arena size, testing against neural data on another. We found this generalization test gives essentially the same rank-order comparison as in the same-arena-size test, with one striking exception: the RNN Tanh model performs well within arena size, but fails to generalize. The BERNN-RReL-Plaae Cell model again performs the best, capturing 90% explained variance of the neural responses on the novel arena size.
Second, we also evaluated the same 2D-pretrained models by running them on a ID track, comparing models to neural data collected from mice in ID virtual reality setup, using the same ElasticNet mapping procedure as in the 2D comparisons (see supplement for more experimental details). We found (Fig. 2b) that 2D arena trained model results had some similar rank order for the original 2D results (0.43 Spearman rank correlation), with the -ERNNN-RLUU-Pace Cell model trained in the 2D arena (and then evaluated on the 1D track) achieving the best match (82%) when trained for place-field construction. We hypothesized that the larger gap between the best model and the inter-animal noise ceiling, as compared to the 2D case, was due to the fact that during the ID experimental data collection, mice were also presented with visual cues. MEC neurons are known to respond to visual input, but as none of our evaluated models had a visual front-end, they could not respond accordingly. To test this hypothesis we trained the UGRNN-ReLLU-PPE but with phantom visual cue locations added as input while performing the path integration task. We then evaluated this model against ID neural prediction (with the real visual cue locations as input), and found that including cues rescued model performance back essentially the noise ceiling, as in 2D (Fig. 2p, orange bars).
Cell model in 2D,
Assessing grid and border score distribution match. The grid score is metric of how stereo- typically grid-like the response pattern of a given unit is (see supplementary material for specific definitions), with "grid cell" typically defined as having a grid score of greater than 0.3. Similarly, the border score is a measure of how responsive a given unit is in the presence of environmental boundaries, with "border cell"' typically defined as having border score greater than 0.5 [Solstad] et al. 2008). The ground-truth distribution of grid and border scores for cells in an unbiased exper- imental population (gray bars in each subpanel of Fig. (.)) characterize the extent which real MEC populations are non-stereotypica in their responses. To further assess model accuracy, we also compared the distribution of grid and border scores within each model to that of the real data, using the (negative) Kolmogorov-S -minnov (KS) distance of the empirical and model grid score distributions as a quantitative metric. This metric is both stronger and weaker than the mapping accuracy metric used above stronger in the sense that, since there is no parameter fitting in the metric, to fit it well a model has to have the correct distribution in its raw feature output; and weaker that it only assesses cells on one component of their profile (i.e. "grid-ness" or "border-ness"). We observed that the low-rank Grid Cell model has poor fit on this metric, essentially because it contains too many grid cells (see Fig. 3h. upper left). In contrast, the same model that achieves best fits on the neural-fit

---

B666
Uig Count
0.00000
2000
Lass)
X5- 00800
m
BOL.
ION ANA. SANI MARL MEAN FADAL MAOC ON CHTA
CITTA LATA CAACEORRIOO
Grdl Gell Model BIAR,
VGRNN
Utg Crant
DGGNN
LGBNN RALU (Ummine))) Ri- CLUSR POCOIGG
(Ptace Gall Long
KF-1La7 2-0C800)
Figure 3: Relationships to grid and border score distribution. (A,C) Example distributions of grid and border scores from selected models against the ground truth distribution in the neural data (grey). Negative Kolmogorov-Sming
distance between the distribution of grid scores of the model units to those of the 620 units in the data. The models have all been trained in a different 2.2m² arena, but are evaluated on 100cm² 2 environment that the animals are in. (B, D) Quantification across all models.
metric -UGRNN-ReLU-IN ReLU-Place Cell) also achieves the best on the grid cell distribution match metric. and is the only architecture that cannot be distinguished from the ground truth at the KS < 0.05 sig- nificance level. Across model architectures, KS KS-distance was generally in line with neural regression fit metrics (see Fig. [d and Fig. S2) One key difference between the metrics, however, is that for most architectures, the untrained filters provided better grid score distribution matches as models with task-trained parameters except for the that best matched architecture, UGRNN ReLU, where the trained place cell construction model better than its untrained counterpart (and similar to it for the border score) and models trained for direct path integration were especially poor this metric. These results suggest the two metrics in Figs. 2 and 3 are complementary which aspects of model correctness they address, and together help zero in on the most effective models overall.
4 Predicting the functional relevance of heterogeneous cells
Correlating task performance and neural predictivity. To begin to address the question of the functional relevance of heterogeneous cells, we first looked at the overall correlation between model task performance and neural fit (Fig. 4a). Though the correlation is imperfect, the most task performant models (e.g. UGRNN- and --TM-ReLLU-PAa
Cell) achieve the best matches to neural fit suggesting that improved task performance may be causally related to the ability capture neuron response patterns across the population. In contrast, there is a comparatively weaker relationship between grid and border score distribution match and model performance (Fig. Ap) While the most task performant models achieve the best match here, models with a gating architecture and ReL.U are strong matches to these metrics with untrained filters, illustrating the importance in matching cell properties other than ggrid-ness" or "border-ness in predicting task performance.
Relative predictivity gain for heterogeneous cells. The above result is put into greater perspective by comparing neural predictivity differential between a task-traine JGRNN-ReIL ReLU-Place Cell model and the Grid Cell model, on per-neuron basis, as function of grid score (Fig F The task-trained model has improved neural predictivity relative to the Grid Cell model for grid cells (grid score > 0.3) and heterogeneous cells, but the improvements are larger than for grid cells. This again suggests that the heterogeneous cells are playing a substantial role in allowing the trained model to achieve improved performance.
the latter (as well border cells, Fig. SS3)
Virtual knockout experiment. To address the question most directly, we performed a cell-type- targeted virtual knockout comparison experiment. In doing this, we used the SHERNN-ReLU-PiN Cell and SSTM-ReLU-Plae Cell models, the two best models emerging from the previous section with essentially similar neural predictivity across multiple metrics. We identified the units in the trained model with high grid score (> 0.3) or border score (> 0.5), and gradually increased the threshold, while measuring task generalization performance (see supplement for details of this
7

---

(A)
(B)
News Pureer
All,
83
Dut Emagade Perfermacc in Ofggnt
(D)
Performance Dage slattinn
UGRNN RelL.
LSTM ReLU
UGRNN ReI.U
LSTM ReLU
050
0.25
2,50
Rosk
0.30
Keni Call
Galls
200
Keediz
1.5%
40.%
0.15
1.3%
100
03
033
1.20
-15
0.5
(65
10:0
000 0.00
Grid Score Threshold
Border Score Threshold
Figure 4: Heterogeneous cells are relevant to navigation. (A) Neural predictivity (median and s.c.m. across 620 units) versus path integration performance (mean and s.c.m. across 20,000 episodes), measured by negative positional decoding error, of the neural network models. The models are either untrained ("X"") or trained with the place cell loss ("O"") in the 2.2m² arena. Positional decoding error measured by taking the top most active place cell outputs each model, evaluated on the 100cm arena. (B) Same as (A). but with grid score (Left) and border score (Right) distribution instead of neural predictivity. (C) (Left) Per unit neural predictivity difference between the UGRNN-Rel -e-N----- Cell and Grid Cell models trained on the 2.2m² arena and evaluated on the 100cm² arena, plotted against that unit's grid score. (Righr) Quantification of this difference aggregated across the grid cell and heterogeneous cell populations, respectively (mean and s.e.m). The p-value obtained from an independent f-test across neurons between populations. (D) For the UGRNN-ReLU-PIN Cell and L.TT--ReL.--Plae Cell models, we measure the normalized performance degradation as evaluated on the 100cm² arena, relative to the full model trained on the 2.2m² arena with the place cell loss. We identify units in this trained model with grid and border scores of varying thresholds and knockout the same number of heterogeneous cells (randomly sampled 100 times). to yield the "heterogeneous cell knockout" (blue). The r-axis denotes the threshold used for the score. Mean and s.e.m. over evaluation episodes. denotes p-value < 0.01 obtained from an independent f-test hetween the performance degradations of the two knockouts at a given grid or horder score threshold.
knockout procedure). We similarly ablated matched numbers of heterogeneous units (grid score < 0.3 or border score < 0.5 or both, see Fig. S4 for the latter), again measuring model performance.
The main result of this experiment (Fig. 4+) is that all networks are highly robust to knockouts, experiencing only at most 1-4% performance degradation relative to the full model even when substantial fractions of units are knocked out (corresponding to 21-24% of single layer's units for grid score > 0.3 and 16-20% of these units for border score > 0.5). At stricter grid and border cell thresholds, the heterogeneous knockout is similarly injurious to the cell-type specific knockouts, across the two model architectures, suggesting that steehl-sterreotypccll "classical" border and grid score cells are not more essential to the path integration behavior than heterogeneous cells. At low thresholds (when counting many relatively heterogeneous cells as "grid" or "border" cells), the two model architectures give divergent predictions, with the UGRNN model showing small but significant effect of grid cells relative heterogeneous cells, and the LSTM model showing the opposite. It would be of substantial future interest confirm reject either of these models' predictions with a real targeted knockout experiment in vivo.
5 Modeling reward-driven modulation in MEC
Recent work in both rodents and humans has uncovered that MEC and HPC neurons represent not only literal space, but also capture spatialized layouts in more abstract sense in modalities other than spatial position (Constantinescu et al. 2016 Aronov et al.) 2017]. It has also been seen [Butler al. 2019 Boccara et al.] that non-spatial rewards can influence the shape of MEC response profiles, restructuring them to incorporate the location of the learned reward. These intriguing phenomena represent natural direction for modeling, but are not captured by any of the neural network models described in Section 3 as they do not have soward-modulated inputs. (Note that while Banino et al. (2088) used a pretrained L-TT-TTahh- Place Cell network as a front-end on which reinforcement-bane based navigation tasks are evaluated downstream, reward state is not input to their network or otherwise propagated back into the MEC-like layers of their model, and thus cannot address the neural modeling question raised here.)
8

---

(A)
(B)m
(D)
Vnaragg Lina
Bocad (mm
fhow
(Esploc)
(ispler
Monred
Rocord 47
Corde din Our (Sonariy
(Faybo4)
(Faplcnd
Figure 5: Reward biased path integration captures remapping of responses in the presence of reward. (A) Schematic of the three conditions. "Explore" is the original set of random bout trajectories used to train the agent as before. "Exploit" involves the agent navigating directly to the reward zone (red box) within a fixed number of timesteps per episode (7), and spending the remaining 13 timesteps only path integrating in the reward zone. "Reward Biased" refers to training with each of the above two trajectories on 0 < & < 1 of the training episodes. (B) Neural predictivity of the UGRNN- ReLU--Pace Cell model, evaluated on the 150cm arena. Median and s.c.m. of 598 cells from 7 rats. Green bars denote the model evaluated with random walk velocity inputs. * denotes comparison of the model each condition separately. Coral bars (to the right of the vertical line) denote the model evaluated with random walk and reward biased velocity input modulation. (C) Neural predictivity of the reward biased UGRNN-Rell Re----LUUUUPPP Place Cell model a function of e. Median and s.e.m. of 598 cells from 7 rats. The p-value obtained from an independent (-test across neurons between selected model pairs brackets. (D) Histogram the correlation coefficient unit rate maps between the random foraging and velocity-inpun modulated reward condition in the E = 0.375 and E models. The same metric applied to the data is shown in grey.
CCGRNN-ReLU-PR Cell
We thus sought to build new network models that respond to the existence of an extrinsic reward in wwwiiiorall-mmmannnhh fashion. One natural approach would be to build full reinforcement- learning (RL) agent in which reward depends on environmental features (e.g.total number of food units encountered during forage path). A model optimized end-to-end to forage under these conditions could be successful in creating a representation that matches MEC reward-response modulation. However, we took a more direct approach: simply change the behavior of the agent in response to reward in the way RL would be expected to do if successful, by modifying the foraging trajectories used during training and testing no longer be pure random walks.
Specifically, we trained the UGRNN-ReLL ReLU-Place Cell model with a variety of foraging policies titrating between pure exploration and exploitation. Paths could either purely exploit known reward resource by directly moving to the reward location when is present (Fig. 5h, E = 0 condition), purely explore with random walk in the original unmodulated model (s 1), or take an intermediate policy (0 < e < 1). In creating these scenarios, we sought to roughly mimic experimental observations showing animals often take rapid and direct paths to the reward zone [Butler et al.) 2019] (see supplement for details of implementation). Throughout, the actual training task of the network remained the same as above place cell construction but the network now would have to be tolerant to reward-modulated input velocity changes while maintaining positional knowledge.
We then compared these networks to neural data from [Butler et al. 2019] for animals in conditions both with reward (reward+) and without it (reward-), in which the animal navigates to 20cm² reward zone within a 150cm2 arena to receive 0.5-1 units of cereal. We first performed inter-animal consistency checks, using the same method we did with the purely spatial-condition data in Section above, finding that inter-animal consistency for neural responses is high both within reward condition (e.g. just spatial modulation in reward+ and reward- separately), and across both spatial and reward-modulated conditions overall (Fig. S66..
As baseline, we then evaluated the ability of the original non-reeaardmmodllaeed UGRNN-ReL.U- Place Cell model to match neural data. Consistent with results reported in the previous sections, this model achieved high neural predictivity in reward+ and reward- separately (Fig. 5b, hatched bars). However, as expected, this non- non-modulaeed model was ineffective at predicting response patterns across both reward conditions (Fig. 5b. reward+/-). Moreover, simply augmenting the network input to receive an additional binary reward state (Fig. 5p. "Reward Input"), ) but without using that input to specifically modulate input velocities output behavior, did not substantially improve neural predictivity, showing the need for nontrivial integration of reward-modten modulatdd state.
We then evaluated the reward-modulated networks, generating network outputs for comparison with the reward+ and reward- conditions by modulating velocity inputs to match each condition during testing (sce supplement for details). We found that the pure-explore (s 1) model is substantially

---

better at matching neural responses across reward conditions than the reward+/- baseline. This perhaps somewhat surprising since in this comparison the underlying neural network model is identical, just evaluated with or without reward-modulate input data during testing. This result suggests that a substantial fraction of the reported reward-modulane simply be input-driven, lending new interpretation to results of [Butler et al. 2019). (It may be useful to note that this result represents a model-based control that was inaccessible to the authors of [Butler et al. 20191.) .) However, we did find that exposing the network to mixture of exploration and exploitation behaviors during training does lead to networks with somewhat improved neural predictivity. Results were largely robust to the specifie proportion of statisticall-siinfffcaall
effect MEC may actually
vs-exploitation (Fig. 5p), though the best model (at e = 0.375) model was statistically significantly better than alternatives and had a substantially closer match (KS 0.13 vs. KS- 0.81) to the data's 's unit-level remapping across conditions than the original e = 1 model dt). These results are consistent with there being some nontrivial WiC reward- late responses beyond simple input modulation alone.
6 Discussion
We have identified goal-driven neural network model of MEC that is quantitatively accurate across a wide variety of common experimental conditions, and that can be used to generate nontrivial insights about the underlying mechanisms and functional roles of mouse MEC. More generally, our results suggest that constraint-driven neural networks may provide fruitful approach for studying navigation and memory in the MEC, and beyond.
Our work suggests the existence not of specialized class of heterogeneous cells that is functionally segregated from classic cell types, but rather a continuum of cells within a single unified network that naturally encompasses grid, border, and heterogeneous cells. Future research the computational foundations of MEC/HPC may thus be well-served by putting less emphasis on identifying cleanly- stereotypical cell types (such as grid cells) or perfecting mathematically simple models of single such cell-types, and looking instead for holistic computational ideas that move beyond "easy to visualize" but perhaps overly-simplise tuning-curve categories.
The nature of the understanding afforded by such a modeling approach comes from their ability to make inferences about what constraints (both structural and functional) are consistent with the data. The current work rules out a variety of simple network connectivity diagrams as inconsistent with MEC cell data, and narrows the space of functional goals MEC circuit weights might be optimized for over evolutionary timescales. Improving the results here will hopefully narrow these constraints yet further, with the ultimate goal of identifying constraints that yield the uniquely correct MEC circuit diagram and synaptic weights - or at least, the narrowest set of such networks consistent with the inherent variability between animals in the real population.
There are some key limitations, however, on our results. First, we attempted to identify the simplest underlying transform that would map animals within a population to each other, using this as the basis for conclusions both about unit-type reliability and model-data comparisons. While the philosophy of this approach may be sound [Cao and Yamins 2021a), in practice it is possible that we could narrow the transform class further by (e.g.) enforcing that it be invariant to one more more properties of classical cell-types (e.g. periodicity). Finding algorithms better identify sharp inter-animal transform classes will be an important topic for future work.
Moreover, we remain unconvinced that MEC just "is" a UGRNN- ReLU-Place Cell network, despite a network of this architecture having explained essentially all the data we had available to challenge it. It is possible that matching all our existing data is too easy a test. Would this network generalize to more complex situations, with increased variability along key axes such as spatial structure (e.g. environments with corridors and looping interconnections). nontrivial but spatially informative cues, and rewards exhibiting complex and temporally variable patterns? We do not think it is at all obvious that it would. The proper conclusion from this work is thus not that current best model is actually correct, but rather that the fairly simplified setup typical of experiments in MEC is not sufficiently complex to falsify it. This present situation is concrete manifestation of the "contravariance principle" of neural modeling [Cao and Yamins) (021b) - the idea that working in a more complex experimental environment might actually make it easier to identify actually correct model, by virtue of reducing susceptibility to spurious models that are apparently consistent with too-simple data. Future experiments should engage mice in more complex environments and behaviors.
10

---

7 Acknowledgements
We are grateful to Alex Gonzalez, David Sussillo, and John Wen for helpful discussions. We thank the anonymous reviewers for their feedback on a draft of this manuscript. A.A. received support from the Swiss National Science Foundation (P2BSP3 181743 and P400PB P400PB_191076) I.L. supported by funding from the Wu Tsai Neurosciences Institute under Stanford Interdisciplinary Graduate Fellowships and a Bertarelli fellowship. A.H.W. W. received funding support from the National Institutes of Health BRAIN initiative (IF32MH122998-01 and the Wu Tsai Stanford Neurosciences Institute Interdisciplinary Scholar Program. S.G. is supported by the James S. McDonnell Foundation, Simons Foundation, and National Science Foundation CAREER Award for funding while at Stanford. L.M.G. is supported by the Office of Naval Research N0014181 Simons Foundation SCGB 542987SPI, the James S. McDonnell Foundation, and the Vallee Foundation. D.L.K.Y. is supported by the James S. McDonnell Foundation (Understanding Human Cognition Award Grant No. 220020469), the Simons Foundation (Collaboration the Global Brain Grant No. 543061), the Sloan Foundation (Fellowship FGG-2018-100 88-1963), the National Science Foundation (RI 1703161 and CAREER Award 1844724), the DARPA Machine Common Sense program, and hardware donation from the NVIDIA Corporation.
References
M. Abadi, P. Barham, Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, al. Tensorflow: A system for large-scale machine learning. 12th {USENIX) symposium on operating systems design and implementation (OSDI) 16), pages 265-283, 2016.
D. Aronov, R. Nevers, and D. W. Tank. Mapping non-spatial dimension by the hippocampal-cntorhinn circuit. Nature, 543(7647):719-722, 2017.
A. Banino, C. Barry, B. Uria, C. Blundell, T. Lillicrap. P. Mirowski, A. Pritzel, M. J. Chadwick, T. Degris, J. Modayil, et al. Vector-based navigation using grid-like representations in artificial agents. Nature, 557 (7705):429 433, 2018.
C. N. Boccara, M. Nardin, Stella. J. O'Neill, and Csiesvari. The entorhinal cognitive map is attracted goals. Science, 363(6434)11443-4447 2019.
Burak R. Ficte. Accurate path integration in continuous attractor network models of grid cells. PLoS Comput Biol, 5(2)xe000029, 2009.
W. N. Butler, K. Hardcastle, and L.. M. Giocomo. Remembered reward locations restructure entorhinal spatial maps. Science, 363364434))1447111522 2019.
M. G. Campbell, S. A. Ocko, C. S. Mallory, I. I. Low, S. Ganguli, and L. M. Giacomo. Principles governing the integration of landmark and self-motion cues in entorhinal cortical codes for navigation. Nature neuroscience, 1096:100961106, 2018.
R. Cao and D. Yamins. Explanatory models in neuroscience: Part 1-taking mechanistic abstraction seriously. arXiv preprint arXiv:2104.011 0144000,0..4..... 2021a.
R. Cao and D. Yamins. Explanatory models in neuroscience: Part 2 constraint based intelligibility. arXiv preprint arXiv:2104.01489, 2021b.
K. Cho, B. van Merriënboer, D. Bahdanau, and Y. Bengio. On the propertics neural machine translation: Encoder-decoderr approaches. In Proceedings of SSS7-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pages 103-111, Doha, Qatar, Oct. 2014, Association for Computational Linguistics. doi: 10.31155v11114-44022 httpl:://://// aclweb........ . org/ anthologyytoyooeaaaaaaaaottooo hwaplloooyy/1444401000 401-4012
J. Collins, J. Sohl-Dickstein, and D. Sussillo. Capacity and trainability in recurrent neural networks. In ICLR, 2017.
A. Constantinescu. J. X. O'Reilly, and T. E. Behrens. Organizing conceptual knowledge humans with gridlike code. Science,
1464-1468, 2016.
J. J. Couey, A. Witoelar, S. Zhang, K. Zheng. J. Ye, B. Dunn, R. Czajkowski, M.-B. Moser, E. L. Moser, Y. Roudi, et al. Recurrent inhibitory circuitry as mechanism for grid formation. Nature neuroscience, 16(3): 318 324, 2013.

---

C. J. Cueva and X.-X. Wei. Emergence of grid-like representations by training recurrent neural networks to perform spatial localization. ICLR 2018, 1rXIv:1803.07770, 2018.
Y. Dordek, D. Soudry, R. Meir, and D. Derdikman. Extracting grid cell characteristies from place cell inputs using non-negative principal component analysis. Elife, 5:e10094, 2016.
J. L. Elman. Finding structure in time. Cognitive Science, 14(2):179-2) 2111 1990.
U. M. Erdem and M. Hasselmo. A goal-directed spatial navigation model using forward trajectory planning based grid cells. European Journal of Neuroscience, 35(6):916-923) 2012.
S. Farhoodi, M. Plitt, L. Giocomo, and U. Eden, Estimating fluctuations in neural representations of uncertain environments. Advances in Neural Information Processing Systems, 33, 2020.
M. C. Fuhs and D. S. Touretzky. spin glass model of path integration in rat medial entorhinal cortex. Journal of Neuroscience, 26( 26(66)4466-4426,, 2006.
T. Hafting, M. Fyhn, S. Molden, M.-B. Moser, and E. I. Moser. Microstructure of spatial map in the entorhinal cortex. Nature, 436(7052):801 806, 2005.
K. Hardcastle, N. Maheswaranathan, S. Ganguli, and M. Giocomo. multiplexed, heterogeneous, and adaptive code for navigation medial entorhinal cortex. Neuron, 94(2)375-337. 2017.
J. R. Hinman, Brandon, R. Climer, Chapman, and M. Hasselmo, Multiple running speed signals in medial entorhinal cortex. Neuron, 91(3):666 679, 2016.
Hochreiter and Schmidhuber. Long short-term memory. Neural Computation, 173:113551 1997.
A. J. Kell, D. L. Yamins, F. N. Shook, V. Norman-Haignere, and J. H. McDermott. task-optimized neural network replicates human auditory hehavior, predicts brain responses, and reveals a cortical processing hierarchy. Neuron, 9833):33,-64, 2018.
D. P. Kingma and Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Kropft. E. Carmichael, M.-B. Moser, and E. Moser. Speed cells in the medial entorhinal cortex. Nature, 523(7561):4194222 2015.
R. Langston, Ainge, J. Couey, C. Canto, T. L. Bjerknes, M. Witter, E, Moser, and M.-B. Moser. Development of the spatial representation system in the rat. Science, 32865985) 2010.
I. Low, A. Williams, M. G. Camphell, S. Linderman, and L.. Giocomo. Dynamic and reversible remapping of network representations in an unchanging environment. bioRxiv, 2020.
C. Mallory, K. Hardcastle, M. G. Campbell, A. Attinger, I. Low, L. Raymond, and L. M. Giocomo. Mouse entorhinal cortex encodes a diverse repertoire of self-motion signals. Nature communications, 12(1):1-20 2021.
J. A. Michaels, S. Schaffelhofer, A. Agudelo- Toro. and H. Scherberger. A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping. Proceedings the National Academy of Sciences. (17(50)332124-3213 2020.
A. Ocko, K. Hardcastle, I.. M. Giocomo, and S. Ganguli. Emergent elasticity in the neural code for space. Proceedings of the National Academy of Sciences, 115(50):E1 (1150)))11798-F1 1798-E1 1806, 2018.
J. O'Keefe and J. Dostrovsky. The hippocampus spatial map: Preliminary evidence from unit activity in the freely-moving rat. Brain research, 1971.
Sargolini. M. Fyhn, Hafting, B. McNaughton, M. P. Witter. M.-B. Moser, and E. Moser. Conjunctive representation of position, direction, and velocity in entorhinal cortex. Science, 312(5774):758-765 2006.
W. E. Skaggs, B. L. McNaughton, K. Gothard, and E. J. Markus. An information-theon -heerreticc approach to deciphering the hippocampal code. In Proceedings of the 5th International Conference on Neural Information Processing Systems, pages 1030-103 1037, 1992.
T. Solstad, C. N. Boccara, E. Kropff, M.B. Moser, and E. I. Moser. Representation of geometric borders in the entorhinal cortex. Science, 322(9909): 110009)::866..
2008
B. Sorscher, G. Mel. S. Ganguli, and S. A. Ocko. A unified theory for the origin of grid cells through the lens of pattern formation. Advances in Neural Information Processing Systems, 32, 2019.
12

---

B. Sorscher. G. C. Mel. S. A. Ocko, L. Giocomo, and S. Ganguli. A unified theory for the computational and mechanistic origins of grid cells. bioRxiv, 2020.
K. L. Stachenfeld, M. Botvinick, and S. J. Gershman. Design principles of the hippocampal cognitive map. Advances in neural information processing systems, 27:2528 2536, 2014.
D. L. Yamins and J. J. DiCarlo. Using goal-driven deep learning models to understand sensory cortex. Nature neuroscience, 19(3):35-365, 2016.
K. Yoon, M. A. Buice, C. Barry, R. Hayman, N. Burgess, and I. R. Fiete. Specific evidence of low-dimensional continuous attractor dynamics in grid cells. Nature neuroscience, 16(8):1077-1084, 2013.
K. Zhang. Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: theory. Journal Neuroscience, 16(6):2112-211 12-2126, 1996.
13


Reviews:
[{"review_id": "vlLVKhTCHqx", "reviewer": "Reviewer_vfhu", "summary": "By adopting 'similarity transform' methods, this study compares the predictability of different computational models of MEC neurons. The best model not only can explain the firing patterns of grid cells but also that of heterogeneous cells. These findings further allow them to examine the role of heterogeneous cells which has not been clearly identified with experimental approaches. This study suggests that the heterogeneous cells might play critical roles in path integration even more than what grid cells contribute. This manuscript is well-written, and the results provide a theoretical foundation for future studies using a more experimental approach. I only have minor clarification questions and suggestions to embrace a broader readership such as neuroscientists. ", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. What does it mean by that the place cell activity is predicted by the network trained in a different size arena at the MEC level? Are neurons (units) identified as grid-cells in one environment more likely to be grid-cells in the larger environments as if expanding the toroidal attractor manifold? If it is the case, how does the level of gridness / heterogeneity changes from one to the other environment? This may be hard to test in experiments since velocity inputs are different even in the same environments but it might be able to test within the computational models.\n\n2. Can the model also predict that the reward attracts the firing fields of the grid cells, as the original data have shown? Is there any possibility that differences in the rate maps between reward+ and reward- conditions stem from different profiles of velocity (such as the animals approaching the rewards rapidly and staying longer) not by the reward itself? I see that the authors argued this a bit, but can your results suggest that this reward-associated predictability is not fully caused by the velocity profiles only but by using the reward input layer?\n\n3. How can we interpret the finding that the place cell activity is better predicted by path integration than the actual coordinate? Is it because neighboring place cells can encode two locations that are actually far from each other? Meanwhile, the high inter-animal consistency might suggest that the place cells mapping might not be totally random but reflect some spatial properties. It would be very helpful if the authors include what the results suggest and how the readers can interpret these findings.\n\n4. Some previous evidence suggests that place cells are independently formed from the grid cells, and often they are stabilized earlier than the grid cells (while many others support the idea of the contribution of grid cells on the place cells). Is this finding compatible with those previous studies (e.g. those studies did not take into account the effects of heterogeneous cells)? Does this study suggest that the place cells are fully dependent on and driven by the MEC neurons? If not, some limitations may need to be included.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "ieIKWmxCrj", "reviewer": "Reviewer_PoER", "summary": "The authors address two important questions regarding the non-grid cells in MEC. First, is their response consistent, and not just noise? Second, can they be used as criteria for testing models of MEC functionality? The second question allows to probe the functional significance of these cells, and the effect of incorporating them on various model properties.\nThe authors’ main tool is a linear mapping from a population of cells into a single target cell. This is done using rate maps of cells, and cross validation on the spatial bins. Several existing models are compared, with one model singled out as corresponding best to experimental data.\nThe functional relevance of grid and non-grid cells is evaluated using ablation studies on the model, where path integration ability is used as a measure. Generalization of the models is assessed using either different arenas or a reward-biased setting.\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**EDIT** *After carefully reading all the reviews and author responses, I am raising my score to a 6.*\n\n\nI think these are important and timely questions. The idea of comparing firing patterns across animals and between animals and models is a good one. It has been used in other studies, but I think not many compared between-animal fit as a reference point. Checking the interplay between fitting the entire population and functional aspects is also a promising approach.\nUnfortunately, as I detail below, the methods and analyses used by the authors do not adequately answer the questions or support the conclusions of the paper.\nThe main tool used is consistency between animals and models. I found very little explanation of the motivation for this choice of metric, and it was also quite hard to understand the details of the process. The authors chose a linear mapping from a population of neurons into a single neuron. Crucially, the train and test data are rate maps and the test data are 50% of the bins, under a random choice. The rate maps are also smoothed with a 1-bin wide Gaussian. This means that the test data is not that different from the train data. Given this procedure, I was actually surprised that not all mappings resulted in very high scores (although the animal-to-animal scores were). I can only assume that something about the statistics of the maps under different architectures is responsible, but this is not detailed or explored. Furthermore, the authors ask on line 120 “What firing patterns of MEC cell populations are common across multiple animals?” This is a great question, but I don’t see how the analysis in the paper answers it. It could be that one animal has grid cells and another animal has stripe cells, but they can be linearly combined to give one another. \nAnother major tool used is the gridness score. A threshold of 0.3 (typical for the literature) is used to divide the cells, and then compare the different populations. The distributions of gridness score, however, show a wide unimodal distribution with nothing special happening at 0.3. It would strengthen the conclusions if the authors check a range of thresholds, or even omit the cells with a gridness score close to 0.3 (for instance only analyze the bottom and top quadrants).\nAs for clarity – reading the figures was very challenging. The legends and captions do not provide a complete explanation, and the figures are extremely dense. \nThere is related literature on the importance of heterogeneous cells that should be probably discussed: Meshulam et al, Neuron 2017; Stefanini et al, Neuron 2020\nSpecific points:\n1.\tLine 13. “that that”\n2.\tLines 52-62. The choice for the similarity transform should be better explained.\n3.\tL100 “solves the neurons”. Not clear when reading this\n4.\tSupplementary: the transform is not explained clearly enough, given its centrality in the paper.\n5.\tSupplementary: All models have the same trajectory. Animals do not.\n6.\tSupplementary: 50% of the spatial bins is quite a dense sample. In general, using different trajectories, and actual responses of the neurons (instead of rate maps) might be a more fair comparison.\n7.\tL136: “between the population rate maps”. This should be stressed (using rate maps and not responses to specific trajectories), and introduced earlier in the paper\n8.\tFigure 1A : mirror text\n9.\tFigure 1C: the boundary at 0.5 is confusing. Not clear what it refers to.\n10.\tFigure 2: Extremely hard to read. The caption doesn’t provide all the information.\n11.\tL180: models are substantially different. How consistent is the pattern of figure 2 across different animals? Error bars are across 2861 units. It would be useful to see error bars for the 8 animals. Also – this can be quantified. What is the (spearman) correlation between the different networks across cells, across animals?\n12.\tFigure 3A caption. Ground truth is gray, not yellow.\n13.\tFigure 3A. The match between gridness distribution for an untrained network is puzzling. Is this due to initialization of the states? Is this due to some other property? Tuning of hyperparameters? \n14.\tFigure 3A. The distribution of gridness in the Dordek et al paper is between 0.8 and 1.2. How come it is so different here?\n15.\tL212 “were is similar rank order”. Please quantify across animals.\n16.\tL217 “also also”\n17.\tL219 the -> they\n18.\tL223 grid score is explained here, but used in figure 1.\n19.\tL238 “KS=0.1 significance level”. Why is this level special?\n20.\tL238 “KS-distance was generally in line” – please quantify\n21.\tFig 4A – It is very problematic to draw a regression line and state that “the empirical relationship is comparatively strong”\n22.\tL244. “which” typo\n23.\tL255. The motivation for this comparison is unclear.\n24.\tL306: “it is quite reasonable”. Perhaps. Perhaps not.\n25.\tThe discussion is extremely short. Why do untrained networks have a “good” gridness distribution? Why are heterogeneous cells more important than grid cells? How does this relate to place cell literature? Mixed selectivity in general?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Societal impact – not relevant.\nLimitations: mentioned above. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "TQbIXsucx9", "reviewer": "Reviewer_7Y6g", "summary": "The authors employ RNN models under different conditions to model MEC cells with both well-understood (e.g., place cells) and poorly-understood (heterogeneous cells) tuning properties. By training the network to minimize the place cell-mediated path integration, the network performed incredibly well in explaining MEC-cell dynamics. By removing neurons in the model, the network's path-integration performance was found to be at least as dependent on the model's heterogeneous neurons as on the stereotypical ones. The authors then finish by introducing a reward-based foraging model, again finding responses that match experimental results very well.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "By testing several different neural network models with different loss functions, the authors provide a strong argument in support of heterogenous MEC cells' functionality as key to path integration, in spite of a lack of clear tuning properties. They have integrated experimental data from different animals together well, and have demonstrated that neural network models can provide a key roll in understanding biological neurons and neuronal networks.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3.5"}, "extra_text": {}}, {"review_id": "PjRlLOBdzka", "reviewer": "Reviewer_GE9u", "summary": "The authors used linear regression to first demonstrate the robustness of non-grid-like responsive cells in experimental recordings of MEC cells. They then used the same regression to assess the similarity of responses derived from the goal driven training of RNNs to MEC responses. They found that responses derived from RNNS contained these non-grid-like responses and explained neural activity better than a low-rank decomposition of place fields.  These non-grid-like responses were crucial for path integration and likely explained the difference in similarity matching between goal-driven training and low rank decomposition.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "There are significant pros and cons with this paper currently. It is written clearly and each part is well-motivated. Adequate controls were performed, and is fairly rigorous.  However, the central finding is simply that goal-based training provides a better fit to real data as compared to a specialized grid cell model. There are some hints that this gap in performance arises from responses that are not easily categorized (Figure 3+4), but there is little insight over the nature of these response types -- are they border cells, heading cells, mixed selectivity cells, etc. I would like to know what these response types are and why they improve similarity matching. I feel that a further exploration of these response types, as detailed in the comments, will greatly strengthen this paper, so I will await author responses.\n\nComments:\n\n- The main contribution of the paper rests upon understanding cells that cannot be categorized classically into the typical navigation variables such as border / grid / head direction cells. However, there is a clear distinction between border / head direction cells and cells with heterogeneous of mixed selectivity (i.e. cells with both heading and grid responses). But are all of these types of responses treated as heterogeneous cells because they have low grid scores? If so, the definition supplied in the introduction does not agree with the definition of heterogeneous cells in the results section. Shouldn't border, heading, and heterogeneous cells be treated independently?\n\n- It is suggested that \"heterogeneous\" cells, operationally defined as having low grid scores in figure 1, are responsible for the performance difference between RNN-based goal-driven training vs NMF (Figure 4C). If you take these cells out, is there less of a difference between NMF vs goal-driven approaches? How does the decrease in the performance gap compare to if border cells or head direction cells were taken out? Put another way, is the failure of NMF in reproducing neural responses simply because it can't reproduce border and heading responses, or because it can't reproduce mixed selectivity responses?\n\n- It's unclear how many of these heterogeneous cells exist in the data. Even if this is reported previously it should still be shown here to provide context. Figure 1c also does very little justice in showcasing that heterogeneous cells constitute a unique functional population.\n\n- I have some concerns over the notion of similarity. For one-to-one mapping, can neuron A1 in animal A be mapped, with replacement, to multiple neurons, B1 and B2, in animal B if A1 is the best match to B1 and B2? If this is allowed, then hypothetically, animal A, despite having only a single grid cell and 99 head direction cells, may actually serve as a good template / have a high similarity using ridge regression to animal B, which has 100 grid cells as long as a sufficient basis set of grid, head, border, etc cells exist in every recorded animal. Perhaps the current notion of similarity does not factor in differences in the relative proportion of unique responses across recorded animals? If so, the notion of similarity in the paper is different from a neuroscientists' notion of similarity when viewing population PSTHs of each animal, and may need some degree of qualification in the text.\n\n- Some simple negative controls are needed to interpret similarity measures. For instance, similarity should not be high when regressing a classical grid response when exclusively using classical head direction cells, and vice versa and also for using other classical response types, such as border cells, etc. The similarity levels will also provide the reader with some baseline to evaluate what the numbers mean in figure 1B. \n\n- I may be reading figure 1C wrong, but it appears that target unit predictivity is marginally higher for het cells than grid cells when using a smaller fraction of source population. Doesn't this mean that het cells actually require fewer source cells to achieve high similarity scores?\n\n- The legends to the different sub-bars in Figure 2 should be made clearer. \n\n- It's unclear to me what the inclusion of reward responses and Figure 5 adds to the central findings of paper.\n\n \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Not applicable.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Dear Authors, 

congratulations on your paper being accepted at Neurips. Reviewers founds this a highly interesting and intriguing submission, with results that will likely be of interest to a wide range of researchers, and in particular researchers aiming to understand how diverse cells in the MEC underly the ability of animals to navigate. At the same time, this paper also triggered substantial discussion-- the primary concern by the reviewers was that this was a dense paper loaded with results, but low on detailed explanations, which made it challenging for the reviewers to assess the validity of the methods and results. During the discussion phase, extensive additional explanations and clarifications were given, and we decided that the paper -- with these additional explanations -- would be a great addition to Neurips. We urge you to ensure that these explanations also appear in the final paper (or, in most cases, in the supplement).

With best regards, your AC
Decision: Accept (Spotlight)