No Free Lunch from Deep Learning in Neuroscience: A Case Study through Models of the Entorhinal-Hippocampal Circuit
Authors: Rylan Schaeffer
Mikail Khona
Ila R Fiete
Venue: NeurIPS 2022 Accept (2022)
Keywords: neuroscience
deep learning
grid cells
path integration
representation learning

Abstract: Research in Neuroscience, as in many scientific disciplines, is undergoing a renaissance based on deep learning. Unique to Neuroscience, deep learning models can be used not only as a tool but interpreted as models of the brain. The central claims of recent deep learning-based models of brain circuits are that they make novel predictions about neural phenomena or shed light on the fundamental functions being optimized. We show, through the case-study of grid cells in the entorhinal-hippocampal circuit, that one may get neither. We begin by reviewing the principles of grid cell mechanism and function obtained from first-principles modeling efforts, then rigorously examine the claims of deep learning models of grid cells. Using large-scale architectural and hyperparameter sweeps and theory-driven experimentation, we demonstrate that the results of such models may be more strongly driven by particular, non-fundamental, and post-hoc implementation choices than fundamental truths about neural circuits or the loss function(s) they might optimize. We discuss why these models cannot be expected to produce accurate models of the brain without the addition of substantial amounts of inductive bias, an informal No Free Lunch result for Neuroscience. Based on first principles work, we provide hypotheses for what additional loss functions will produce grid cells more robustly. In conclusion, circumspection and transparency, together with biological knowledge, are warranted in building and interpreting deep learning models in Neuroscience.

No Free Lunch from Deep Learning in Neuroscience: A Case Study through Models of the Entorhinal-- Circuit
Rylan Schaelfer
Mikail Khona
lla Rani Fiete Brain and Cognitive Sciences MIT fiete@mit.edi
Computer Science
Physics MIT mikail@mit.ed
Stanford rschaef0cs. sta.soodc..ssaaoooddded.
Abstract
Research in Neuroscience, in many scientific disciplines, is undergoing a re- naissance based on deep learning. Unique to Neuroscience, deep learning models can be used not only as tool but interpreted as models of the brain. The cen- tral claims of recent deep learning-based based models of brain circuits are that they make novel predictions about neural phenomena or shed light on the fundamental functions being optimized. We show, through the case-study of grid cells in the entorhinal- hhiptiiinal-ppppappallll circuit, that one may get neither. We begin by reviewing the principles of grid cell mechanism and function obtained from first-principles modeling efforts, then rigorously examine the claims of deep learning models of grid cells. Using large-scale architectural and hyperparameter sweeps and theory- driven experimentation, we demonstrate that the results such models may be more strongly driven by particular, non-fundamental. and post-hoc implementation choices than fundamental truths about neural circuits or the loss function(s) they might optimize. We discuss why these models cannot be expected produce accu- rate models of the brain without the addition of substantial amounts inductive bias, informal No Free Lunch result for Neuroscience. Based on first principles work, we provide hypotheses for what additional loss functions will produce grid cells more robustly. In conclusion, circumspection and transparency, together with biological knowledge, are warranted building and interpreting deep learning models in Neuroscience.
Introduction
Over the past decade, deep learning (DL) has underpinned nearly every success story in machine learning, c.g., [57 6 and increasingly many advances in fundamental science research, e.g., (36). In neuroscience, DL similarly gaining widespread adoption as an indispensable method for behavioral and neural data analysis
[52 50 28
40 46).
But DL offers a unique contribution to neuroscience that goes beyond its role in other fields, in that deep networks can be viewed as models of the brain. The success of DL in matching surpassing human performance suggests it is now possible to construct models of circuits that may underlie human intelligence. As recent review wrote, "researchers are excited by the possibility that deep neural networks may offer theories of perception, cognition and action for biological brains. This approach has the potential to radically reshape our approach to understanding neural systems" (54). Further, DL a democratizing force for building neural circuit models of complex function.
Here, we examine the essential claims (and promises) of DL-based models of the brain, which are that Because the models are trained on a specific optimization problem, if the resulting representations
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

serreticallygguidee match what has been observed in the brain, then the models reveal which optimization problem(s) the brain is solving, or 2) These models, when trained on sensible optimization problems, should generate novel predictions about the brain's 's representations and behavior.
These are extremely valuable potential contributions. However, given the nascent nature of such approaches and the exuberance accompanying some claims in current work, we should examine them carefully. In DL, some successes attributed to novel algorithms have been shown to instead stem from seemingly minor or unstated implementation choices 65] 21 351 In this paper, we ask whether Neuroscientists should similarly be cautious that DL-based models of neural circuits that make specific claims about revealing the brain's 's optimization functions or that generate specific neural tuning curves may tell us less about fundamental scientific truths and more about programmers' particular implementation choices, and might be more post hoc than predictive.
To explore these questions, we evaluate recent DL-based models of grid cells in the entorhinal- hippocampal circuit. The medial entorhinal cortex (MEC) and hippocampus (HPC) are part of the hippocampal formation, a brain structure critical for learning and memory. In a pair of -obel-prize winning discoveries, HPC was shown to contain place cells [48), and MEC, its cortical input, was shown contain grid cells 30 Place cells each fire one or several seemingly random locations in small and large environments [5I]. while grid cells fire in a spatially periodic hexagonal lattice pattern in all two- vro-dimensional environments 30]. Over five decades, the hippocampal formation has been central to understanding how the brain organizes spatial and episodic memory, for experimentalists and theorists alike, with many mysteries remaining. recent series of DL-based models of the circuit 15 3 59
68 47] present story that training neural circuits on the task of path integration (PI) (i.e., updating one's positional estimate by integrating velocity from self-motion), possibly with the addition of a non-negativity constraint on firing rates [59]. results in the emergence of grid cells.
We use code from prior publications to demonstrate these results are due not to the core (path integration) task the network was trained to perform but to separate and specific post-hoc implemen- tation choices that implicitly made the known tuning shapes of grid cells part of the target, even though the narrative accompanying many of these papers suggests that the emergence of those tuning curves rather naturally "falls out" from training on the core task. By leveraging large-scale exploration and hypothesis-drin driven experimentation, we show:
1. Networks trained on path integration tasks almost always learn to optimally encode position, but almost never learn grid-like representations to do
2. The emergence of grid-like representations depends wholly on specifically chosen structural choices of the network and readouts, rather than on the path integration task, and the structural choices are based on the implicit goal of obtaining grid-like responses.
3. Under more-rcalistic structural choices for the network readouts, grid cells disappear.
4. Even with the structural choices, grid emergence can be hyperparameter and seed sensitive and non-generic.
5. Multiple grid modules, a fundamental characteristic of the grid cell system, do not emerge from path integration.
6. Grid periods and period ratios, contrary to assertions 33, are not determined by the task and are not fundamental properties that can serve as predictions about observed values in the brain; rather, they are set by hyperparameters selected by the programmer.
In short, deep learning models of MEC-HPC yield grid-like units only when a sequence of specific and biologically implausible implementation choices are made to intentionally bake grid-like units into the task-traing -rained networks, and the emergent grid-like units lack key grid cell properties. Given the non-genericity of grid cell emergence in successfully path integrating networks, it is highly improbable that DL models of path integration would have produced grid cells as a novel prediction from fask-training, had they not already been known to exist.
Moreover, it is unclear what new understanding the current models contribute, beyond or even up to what has already been shown for these circuits by existing models. Our results challenge the notion that deep networks offer a free lunch for Neuroscience in terms of discovering the brain's 's optimization problems or generating novel priori predictions about single-neuron representations, and warn that caution, transparency, and biological knowledge are needed when building and interpreting such models.
2

---

a
readout
spatial
b
deocaled pat sath
Pan resgrzion
cals
d
Place
Path
velocily Inpurs
encoding
Placo oa single fieles & homageeauu ffricicc Gaussion nuring Narrow range
supervised
learning
Vyls
Sea
Figure I: Setup and claim. (a-b) Schematic of recurrent neural network setup to predict some encoding of 2D position from 2D velocity. (b-c) Grid cells in recent DL. papers, obtained in part by learning to path integrate [1S 3 59 60 47]. conclude that path integration creates grid cells. (d) We show that most artificial neural networks (ANNs) trained path integrate can do so, but only very small subset of such networks yield grid cells, and that grid cell emergence results in ANNs are post hoc: they result from post-facto selection of architectures, functions, and hyperparameter settings that specify grid cells as an implicit target.
Code availability: Our work benefited from previous publications' published (47) To facilitate further research, we similarly release ours: gittabbccnniieee..a//////22222/0
2 Background: Grid Cells
Grid cells 30 are found in the medial entorhinal cortex of mammals and are tuned to represent the spatial location of the animal as it traverses 2D space. Each cell fires at every vertex of a triangular lattice that tiles the explored space, regardless of the speed and direction of movement through the space. As population, grid cells exhibit several striking properties that provide support for specialized and modular circuit. Grid cells form discrete modules (clusters), such that all cells within a module share a common period and orientation, while different modules express discretely different spatial periods [62). The period ratios of successive modules have values in the range 1.2-1.5.
The mechanism underlying grid cells is widely supported to be through attractor dynamics: Translation-invariann lateral connectivity within the grid cell network results in a linear Turing instability and pattern formation 9 23 [7) These models explain how grid cells can convert velocity inputs into updated spatial estimates, and make several predictions that have been confirmed in exper- iments, including most centrally the stability of low-dimensional cell-cell relationships regardless of environment and behavioral state, that define a toroidal attractor dynamies 24
72
26
25
Experimental approach
The central messaging of existing DL models of grid cells is that training ANNs Path Integration (PI) - using velocicy estimates to track one's ' spatial position - causes the networks to learn grid cells [15 59 47), even when the technical portions and code implementations of the papers involve many other critical choices without which grid cells would not emerge.
Thus, we focus on asking: if a recurrent neural network is trained on PI, what is the approximate probability that it will exhibit grid cells? We follow the setup used by many previous papers: 2.2 m 2.2 arena is created, then, spatial trajectories (i.e. sequences of positions and velocities) are sampled. Networks receive as inputs the initial position and velocities, and are trained to output (some encoding of) the positions in supervised manner (Fig. [Tb)). There are multiple possible encodings of position, and, as we will show, this choice is critical. Two simple encodings are Cartesian [15) or polar (. Another encoding scheme is via bump functions in 2D space, with each output assigned different positions that collectively tile the space with similar tuning curve shapes 59 59 47). This
encoding has been equated with place cells, even though place cells' fields tend to be heterogeneous in size and shape [1 19], as well as in number [51], and, unlike the choice of a difference-on Gaussians (DoG) difference-on of Softmaxes (DoS) readout tuning in ANN models, do not exhibit any inhibitory surround. See Appendix for position encoding details. For all encodings, supervised learning is used train the network via backpropagation through time.
Spatial tuning assessments The spatial tuning ratemaps of hidden units in the networks are the primary basis for comparison with the brain's grid cells. To compute ratemaps, trained network is
3

---

X,- (x.y)
home base
Polar:
Counstonn AlA) no 20
Half readout unis radaa::
Gaaslane
"Low--imensional" Cartesian:
An A matris wet entnis Unfom(-1.1)
(())))))))))))))))))()) me( mt
"High- dimensional" Cartesian:
readout unis arguaa:
Ac R2 R Unlford
exp con(6) wd) [-
Difieennc of Sahmasss
whene enies give Sohruc 14-44
Abb Sotree
30g
Paothoed Ait Nodice Docotin Eve wn
Cartesian
1.0
Cartesian High Dim N 5793
CCumulatvve Densty
Polar High Dim N 1171
0.6
Prento Furm Freele Call Man Comer CCN Predee Eay
1255
DoG
0.2
DoB
N 1689
1810
117
1,22
1.19
0
0.26
1.26
Cartesian
Polar
Gaussian
Difference of Softmaxes (Do5)
Figure 2: Grid-like response requires highly specifie target encoding. (a) Readout encodings of spatial position. (b) Top: Across readout encodings, most networks learn the PI task. Bottom: Few networks display possible grid-like representations (threshold = 0.8). (c) Survival functions of grid scores per readout encoding. (d) Rate maps of topgrid-scoring units in ANNs performing good PI with i) Cartesian, ii) Polar, iti) Gaussian, iv) specifically selected (tuned) Difference-of-Softman (DoS) readouts. i)-i do not learn grid cells. Numbers above rate maps 60" grid scores.
evaluated long trajectories that the 2D environment, then each hidden unit's average activity per spatial bin is computed. Ratemaps of units are compared with grid cells through the gridness We are extremely lenient with classifying particular network training run success: if even single hidden unit has grid score above a certain threshold, we say the model possibly possesses grid cells. The grid score, when applied ANN units without additional criteria, is not perfect since cells classified by grid scores represent only an upper bound on the total number of grid cells (e.g. the high grid score given units with triangular symmetry without periodic pattern, Fig. 21 (Gaussian readouts)); for details, see Appendices and
score by 3 559
60 47
4 Networks trained on path integration tasks learn to estimate position, but rarely learn grid cells
We demonstrate that most aath-integrating networks do not converge to a grid-like solution, instead requiring very specific architectural choices including readout tuning functions. Grid-like representa- tions emerge when the programmer makes choices that, rather than relating to the path integration objective biologically realistic place cells, designed post-hoc to produce grid cells.
ran large-scale hyperparameter sweeps across common implementation choices: 1) Architectures: RNN 20 LSTM 33 GRU [3;; UGRNN [4]; 2) Activation: Sigmoid; Tanh; ReI.U; Linear;
3) Optimizers: SGD, Adam 41: RMSProp 32 4) Supervised Targets: Cartesian; Polar; high-
dimensional hump-like readout population code with Gaussian (3), cffference-of-Sofime (DoS)

---

or Difference-of -faeeecceoo-GGuusians (DoG) tuning curves. 5) Loss: mean squared error on the agent's Cartesian position [5) geodesic distance on the agent's polar position Mi cross entropy on a high-dimension & readout dropout, initialization, parameter 1.2 regularization, seed.
15 60 47
population of bump-like readout units 3 59 47 6) Miscellaneous: recurrent
For networks with bump-like population readouts, we additionally swept: 1) Width o of Gaussian readouts; 2) Whether the bump-like readouts have homogeneous or heterogeneous field widths; 3) For DoG or DoS readouts, the surround scale 8, i.e., the ratio between the inhibitory and excitatory def
Gaussian standard deviations (s (s=0f 01 0 ; 4) For DoG readouts, the ratio of amplitudes agg/aj between the two Gaussians. 5) Number of fields per readout unit. Evaluating the entire hyperparameter volume is computationally prohibitive, so we evaluated subvolume most consistent with previous papers, focusing our exploration around conditions that did produce grid cells. In this sense, our search was biased toward configurations shown to produce grid cell emergence and thus our findings about the fragility of these solutions conservatively favored these solutions much as possible. All sweeps are provided in Appendix F
To evaluate whether network learns to optimally estimate spatial position from velocity inputs, we measured its position decoding error using previous papers' methods (355 47
using the network's output Cartesian positions (if trained on Cartesian targets) by decoding position from the network's ' outputs. Any network with error < 10 cm was considered to have achieved optimal position encoding.
In total, we trained >> 11, 000 networks and found that most succeed in learning to path integrate (Fig. 24. Top). but few learn grid cell representations (Fig. [2. Bottom). This is consistent with earlier work [37 demonstrating that networks can learn to path integrate and solve other hard navigational problems (e.g. self-localization across multiple environments and identification of spatial environment from ambiguous cues, case of self localization and mapping or SLAMM) without grid-like units emerging as solution.
5 Grid-like unit emergence requires specific supervised target functions
We next sought to characterize when grid cells are learnt under different encodings of 2D spatial position in the readout units (i.e. supervised targets). We tested multiple encodings: i) Cartesian, ii) Polar, iii) Gaussian, iv) Seaerrenceeo--Guusiinee (DoG), and v) Difference-of-Softune
(DoS).
We found we that in the ANN network architecture of Fig. Ih. grid cells do not emerge from Cartesian or Polar readouts, consistent with earlier work (37]. Similarly, they do not emerge from Gaussian encodings (Fig. 2 337. Consistent with this result, the Gaussian readouts of [3] used in tandem with 50% dropout and a different architecture do not yield (square or hexagonal) grid cells without dropout (result not shown). and, as shown recently and independently by (70.. although grid-like responses can be obtained with Gaussian readouts after the addition another constraint, they disappear without. [59] 60) critiqued [ to show that the hexagonal patterning of cells in B was indistinguishable from low-pass filtered noise. However, their (59]6) 60) focus was to argue that a neural nonlinearity (which they termed non-negativity constraint, though any non-odd function suffices) robustly produces hexagonal firing- which they showed by replacing the simpler Gaussian-like readouts of with specific DoG/DoS readout.
We found that DoG/DoS readouts 59 60 were critical for producing grids (Fig. reproducing the main results of [59), even with non-negativity constraints: Lattices of any geometry (hexagonal or square) only emerge with DoS fields, corresponding to a small and particular subset of DoG fields.
Grid period values set by hyperparameters, and multiple modules do not emerge. Next, two prominent features of grid cells are their intrinsically set periods (invariant to the external environment) and the existence of a discrete set of grid periods that scale by a rough factor 1.4 between adjacent scales (62). Multi-periodicity is critical for unambiguous spatial coding over large scales. We asked whether ANN models generate multiple periods and whether their the period values are fundamental or hyperparameter dependent.
To ensure we would obtain at least some grid cells, we fixed the readouts to DoS, and swept over different scales og of the DoS. We found that almost all runs had unimodal distribution of grid periods (Fig. (3),, meaning the networks learnt only one module of grid cells. Contenporaneously with the Ne review process, other researchers independently reported the same result (56).
5

---

a
Grid Score Threshold: 0.8
0.20
b
Grid Score Threshold: 0.8
0.20
C
1.15
Grid Score Threshold: 0.8
30
Grdd Period Mode Ratoo
0.18
0.18
0.03
0.36
Mode
23
0.16
Densty
0.02
0
0.14 b
1.05
0.12
20
0.12
0.01
0.10
0.10
1.00
15
40
0.10
0:15
0.20
0.00
2
1001112
60' Grid Pertod
(m)
Index
The spatial scale of grids is set by hyperparameters and multiple modules do not emerge. (a) Over a wide sweep of DoS (Fig. [h bottom) target field widths gg the distribution of grid periods is unimodal (each color: period distribution from 3 runs with same gg value; different periods are only obtained by varying σE), meaning multiple grid modules do not emerge, in contrast to the brain's grid cell circuit. (b) The chosen target field width Or determines the grid period mode, meaning that grid period is not a prediction of the models. (c) If we smoothly sweep dg proxy for simulating different modules, the distribution of adjacent periods produces ratios closer to than the experimental ratios of ~ 1.4.
Figure 3:
Further, we found that the period of formed grid-like representation is completely determined by the width a of the externally imposed readout DoS (Fig. 3) and other specific parameter choices. The period of the grid-like responses increased monotonically with the width of the DoS readout (Fig. (3))
Since individual networks did not learn multiple modules, we used the somewhat discrete distribution of peaks of the single module formed when sweeping the DoS og more continuously to compute grid period ratios. These period ratios from adjacent peaks led to non-biological values (Fig. 3b).
Other details with DoG/DoS readouts affect grid emergence. In all DL-grid cell papers we examined [S 55 60 47,, we discovered implementation details critical the emergence of grid cells that were not stressed in the claims. As one example, we discovered an implementation detail essential for the emergence of grid cells in series of papers [59 60 47 that is unmentioned in main texts and supplements. These papers report using unnormalized equi-norm
Saareence-f-GGuusiin (DoG) readout target function (159) Appendix C1. (1, Methods 4.2, Appendix C1), i.e. DoG with amplitude parameters set to 1, but their code uses Difference-of -ffoott-aessss (DoS) target function. When we trained ideal grid-forming ReLU networks with equi- squi-norm DoG tuning curves, sweeping the receptive field and surround scale s, they did not result in grids (Fig. 4b). The Fourier analysis (below) explains why equinorm-Der -DG tuning should not produce lattices (Fig. 4b).
We next trained ReLU RNNs on general DoG readouts, sweeping the component amplitudes ag, ai while holding Øg, σz fixed at ideal values. The theory of [59] predicts that outside the feasible region (blue boxed region of Fig. 4c), no lattices should emerge, but inside the feasible region, all/most RNNs should learn grids. We found the first prediction held, but the second did not: most display grid score distributions comparable to or worse than low- www-pas-fiee siftered-then-thresee below the ideal- width DoS grid score distribution (Fig. 4d) To investigate, we swept densely inside the feasible region, additionally matching the amplitudes created by DoS (Fig. c Fig. I0] one run out of 1086 surpassed the DoS grid score distribution (a'g/ar RS 3.5714, seed=l), but its two "sibling" runs (all hyperparameters same: seeds: 0, 2), Fig. 4.. Thus, and contrary to 's theory based on static function- fitting, DoG readouts at most rarely produce lattices when implemented in actual RNN simulations.
noise, and well
Fourier analysis of Turing instability provides intuition for the preceding empirical results. Why do only
(DoS) or very specific DoG readouts produce grid-like units? We shed light on our present findings by restating the essence of previous analyses of first-principls models 99. 38 here, and leveraging the connection made between these models and trained deep networks [59]. In the first-principles continuous attractor models, neural dynamics are given by r(x) = (r(x) + g(W * r). where x designates the neural index (in continuum approximation for neurons), W * r designates the total (integrated) inputs from the network to the neuron at index I, and g is the non-lincarity. if the recurrent weight interaction is translationally invariant, then W (x,x'' = ((x x x') = W(A (Ar). Under DoG interactions:
W(Ar) = f(Ar) 0g exp (- (Ar)²
(Ax)²
- az exp
(1)
20
20
6

---

b
30
Cumulelive Denety
Eqnomm-Doo
Grid Score Threshold
0.75
Do8
20
Grid
0.75
0.50
0.8
110
0.00
0.10
(m)
0.15
0.20
Grid Score
Dec
d
Annuluus Radius (Units:: 2nil)
Do5
Cumuleliee Denaiyy
10"
50
70
10"
Send-0
10-00
0.0
0.5
1,0
60'
Orid Soor
0g/d;
Figure 4: Other details with DoG/DoS readouts affect grid emergence: Even with DoS/DoG cunslation-invariaan targets, grid solutions are sensitive: (a) Modestly varying the DoS width og can cause grids to disappear.
(b) Left: Grid scores of networks trained with equinorm-DoG versus DoS readouts [59] 47 600 shows DoS is critical for high grid scores. Right: Rate maps of top grid scoring units from equinorm- DoG networks. (c) necessary condition for grid emergence with DoG readouts is that the Fourier transform of the readout correlation matrix contain an annulus of radius >> 2T L (L the size of the enclosure in which the RNN is trained). The hyperparameter region where this condition is met is < order of magnitude (FT annulus radius computed analytically (orange) and through numerical construction of fields in finite environment (blue)). RNNs do not learn periodic responses outside this region, but most RNNs inside the region do not either, meaning the annular radius criterion is insufficient. DoS readouts are similar to one particular choice of DoG amplitudes, and only DoG amplitudes very close to the DoS point succeed in producing periodic responses. (d) Across a /ay DoG ratios, DoG networks generally score worse than filtered-and- theeeshshoeeeeeddddddd noise (blue) and worse than DoS (black). ag/ faz DoG region, and choosing ag and a magnitudes closely matching DoS, shown in (c) still showed only one ratio that did at least as well as DoS (App. E), but this result true for one out of three seeds; two "sibling" runs with otherwise identical settings produced poor gridness.
(e) More densely sweeping within the theoretically feasible
where Ar refers to the difference of indices between the neural pair linked by the weights. The evolution of activity can be decomposed into the growth and decay of Fourier components of the rate vector, which is fully determined by the Fourier transform of W, which is given by:
f(k) = 1 d(Ax)) ((Ar)) JR.
ogggg exp
-070 exp
Here og (ar)) denotes the strength and og (az) denotes the scale of excitation (inhibition). For linearized dynamics that approximate i(z) -r(x) + f(Ax) * r (i.c., g has been linearized), the solution will be periodic if the maxima of f(k), given by [k-|2 - log (eroo//ooof)
contains sufficient power and if A * 0. Specifically, the condition for pattern formation is f(k*) > 1 [0 In particular, the inhibitory surround contained in f(ax), with strength 0I, is key to pattern a 0, the maximum of the tranforrtr weights is at the origin (k*=0) = 0), corresponding to a DC (non-pain pattereed/noon periodicc activity state. Similarly, only particular choices of the ratio ag/a /ay work, Fig. 4c. In sum, Gaussian recurrent interaction cannot produce periodic patterns in the continuum limit (relatively large number of cells, large environment), as known from first principles models.
formation; if o
The theory of 55 contributes a connection between these first principles models and feedforward networks performing supervised least-squares regression ("function approximation framework") onto a target readout P. through the observation that gradient optimization of the MSE reconstruction loss || W r| can be approximated as i - + Σr. where Σx,x' E, (()) is the
7

---

a
Prat Runo with Low Modion Duscding tri (m100
b
1.0
1.22
1.0
0.97
0.96
0.8
Do5
Reid
Singi Scales
--umuttrba
0.6
N
0.87
Do5
Multiple Reeds
Multiple
0.
0.2
N
00
0.0
-0.5
0.0
0.5
1.0
0.71
0.71
0.67
0.66
NLBGW FOREL
Cord Score
Figure 5: Adding place cell-like heterogeneity to readouts prevents grid emergence. We selected DoS RNNS with the best hyperparameters for grid cell emergence (RNN UGRNN, ReLU, og = 0.12 em, s = 2.0, 3 seeds), then tested the effect of multiple fields per place cell (~ 1 (3.s( and multiple scales (receptive field width og Unif(0.06 1.0) m and surround scale ~ 1.25(..25, 4.5)). (a) Networks with multi-scale multi-field DoS readouts all obtain low position decoding error. (b) Multi-scall
DoS readouts do not learn grid cells. (c) Highest- scoring rate maps from multi-field multi-scale networks.
specified spatial correlation matrix of the target readouts and À regularization parameter. This dynamics now resembles that of first-principles grid cell models, provided the readout correlation matrix has the same form as the first-principles recurrent interaction matrix: W(x, xx' = Egg', upto scaling factors. If the readout target functions are set to be translationally invariant with DoG tuning curves, the readout correlation matrix is difference of multiple Gaussians (Appendix D). Fig 4c. By the linear stability analysis outlined above, it follows that DoG tuning can sometimes produce grids, but simple Gaussian tuning curves will not generically produce periodic patterns without additional assumptions. This is true whether we consider discretized real and Fourier space or take the continuum limit in both cases: Gaussian readouts generate roughly uniform (non- non-periodic) activation as the dominant state. Only under the further assumptions of discretization induced by a small spatial environment, higher-order non-linear effects, and orthogonality of hidden units, might Gaussians be theoretically predicted to produce periodic responses [59], and then the periods and shapes of the grids will depend on the environment properties. However, neither hexagonal nor square grids emerge significance level beyond filtered and thresholded noise in actual RNNs trained to PI with Gaussian and most DoG readouts.
6 Grid cells disappear with realistic readout population heterogeneity
Place cells, to which grid cells project, differ significantly from the highly idcalized single-scale, single-field translation-invaiian ANN readouts, even ignoring the particular center-surround shape for each field. Place cells have heterogeneous field widths, many with multiple fields [51] 19 and non-uniform spatial correlations. Place cells at similar dorsoventral locations can exhibit a range of field sizes, and single place cells themselves exhibit diversity of field widths [19]. This naturally leads to the question: Will readout targets with more place cell-like heterogeneous responses still produce grid cells? We found that networks trained with multiple-field, multiple-scale DoS readouts achieve position decoding error as low as single-field single-scale DoS encodings (Fig. 54), but do not learn grid cells (Fig. 5pc)). This finding is consistent with the strong requirement ANN models of laaslation-invarian
readout code for grid emergence [3] [59]. Translation invariance is a specific property of grid cells, but it is not likely true of place cells, which as a population over-represent
borders, landmarks and reward locations 53 49 69, 31 34 I8 16 73 27 5. These observations further demonstrate that ANN models essentially build the known structure of grid cells into their targets, rather than obtaining them from training on simple tasks with plausible readouts.
7 Why path-integrating ANNs might achieve high predictivity of MEC data
We conclude by introducing a puzzle, A recent NeurIPS spotlight 47 notes that networks trained on single-field single-scale DoS readout encodings explain variance in mouse MEC neural activity at nearly 100% of variance explained by other mice. In contrast, our results demonstrate that these networks learn few grid cells, produce unimodal grid period distributions inconsistent with biological
8

---

grid cells, and require readout encodings inconsistent with biological place cells to do so. How are these networks able to predict mouse MEC neural activity so well?
The analysis code is not open source, so we are unable to investigate this puzzle. However, we offer a conjecture with preliminary evidence. The analysis of 47 linearly regressed rate maps from one agent (mouse or network) onto rate maps from another mouse, and used Pearson correlation as measure of "neural predictivity." We conjecture that different wichiteturee-actiaaiie pairs achieve different neural predictivity scores because different pairs learn different intrinsic dimensionalities that then provide richer/poorer bases for linear regressions.
To explore our conjecture, we trained (47)'s ar- chitectures (RNN, LSTM, GRU, UGRNN, SRNN) with DoS readouts and ReLU activations (5 seeds per architecture-actiiatton pair). For each trained network, we extracted rate maps and computed a standard linear measure of dimensionality called par- ticipation ratio [42], then plotted each pair's ' average participation ratio against the published neural predic- tivity. We found networks with higher dimensional rate maps have higher neural predictivity (Fig. (
Noural Prodictivity (Pearson
Correin Correlation
0.80
RNN
0.75
UGRNN
SRNN
relu
10 Participation Rate Maps
Figure 6: Networks with higher (lower) di- mensional rate maps display higher (lower) "neural predictivity" of mouse MEC rate maps. Each dot an architecture-activvie pair; participation ratio averaged over 5 seeds. Neural predictivity from
We caution this correlation between dimensionality and neural predictivity is not (yet) strong evidence. However, we predicted that similar results would be found for other species and modalities, and during the subsequent NeurIPS review process, two independent research groups confirmed our predictions in macaque vision and human audition [66] correct, the conjecture raises the interesting research question of whether linear regression- bassd comparisons of ANNs with neural data might produce better matches to biology more because ANNs have higher dimensional representations than competitors, than because of any detailed similarity.
Discussion
For research that deep networks models of the brain, there is a fundamental obstacle to making the claim that given optimization prob- lem is what the brain is solving: If we know the responses of a significant fraction of units from biological networks performing a certain task, we cannot infer the loss function that the brain is optimizing since in principle, numerous dif- ferent loss functions can have the same/similar minima (Fig. 7 top). In other words, there is typically many-to-one mapping between loss functions and some point in model space where the losses have minimum. Some of the dif- ferent grid models from DL and first principles show that this possible [15] 3 59 9. Con- versely, given reasonable optimization prob- lem that we select based on an organism's eco- logical niche, cannot infer single solution (and thus build truly predictive single-cell tuning models), since there exist several minima to that optimization problem (Fig. 7 bottom). In other words, there is typically a one-to- one-to-many mapping from a loss function to its set of solutions.
C
C
Mace
Figure 7: Challenges in achieving the two tral claims of recent DL models of neuroscience: Top: Building model that replicates observed neural responses does not guarantee that the loss function used is the brain's 's objective, multiple objectives can share a solution. Bottom: Training the
network a plausible loss function correct loss function need not yield the solution the brain has selected because the loss function may have multiple minima, of which the brain selects one based on its constraints, while an ANN selects another, based on the optimization technique used.
To break this uniqueness problem and arrive at truly predictive models and a better understanding of the brain's optimization problems, we must understand the specific inductive biases and constraints present in the biological system are trying

---

to model. It is untenable to expect success without doing so. This is what we refer to as an informal neural 'No Free Lunch'.
Can we learn about brain circuits from DL models without considering biological inductive biases? Low-dimensional latent representations and dynamics that emerge as necessary for solving difficult problems are possibly sufficiently robust and abstract to be predictive of populations in neural circuit. For instance, any model solving the task of finding ripe apples in color photos, will create some abstracted representation of round red objects; this would also be a robust prediction for neural systems, but not unique to them. On the other hand, we should generally not expect detailed single neuron tuning correspondences without specific additional constraints or inductive biases: If a -iwdeeeeniioall latent representation necessary to solve a task, there are a multiplicity of ways to project it onto the activities of a large number of neurons. Which projection the brain selects depends on factors including energetics, neuron number, downstream uses, and the vagaries of evolutionary dynamics; the projections of ANNs depend on similar factors but specific to the ANN's loss and the vagaries of gradient descent learning. Consistent with this, models of the visual pathway 7 and circuits that solve latent inference tasks 663 55 67 exhibit population-Ievel representations of abstract variables necessary to solve the task. By contrast, DL-based grid cell models make fine-grained claims about single-neuron tuning, which should be surprising without the incorporation of significant additional constraints. Only in cases where task constraints completely overwhelm all settem--pccific constraints, might expect the natural emergence of alignment at the single-umit level.
Returning to grid cells, since they do not generically arise in networks trained to path integrate, path integration not a sufficiently constraining task. Theoretical work on grid cell representations [22 61 45) suggests additional critical features of the code: an exponentially large coding range and robustness/intrinsic error correction, both of which translate into the problem of packing and
maximally separating large set of coding states into compact space [61]. We hypothesize that the following key properties of the grid cell code may form a biologically relevant sufficient set for their emergence: 1) non-negative activity: 2) path integrating (PI) code that is translation
invariant [23 9 [2 3) exponential representational capacity (8] 61 [4]; 4) intrinsic correciong
capabilities 88 61]; and 5) uniformly distributed (whitened) information across cells. Several of these are general properties of neural codes, and could increase the ability of ANN models to make de novo
rather than post hoe predictions. For recent promising work, see (17).
First-prineto First-principles continuous attractor network (CAN) models of grid cells made several novel predic- tions subsequently confirmed in experiments [39] the invariance of cell-cell relationships acrossen- vironments and behavioral states (constituting, in the terminology of machine learning, "far out of
distribution" predictions) that define invariant toroidal attractor manifold 772 64 26 25 grid-like patterning the cortical sheet (29), and many others that remain to be tested. Deep learning-basse models should be held to similar standards.
In sum, ANN models of the brain that reproduce specific tuning curves should not center their claimed achievement on producing the curves if these are used as implicit or explicit parts of the training target (given the expressive power of deep networks, it is no revelation that training them to generate given tuning will in fact succeed), but rather should characterize the conditions under which the particular tuning does and does not emerge, to consider which inductive biases are critical, and to explicitly state what principles and de novo predictions can be extracted from the models.
9 Acknowledgements
This work was supported by the ONR, the NSF, the Simons Foundation through the SSCGB program, and the HHMI through the Faculty Scholars Program. MK was supported by the Math Works Science Fellowship.
10

---

References
[1] Emergence of dynamically reconfigurable hippocampal responses by learning to perform probabilistic spatial reasoning. biorxiv.
[2] High-performing neural network models of visual cortex benefit from high latent dimensionality, July 2022. Pages: 2022.07.13.499968 Section: New Results.
[3] Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel. Martin J. Chadwick, Thomas Degris, Joseph Modayil, Greg Wayne, Hubert Soyer, Fabio Viola, Brian Zhang, Ross Goroshin, Neil Rabinowitz, Razvan Pascanu, Charlie Beattie, Stig Petersen, Amir Sadik. Stephen Gaffney, Helen King, Koray Kavukeuoglu, Demis Hassabis, Raia Hadsell, and Dharshan Kumaran. Vector-based navigation using grid-like representations in artificial agents. Nature, 557(7705):429 443, May 2018. 8.
[4] Pouya Bashivan, Kohitij Kar, and James J. DiCarlo. Neural population control via deep image synthesis. Science, (6336 May 2019. Publisher: American Association for the Advancement of Science.
[5] Romain Bourboulou, Geoffrey Marti, François- Xavier Michon, Elissa El Feghaly, Morgane Nouguier, David Robbe, Julie Koenig, and Jerome Epsztein. Dynamic control of hippocampal spatial coding resolution by local visual cues. Elife, 8:e44487, 2019.
[6] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Merbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language Models are Few-Shot Learners. arXiv:2005.14165 [cs]. July 2020, arXiv: 2005. 14165.
[7] Yoram Burak and Ila Fiete. Do We Understand the Emergent Dynamics of Grid Cell Activ- ity? Journal of Neuroscience, 26(37):952-9954, September 2006. Publisher: Society for Neuroscience Section: Journal Club.
[8] Yoram Burak and lla R Fiete. Unpublished observations. 2008.
[9] Yoram Burak and lla R. Fiete. Accurate Path Integration in Continuous Attractor Network Models of Grid Cells. PLOS Computational Biology, 1(2)2210 February 2009. Publisher: Public Library of Science.
[10] Yoram Burak and Ila R Fiete. Accurate path integration in continuous attractor network models of grid cells. PLoS Comput Biol, 5(2) 1000291, Feb 2009.
[11] Malcolm G. Campbell, Samuel A. Ocko, Caitlin Mallory, Isabel I. C. Low, Surya Ganguli, and Lisa M. Giocomo. Principles governing the integration of landmark and self-motion cues in entorhinal cortical codes for navigation. Nature Neuroscience, 2196) August 2018. Number: 8 Publisher: Nature Publishing Group.
[12] Rishidev Chaudhuri, Berk Gerçek, Biraj Pandey, Adrien Peyrache, and Ila Fiete. The intrinsic attractor manifold and population dynamics of canonical cognitive circuit across waking and sleep. Nat Neurosci, 12(9):1512-1522 09 2019.
[13] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical Evaluation of Gated Recurrent Neural Networks Sequence Modeling. NIPS Workshop Deep Learning and Representation Learning, December 2014. Number: arXiv: 1412.3555 1XXi::141.33555
[cs].
[14] Jasmine Collins, Jascha Sohl-Dickstein, and David Sussillo. Capacity and Trainability in Recurrent Neural Networks. International Conference on Learning Representations, March 2017. Number:
arXiv:161 1.0999 [cs, stat].
11

---

[15] Christopher Cueva and Xue-Xin Wei. Emergence of grid-like representations by training recurrent neural networks to perform spatial localization. International Conference on Learning Representations, page 19, 2018.
[16] Nathan B Danielson, Jeffrey D Zaremba, Patrick Kaifosh, John Bowler, Max Ladow, and Attila Losonezy. Sublayer-specific coding dynamics during spatial navigation and learning in hippocampal area cal. Neuron, 91(3):652-66 (66)),62266,,,, 2016.
[17] William Dorrell, Peter Latham, Timothy EJ Behrens, and James CR Whittington. Actionable neural representations: Grid cells from minimal constraints. arXiv preprint arXiv:2209. 15563, 2022.
[18] David Dupret, Joseph neill, Barty Beeydell-Bouverie and Jozsef Csicsvari. The reorganization and reactivation of hippocampal maps predict spatial memory performance. Nature neuroscience, 13(8):955-1002 2010.
[19] Tamir Eliav, Shir Maimon, Johnatan Aljadeff, Misha Tsodyks, Gily Ginosar, Liora Las, and Nachum Ulanovsky. Multiscale representation of very large environments in the hippocampus of flying bats. Science, 372(6545),aabg4020, May 2021. Publisher: American Association for the Advancement of Science.
[20] Jeffrey L. Elman. Finding Structure in Time. Cognitive Science, 14(2):19-2211 1990. _eprint: http://wwwwalleae.oooooo/o/ooooa
402_11.
[21] Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, and Aleksander Madry. Implementation Matters in Deep Policy Gradients: A Case Study PPO and TRPO. 22Xi::005.122299 [cs, stat), May 2020. arXiv: 2005.12729
[22] Ila R Fiete, Yoram Burak, and Ted Brookings. What grid cells convey about rat location. J Neurosci, 28(27):6858-77 71, Jul 2008.
[23] Mark Fuhs and David Touretzky. spin glass model of path integration in rat medial entorhinal cortex. J Neurosci, 26(16):4266 4276, 2006.
[24] Marianne Fyhn, Torkel Hafting, Alessandro Treves. May-Britt Moser, and Edvard Moser. Hippocampal remapping and grid realignment entorhinal cortex. Nature, (44(6733)):10-11 2007.
[25] Richard J. Gardner, Erik Hermansen, Marius Pachitariu, Yoram Burak, Nils A. Baas, Ben- jamin A. Dunn, May-Britt Moser, and Edvard I. Moser. Toroidal topology of population activity in grid cells. Nature, 602(7895): 122-118, February 2022. Number: 7895 Publisher: Nature Publishing Group.
[26] Richard J. Gardner, Li Lu. Tanja Wernle, May-Britt Moser, and Edvard I. Moser. Correlation structure of grid cells is preserved during sleep. Nature Neuroscience, 22(():598-60 April 2019. Number: 4 Publisher: Nature Publishing Group.
[27] Jeffrey L. Gauthier and David W Tank. A dedicated population for reward coding in the hippocampus. Neuron,
2018.
[28] Joshua Glaser, Ari S. Benjamin, Raeed Chowdhury, Matthew G. Perich, Lee Miller, and Konrad P. Kording. Machine Learning for Neural Decoding. eNeuro, 7(4), July 2020. Publisher: Society for Neuroscience Section: Research Article: Methods/New Tools.
[29] Yi Gu, Sam Lewallen, Amina A Kinkhabwala, Cristina Domnisoru, Kijung Yoon, Jeffrey L Gauthier, ll R Fiete, and David W Tank. A map-like micro-organization of grid cells in the medial entorhinal cortex. Cell, 175(3):736-70.ee0) 10 2018.
[30] Torkel Hafting, Marianne Fyhn, Sturla Molden, May-Britt Moser, and Edvard I. Moser. Mi- crostructure spatial map in the entorhinal cortex. Nature, 436(7052):80188066 August 2005.
Number: 7052 Publisher: Nature Publishing Group
[31] Phil A Hetherington and Matthew L Shapiro. Hippocampal place fields are altered by the removal of single visual cues in a distance-dependent manner. Behavioral neuroscience, 111(1):20, 1997.
12

---

[32] Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. Lecture 6e be-RMMProp.
[33] Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural Computation. 18): November 1997.
[34] Stig A Hollup, Sturla Molden, James G Donnett, May-Britt Moser, and Edvard I Moser. Accumulation of hippocampal place fields at the goal location in an annular watermaze task. Journal of Neuroscience, 1165)1633-1164, 2001.
[35] Andrew Ilyas, Logan Engstrom, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, and Aleksander Madry. A Closer Look at Deep Policy Gradients. arXiv:1811.00 1.22533 [cs, stat], May 2020. arXiv: 1811.02553.
[36] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew Ballard, Andrew Cowie, Bernardino PoredesParedes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back. Stig Petersen, David Reiman, Ellen Clancy, Michal Ziclinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Pushmeet Kohli, and Demis Hassabis. Highly accurate protein structure pre- diction with AlphaFold. Nature, 596(7873):583 589, August 2021. Number: 7873 Publisher: Nature Publishing Group.
[37] I. Kanitscheider and I. R. Fiete. Training recurrent networks generate hypotheses about how the brain solves hard navigation problems. Advances in Neural Information Processing Systems (NeurIPS). 2017.
[38] Mikail Khona, Sarthak Chandra, and lla R. Fiete. From smooth cortical gradients to discrete modules: spontaneous and topologically robust emergence of modularity in grid cells. bioRxiv, page 20.1.10.28.46660
January 2022.
[39] Mikail Khona and Ila R Fiete. Attractor and integrator networks in the brain. preprint at https://arziv. org/abs/21 12.03978, 2021.
[40] Timothy D. Kim, Thomas Z. Luo, Jonathan W. Pillow, and Carlos D. Brody, Inferring Latent Dynamics Underlying Neural Population Activity via Neural Differential Equations. In Pro- ceedings of the 38th International Conference on Machine Learning, pages 5551-5566 PMLR, July 2021. ISSN: 2640-3498.
[41] Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. Interna- tional Conference on Learning Representations, January 2017. Number: arXiv:1412.6981 arXiv:1412.6988 [cs].
[42] Ashok Litwin-Kuman Kameron Decker Harris, Richard Axel, Haim Sompolinsky, and L. F. Abbott. Optimal Degrees of Synaptic Connectivity. Newron, 1533:1113-114..77 March 2017.
[43] Jesse A. Livezey, Kristofer E. Bouchard, and Edward F. Chang. Deep learning tool for neural data analysis: Speech classification and cross- cross-freyuncc coupling in human sensorimotor cortex. PLOS Computational Biology, 10070911 September 2019. Publisher: Public Library of Science.
[44] A. Mathis, A. Herz, and M. Stemmler. Optimal population codes for space: grid cells outperform place cells. Neural Comp., 24:2280-233 2012.
[45] Alexander Mathis, Andreas V. M. Herz, and Martin Stemmler. Optimal Population Codes for Space: Grid Cells Outperform Place Cells. Neural Computation, 249))2280-2317, September 2012.
[46] Alexander Mathis, Pranav Mamidanna, Kevin M. Cury, Taiga Abe, Venkatesh Murthy, Mackenzie Weygandt Mathis, and Matthias Bethge. DbeplabCut: markerless pose estímation of user-defined body parts with deep learning. Nature Neuroscience, 12(9):12811122 September 2018. Number: 9 Publisher: Nature Publishing Group.
13

---

[47] Aran Nayebi, Alexander Attinger. Malcolm Campbell, Kiah Hardcastle, Isabel Low, Caitlin S Mallory, Gabriel Mel, Ben Sorscher, Alex H Williams, Surya Ganguli, Lisa Giocomo, and Dan Yamins. Explaining heterogeneity in medial entorhinal cortex with task-driven neural networks. In Advances in Neural Information Processing Systems, volume 34, pages 12167-12179. Curran Associates, Inc., 2021.
[48] J. O'Keefe and J. Dostrovsky. The hippocampus spatial map: Preliminary evidence from unit activity in the freely-moving rat. Brain Research, 34:171-175, 1971. Place: Netherlands Publisher:
Elsevier Science.
[49] John O' Keefe and Dulcie Conway. Hippocampal place units the freely moving rat: why they fire where they fire. Experimental brain research, 31(4):573-590, 1978.
[50] Talmo D. Pereira, Diego E. Aldarondo, Lindsay Willmore, Mikhail Kislin, Samuel S.HH. Wang, Mala Murthy, and Joshua W. Shaevitz. Fast animal pose estimation using deep neural networks. Nature Methods, 16()):117-125, January 2019. Number: 1 Publisher: Nature Publishing Group.
[51] D. Rich, H.-P. Liaw, and A. K. Lee. Large environments reveal the statistical structure governing hippocampal representations. Science, 345(6198):814-817, August 2014.
[52] Muhammad Saif-ur Rehman, Robin Lienkämper, Yaroslav Parpaley, Jörg Wellmer, Charles Liu, Brian Lee, Spencer Kellis, Richard Andersen, Ioannis Iossifidis, Tobias Glasmachers, and Christian Klaes. SpikeDeeptector: deep-learning based method for detection of neural spiking activity. Journal of Neural Engineering, 16(5):056003, July 2019, Publisher: IOP Publishing.
[53] Masaaki Sato, Kotaro Mizuta, Tanvir Islam, Masako Kawano, Yukiko Sekine, Takashi Takekawa, Daniel Gomez-Dominguez, Alexander Schmidt, Fred Wolf, Karam Kim, al. Distinct mech- anisms of over-representation of landmarks and rewards in the hippocampus. Cell reports, 3271)4107864, 2020.
[54] Andrew Saxe, Stephanie Nelli, and Christopher Summerfield. If deep learning is the answer, what is the question? Nature Reviews Neuroscience, 22(1):55-67 67, January 2021. Number: 1 Publisher: Nature Publishing Group.
[55] Rylan Schaeffer, Mikail Khona, Leenoy Meshulam, Brain Laboratory International, and Ila Fiete. Reverse-engineering recurrent neural network solutions to hierarchical inference task for mice. Advances in Neural Information Processing Systems, 33:4584 4596, 2020.
[56] Vemund Sigmundson Schøyen, Markus Borud Pettersen, Konstantin Holzbausen, Anders Malthe-Sprensen, and Mikkel Elle Lepperød. Navigating multiple environments with emergent grid cell remapping. bioRxiv, 2022.
[57] David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driess- che. Julian Schrittwieser, Ioannis Antonoglou, Veda Pannecrshelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):484 489, January 2016. Number: 7587 Publisher: Nature Publishing Group.
[58] Trygve Solstad, Charlotte N. Boccara, Emilio Kropff, May-Britt Moser, and Edvard I. Moser: Representation of Geometric Borders in the Entorhinal Cortex. Science, 322(5909):1865 1868, December 2008. Publisher: American Association for the Advancement of Science
[59] Ben Sorscher, Gabriel C Mel, Surya Ganguli, and Samuel A Ocko. A unified theory for the origin of grid cells through the lens of pattern formation. Advances in Neural Information Processing Systems, page 18, 2019.
[60] Ben Sorscher, Gabriel C. Mel, Samuel A. Ocko, Lisa Giocomo, and Surya Ganguli. unified theory for the computational and mechanistic origins of grid cells. Technical report, bioRxiv, December 2020. Section: New Results Type: article.
[61] Sameet Sreenivasan and lla Fiete. Grid cells generate an analog error-coreccing code for singularly precise neural computation. Nat Neurosci, 14(10):1330-7, Sep 2011.
14

---

[62] Hanne Stensola, Tor Stensola, Trygve Solstad, Kristian Frøland. May-Britt Moser, and Edvard I. Moser. The entorhinal grid map discretized. Nature, 492(7427):72-78, December 2012. Number: 7427 Publisher: Nature Publishing Group.
[63] David Sussillo and Omri Barak. Opening the Black Box: Low-Dimensional Dynamics in Biggh-Dimensional Recurrent Neural Networks. Neural Computation, 25(3):626-66) 649, March 2013.
[64] S.G. Trettel, J.B. Trimper, E. Hwaun, LRR. Fiete, and L.L. Colgin. Grid cell co-activity pat- terns during sleep reflect spatial overlap of grid fields during active behaviors, Nat Neurosci, 22(4):609-617, 04 2019.
[65] George Tucker, Surya Bhupatiraju, Shixiang Gu, Richard E. Turner, Zoubin Ghahramani, and Sergey Levine. The Mirage of Action- DeiionDDependntt Baselines in Reinforcement Learning. 1802. [es, stat], November 2018. arXiv: 1802.10031.
[66] Greta Tuckute, Jenelle Feather, Dana Boebinger, and Josh McDermott. Many but not all deep neural network audio models capture brain responses and exhibit hierarchical region correspondence. bioRxiv, 2022.
[67] Jakob Voigts, Ingmar Kanitscheider, Nicholas J. Miller, Enrique H. Toloza, Jonathan P. Newman, Ila R. Fiete, and Mark T. Hamett. Spatial reasoning via recurrent neural dynamics in mouse retrosplenial cortex. biorxiv, April 2022.
[68] James C. R. Whittington, Timothy H. Muller, Shirley Mark, Guifen Chen, Caswell Barry, Neil Burgess, and Timothy E. J. Behrens. The Tolman- Eichhnbbum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation. Cell, 1249 1263.c23, November 2020.
[69] SI Wiener, CA Paul, and Fichenbaum. Spatial and behavioral correlates of hippocampal neuronal activity. Journal of Neuroscience, 98))273772263,, 1989.
[70] Dehong Xu, Ruiqi Gao, Wen-Hao Zhang, Xue-Xin Wei, and Ying Nian Wu. Conformal isometry of group representation in recurrent network of grid cells. arXiv preprint arXiv:2210.02664, 2022.
[71] Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan Solomon, Darren Seibert, and James DiCarlo. Rertornance-opimizzed hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences, 111(23):619-8624, 2014.
[72] K.J. Yoon, M.A. Buice, Barry, C.and Hayman, N. Burgess, and LRR. Fiete. Specific evidence of cow-dimensional continuous attractor dynamics in grid cells. Nat Neurosci, 1678)88077-84 Aug 2013.
[73] Jeffrey Zaremba, Anastasia Diamantopoulou, Nathan Danielson, Andres Grosmark, Patrick W Kaifosh, John C Bowler, Zhenrui Liao, Fraser T Sparks, Joseph A Gogos, and Attila Losonezy. Impaired hippocampal place cell dynamics in mouse model of the 22q 11. deletion. 2017.
Nature neuroscience,
15

---

10 Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [No] We do not feel our paper has potential negative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, cloud provider)? [No] We haven't yet had time collect this information, but we will add this information to the final version if accepted.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
16


Reviews:
[{"review_id": "y15aW50WFHi", "reviewer": "Reviewer_PZh9", "summary": "In this paper it is argued that the relevance of deep learning models for computational neuroscience should be carefully assessed by more explicitly stating the inductive biases needed to reproduce the target biological phenomena. Through extensive experiments, the authors show that recent deep learning simulations supposed to explain the emergence of grid cells in the hippocampal formation might have been driven by specific (and not always clearly justified) implementation choices, and use these findings to challenge the notion that deep learning is a panacea for building predictive models in neuroscience.", "questions": "Despite the results presented in this paper are interesting and provocative, I think that they do not undermine the key role that deep learning models are having for advancing computational (cognitive) neuroscience. The argument made by the authors should be better justified and contextualized in the broader field of computational cognitive neuroscience, not just on the specific case of entorhinal-hippocampal circuit: there are many successful stories of deep learning models that lead to important advances in neuroscience, and those should also be taken in consideration when drawing conclusions.", "limitations": "The authors did not explicitly mention any limitations of their research approach.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: I found this paper interesting and provocative. Related research is adequately referenced (actually, there might even be too many references and the authors might consider removing the less relevant ones). At the same time, the overall novelty of the work appears limited, since the basic argument is that neuroscientists exploring computational models are affected by strong confirmation biases and seek models that display the kind of emergent behavior they are looking for. Though important, such claim is nothing new in the modeling literature.\n\nQuality: The authors put lot of efforts to reproduce previously published models (through open access source code) and to carefully investigate the impact of hyperparameters in driving model behavior. Still, the hyperparameter sweeps were not exhaustive, and it seems that in some cases the level of detail required to reproduce the original simulations was not achieved. Though the latter issue is mostly due to the limited information provided in the original publications, it might still have affected the present findings. Moreover, it seems that in some cases the results and interpretations depend on the arbitrary choice of some threshold (e.g., grid scores in Fig. 2) over which there might not be an objective consensus in the literature.\n\nClarity: The paper is well-written and appropriately structured. The reference list should be double-checked and shortened (there are also some duplicates, e.g., [20] vs. [21]). Readability of the figures can be improved, for example by better organizing the spacing layout and by increasing font size.\n\nSignificance: This paper could stimulate further research work. However, in my opinion the fact that a result (e.g., emergence of lattice cells) depends on the careful choice of some model’s hyperparameters does not necessarily undermine the relevance of the model. Rather, it calls for further investigations to better establish how and why certain hyperparameters could assume such specific values. The work carried out by the authors is certainly useful (and remarkable): given that the replication crisis is extending from experimental sciences to modeling sciences, pointing out to issues with published models is a good way to better define their implicit assumptions and possible shortcomings. However, I am not particularly impressed by the finding that only certain combinations of hyperparameters lead to the simulation of relevant emergent phenomena. For example, I do not think that deep learning models should necessarily be robust to variations in the type of activation function: if only one kind of function “works”, then the modeler task should be to try explaining which biological features might justify the presence of such activation function. Overall, these findings/discussions seem more appropriate for a computational neuroscience conference / journal (and maybe one focused on hippocampus).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ixKkpjuxNn", "reviewer": "Reviewer_5qo3", "summary": "This work critically reanalyzes previously published deep neural network models of grid cell emergence. It shows that grid-cell-like representations arise only given overly specific implementations of the path integration task and might even depend on undocumented implementation choices. Hence, the training task alone does not predict the emergence of grid cells. The authors suggest that the overly specific occurrence of grid-cell-like representations in NNs trained on path integration reflects the posthoc nature of these modeling studies: Presumably, the implementations of these models were informally tweaked until they reproduced the existing biological phenomenology. The authors analyze the emergent grid-cell-like representations and provide some analytical results on why they arise. Finally, the authors argue that neural network-based modeling must rely more strongly on biological and theoretical constraints. Multiple training objectives may map to the same neural activity pattern. Conversely, the \"correct\" objective will not lead to the empirical neural activity pattern without the relevant biological constraints.", "questions": "* Why shouldn't we think about the specificity of grid-cell-like representation emergence as a feature rather than a bug of deep neural network modeling? (see above).\n* How was the grid score defined? I could not find the relevant equation.\n* Is there a repository with the code required for reproducing the study's experiments and results? Adhering to open research practices would increase the impact of this work (and my assigned NeurIPS score). It would also be fairer, given that this study was enabled by the open research practices of the criticized works.\n\ncomments:\n* Pie charts (figure 1) are an inefficient, outdated, and bias-prone visualization method. An empirical CDF or histogram of grid scores or a scatter plot of position error vs. grid score would convey much more information to the readers.", "limitations": "Limitations seem to be sufficiently addressed.", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This is a deep, quantitive critique of current connectionist approaches to explaining grid cells. It is systematic and well presented. \n\nHaving said that, I am left unconvinced by the claim of many-to-one mappings of training tasks and activity patterns (figure 8 top); the empirical results seem to demonstrate the opposite pattern - surprisingly high sensitivity of grid-cell emergence to the specific operationalization of the training objective. In principle, this result can be interpreted as a strength of the NN-based modeling approach - we can reject incorrect operationalization. This sensitivity contrasts with traditional neurobiological \"word models\" that always magically work since they do not commit to an implementation. Therefore, I believe it would be fairer and more constructive to frame the paper's results as a description of a considerable remaining modeling gap and avoid sweeping statements such as  \"...that one often gets neither\".", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "TrNsGKyryDD", "reviewer": "Reviewer_M9Ch", "summary": "This paper casts doubt on whether DNNs can serve as good mechanistic models to understand the brain. In particular, this paper revisits the existing literature on how task-related loss functions (e.g., path integration) of RNNs produce grid-like neural tuning, a key finding supporting the applications of DL models in neuroscience. The authors leveraged the published code in earlier work and performed large-scale hyperparameter sweeps across several dimensions, including network architectures, activation functions, optimizers, supervised targets, loss functions, and other miscellaneous aspects of model training. Surprisingly, almost all trained networks learn to perform path integration tasks very well but only a few exhibit grid-like codes. In particular, they found that networks whose neurons are assumed to have DoG receptive fields exhibit grid-like codes, but even under this regime grid-like cells only emerge for a small range of receptive fields parameter values. The authors also provide theoretical explanations from the perspective of Fourier analysis for why only a small range of parameter values of DoG tuning curve can lead to lattice cells. The authors also criticize for inconsistency between method descriptions in paper and true code implementations. Lastly, the authors investigate the intrinsic dimensionalities of RNN and how it could help predict neural activity.", "questions": "\n* Choosing appropriate hyperparameters is a fundamental aspect of machine learning and neuroscience research. For example, in the majority of instances, we say a model \"state-of-the-art\" only on the condition of certain hyper-parameters. Commonly, a model may no longer be the best for other hyper-parameter settings. In neuroscience, certain neural properties only hold for some task parameters (e.g., stimulus size, contrast). This is why reproducibility rises as an issue in neuroscience because readers typically have no idea of the boundary of appropriate hyper-parameters/experimental settings. In my opinion, hyper-parameter _per se_ is not an issue. The issue here is whether a paper is transparent for the hyper-parameters being chosen such that others can reproduce their results. Do you think a claim can be accepted only after it is confirmed on every possible hyper-parameters? I don't think so. I hope the authors could clarify this. \n* In section 9. The authors investigated the correlation between dimensionalities and neural prediction scores, and speculate that DL models outperform alternatives because they provide higher-dimensional bases. I don't quite get the logic. We can evaluate a model from many perspectives. One important aspect is its explanatory power for data. Higher-dimensional bases usually indicate richer representational power albeit possible suffering of overfitting. I don't think high-dimensional bases _per se_ is bad if it is well cross-validated. Also, this cannot be easily generalized to vision and language models. Vision DNN models typically have no temporal domain and they are not dynamic systems. \n* The authors seem to overly generalize their pessimism to other fields in neuroscience, e.g., vision and language. This is another form of exaggeration in the opposite direction. I do agree that this paper reports a disaster failure of RNN models in explaining grid-like codes. But critics should be reviewed case by case. In other words, unless similar analyses are done in vision or language models to illustrate their deficiency, we should not make such analogy to DNN in other fields.\n* One central claim in the Discussion part is universally true and applies to all computational models, not specific to DNNs. Given non-linearity in the majority of computational models, there always exists the possibility of multiple solutions for a given loss and multiple losses for a given solution. The question is the extent of such bad interpretability. For example, trained sparse coding models contain neurons whose receptive fields bear striking resemblance to receptive fields in V1. And people conclude that sparse coding may be one of the loss functions the visual system attempts to optimize. We can certainly find other loss functions giving rise to similar results. But the sparse coding studies are still informative and valuable. ", "limitations": "I find no specific limitations here.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Before going through this paper, I want to be transparent that I am familiar with DNN and RNN training but not an active researcher in the field of gird cells and spatial navigation. So I may misunderstand some aspects of this paper. I explicitly list the parts I don't understand here for meta-reviewers or area chairs to accurately evaluate this manuscript.\n\nI understand the majority of this paper except that:\n\n* The  Fourier analysis part for why some specific DoG tuning curve values can lead to grid-like codes in Section 6.\n* Related to the above point, the difference between DoG and DoS in Section 8.\n\n\n\n# Strengths\n\n* This paper is interesting and should be published. All researchers in the field should be aware of this. I appreciate the thorough analyses on where the trained networks do or do not produce grid-like codes. Although DNNs have recently been touted as good models in neuroscience, the field needs this sober consideration and careful examination of the validity of existing results. People should be more objective and critical to papers even published in prestigious journals.\n* The analyses are good. At least I cannot find any fundamental flaw in terms of analyses. \n* In Discussion, the authors also sketch out the possible ingredients that could promote existing RNNs to obtain grid-like codes, which is very constructive.\n\n\n# Weakness:\n\nMy major concern is that, although we should criticize the seemingly exaggerated claims in existing literature, we should also be careful and objective to the claims in this paper. I think some claims here are going to the opposite extreme. See my questions below.\n\nMinor\n* Ref. 20 and 21 seem identical", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "PVIZEzYGSGG", "reviewer": "Reviewer_B3Mb", "summary": "The theme of this paper is to demonstrate that directly using deep learning models as models of brain functions might lead to wrong or incomplete conclusions. By performing extensive hyper-parameter searches, authors show that the emergence of grid cell patterns by training a path integration RNN depends solely on particular design choices such as DoG firing fields rather than the path integration task itself. Such observations can hardly support conclusions made by previous papers that the emergence of grid cell patterns is due to performing the path integration task by an RNN. As a result, authors argue that neuroscientists should be more cautious to use deep learning models directly as models of brain function.", "questions": "1. Whether there are other examples in neuroscience that use deep learning but lead to misleading conclusions?\n2. Any new ideas to correctly and cautiously use deep learning models to interpret brain functions?\n3. Possible comparisons of expert-designed computational models (CANN) and deep learning models?", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ The considered question is important and remains an open and challenging one.\n+ The perspective adopted by this paper is interesting, namely to demonstrate fundamental limitations of previous approaches.\n\nWeakness:\n- This paper does not figure out any potential solutions to mitigate the observed mismatch between deep learning models and brain functions.\n- Many assumptions such as DoG firing fields or special regularizations are indeed clearly claimed in previous papers. So it seems natural that without these assumptions the grid cell patterns should disappear. \n- Only one example (grid cells) is used to show the no-free-lunch claim. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept