Venue: ICLR 2018 conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The authors train an RNN to perform deduced reckoning (ded reckoning) for spatial navigation, and then study the responses of the model neurons in the RNN. They find many properties reminiscent of neurons in the mammalian entorhinal cortex (EC): grid cells, border cells, etc. When regularization of the network is not used during training, the trained RNNs no longer resemble the EC. This suggests that those constraints (lower overall connectivity strengths, and lower metabolic costs) might play a role in the EC's navigation function. 

The paper is overall quite interesting and the study is pretty thorough: no major cons come to mind. Some suggestions / criticisms are given below.

1) The findings seem conceptually similar to the older sparse coding ideas from the visual cortex. That connection might be worth discussing because removing the regularizing (i.e., metabolic cost) constraint from your RNNS makes them learn representations that differ from the ones seen in EC. The sparse coding models see something similar: without sparsity constraints, the image representations do not resemble those seen in V1, but with sparsity, the learned representations match V1 quite well. That the same observation is made in such disparate brain areas (V1, EC) suggests that sparsity / efficiency might be quite universal constraints on the neural code.

2) The finding that regularizing the RNN makes it more closely match the neural code is also foreshadowed somewhat by the 2015 Nature Neuro paper by Susillo et al. That could be worthy of some (brief) discussion.

Sussillo, D., Churchland, M. M., Kaufman, M. T., & Shenoy, K. V. (2015). A neural network that finds a naturalistic solution for the production of muscle activity. Nature neuroscience, 18(7), 1025-1033.

3) Why the different initializations for the recurrent weights for the hexagonal vs other environments? I'm guessing it's because the RNNs don't "work" in all environments with the same initialization (i.e., they either don't look like EC, or they don't obtain small errors in the navigation task). That seems important to explain more thoroughly than is done in the current text.

4) What happens with ongoing training? Animals presumably continue to learn throughout their lives. With on-going (continous) training, do the RNN neurons' spatial tuning remain stable, or do they continue to "drift" (so that border cells turn into grid cells turn into irregular cells, or some such)? That result could make some predictions for experiment, that would be testable with chronic methods (like Ca2+ imaging) that can record from the same neurons over multiple experimental sessions.

5) It would be nice to more quantitatively map out the relation between speed tuning, direction tuning, and spatial tuning (illustrated in Fig. 3). Specifically, I would quantify the cells' direction tuning using the circular variance methods that people use for studying retinal direction selective neurons. And I would quantify speed tuning via something like the slope of the firing rate vs speed curves. And quantify spatial tuning somehow (a natural method would be to use the sparsity measures sometimes applied to neural data to quantify how selective the spatial profile is to one or a few specific locations). Then make scatter plots of these quantities against each other. Basically, I'd love to see the trends for how these types of tuning relate to each other over the whole populations: those trends could then be tested against experimental data (possibly in a future study).
review_title: Interesting paper, and quite thorough work

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: Congratulations on a very interesting and clear paper.  While ICLR is not focused on neuroscientific studies, this paper clearly belongs here as it shows what representations develop in recurrent networks that are trained on spatial navigation. Interestingly, these include representations that have been observed in mammals and that have attracted considerable attention, even honored with a Nobel prize. 

I found it is very interesting that the emergence of these representations was contingent on some regularization constraint. This seems similar to the visual domain where edge detectors emerge easily when trained on natural images with sparseness constraints as in Olshausen&Field and later reproduced with many other models that incorporate sparseness constraints. 

I do have some questions about the training itself. The paper mentions a metabolic cost that is not specified in the paper. This should be added. 

My biggest concern is about Figure 6a. I am puzzled why is the error is coming down before the boundary interaction? Even more puzzling, why does this error go up again for the blue curve (no interaction)? Shouldn’t at least this curve be smooth?

review_title: Great paper on emergence of representations as observed in mammals  

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper aims at better understanding the functional role of grid cells found in the entorhinal cortex by training an RNN to perform a navigation task.

On the positive side: 

This is the first paper to my knowledge that has shown that grid cells arise as a product of a navigation task demand. I enjoyed reading the paper which is in general clearly written. I have a few, mostly cosmetic, complaints but this can easily be addressed in a revision.

On the negative side: 

The manuscript is not written in a way that is suitable for the target ICLR audience which will include, for the most part, readers that are not expert on the entorhinal cortex and/or spatial navigation. 

First, the contributions need to be more clearly spelled out. In particular, the authors tend to take shortcuts for some of their statements. For instance, in the introduction, it is stated that previous attractor network type of models (which are also recurrent networks) “[...] require hand-crafted and fined tuned connectivity patterns, and the evidence of such specific 2D connectivity patterns has been largely absent.” This statement is problematic for two reasons: 

(i) It is rather standard in the field of computational neuroscience to start from reasonable assumptions regarding patterns of neural connectivity then proceed to show that the resulting network behaves in a sensible way and reproduces neuroscience data. This is not to say that demonstrating that these patterns can arise as a byproduct is not important, on the contrary. These are just two complementary lines of work. In the same vein, it would be silly to dismiss the present work simply because it lacks spikes. 

(ii) the authors do not seem to address one of the main criticisms they make about previous work and in particular "[a lack of evidence] of such specific 2D connectivity patterns". My understanding is that one of the main assumptions made in previous work is that of a center-surround pattern of lateral connectivity. I would argue that there is a lot of evidence for local inhibitory connection in the cortex. Somewhat related to this point, it would be insightful to show the pattern of local connections learned in the RNN to see how it differs from the aforementioned pattern of connectivity.

Second, the navigation task used needs to be better justified. Why training a network to predict 2D spatial location from velocity inputs? Why is this a reasonable starting point to study the emergence of grid cells? It might be obvious to the authors but it will not be to the ICLR audience. Dead-reckoning (i.e., spatial localization from velocity inputs) is of critical ecological relevance for many animals. This needs to be spelled out and a reference needs to be added.  As a side note, I would have expected the authors to use actual behavioral data but instead, the network is trained using artificial trajectories based on "modified Brownian motion”. This seems like an important assumption of the manuscript but the issue is brushed off and not discussed. Why is this a reasonable assumption to make? Is there any reference demonstrating that rodent locomotory behavior in a 2D arena is random?

Figure 4 seems kind of strange. I do not understand how the “representative units” are selected and where the “late” selectivity on the far right side in panel a arises if not from “early” units that would have to travel “far” from the left side… Apologies if I am missing something obvious.

I found the study of the effect of regularization to be potentially the most informative for neuroscience but it is only superficially treated. It would have been nice to see a more systematic treatment of the specifics of the regularization needed to get grid cells. 
review_title: An interesting paper

Meta-review: This work shows how activation patterns of units reminiscent of grid and border cells emerge in RNNs trained on navigation tasks. While the ICLR audience is not mainly focused on neuroscience, the findings of the paper are quite intriguing, and grid cells are sufficiently well-known and "mainstream" that this may interest many people.
Decision: Accept (Poster)