Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: ## Overview
This paper studies whether a recurrent neural network trained to solve a particular task (integration of angular velocity to generate head direction). Given trained recurrent networks that solve the task, the paper analyzes these using a number of different methods (visualizing tuning curves, perturbation experiments, and varying input statistics). Overall, I think this is a nice paper that shows the power of using artificial networks as a model system to answer neuroscientifically motivated questions. In particular, I found the perturbation analyses particularly illuminating. These kinds of experiments are only possible in these artificial networks, and can highlight/guide future biological experiments.

## Major comments
- "Units with minimal tuning to both variables are discarded from further analysis"  -- how many units end up being discarded? The reason I ask is because the number of discarded neurons affects this statement: "Therefore, units in the trained RNN could be mapped on to the biological head direction system both in terms of general functional architecture and detailed tuning properties.". If a large number of neurons are discarded, then the statement should be amended to say that "a fraction of units in the trained RNN" can be mapped onto the HD system. However, my understanding is that many biological neurons have tuning properties that are hard to classify, perhaps the discarded neurons could map on to these previously uncharacterized neurons?
- The paper demonstrates a number of qualitiative similarities between artificial and biological networks (e.g. both contain neurons tuned for HD or AV). It would be even more compelling if, wherever possible, these comparisons were made to be quantitative.
- "We conjecture that Ring units in the trained RNN serve to maintain a stable activity bump in the absence of inputs". I think a cool direct test of this idea would be to use the techniques in Barak & Sussillo 2013 (find fixed points of the recurrent network dynamics, and analyze the linearized system at those fixed points) to identify the ring attractor structure. In particular, numerical auto-differentiation can be used to automatically identify these points (c.f. the methods in https://arxiv.org/abs/1907.08549). Using these tools, can one find the ring attractor hidden in these networks?

## Minor comments/questions
- The layout in Fig 2a (and Fig 5c, 5f, and 5i) is a little misleading. If I understand correctly, the axes labels apply to each individual panel (which shows tuning for a particular neuron). However, since the labels extend across many panels, it looks as if the panels themselves are organized according to angular velocity and head direction, which doesn't make sense.
- Missing citation towards the end of pg. 3
- Consider setting the panels in Fig 5a-h to have the same axes limits, for easier comparison.
review_title: Official Blind Review #1

final_score: rating: 8: Accept
review_content: This paper examines head direction representations in a RNN. The RNN was trained to report current head direction using initial head direction and angular velocity information as inputs. The authors compare the representations in the RNN to the representations found in the head direction systems of mice and flies. They find that the representations and connectivity matrices of the RNN recapitulate many aspects of the real brains’ head direction systems. These include the presence of cells tuned only for head direction (“ring cells”) and cells with combined head direction and angular velocity tuning (“shift cells”). These cells tile the space in a functional torus structure, as revealed by t-SNE, which resembles the structures seen in real brains. Similarly, the connectivity profiles and functional role of these cell types matches what is observed in the brains. Altogether, these results demonstrate that RNNs optimized for this task can capture both the functional and anatomical aspects of the systems found in animals. This validates the use of RNNs for studying these systems. In my opinion, it also suggests that information about the structure of brains could inform strong priors for ANNs.

I think this is a great paper. It was a pleasure to read, the data was very clear, and the results very interesting. It should be accepted in my opinion. I have only a few minor notes for improvement.

- Per my last note in the summary, it would be nice if the authors made the paper a little bit more relevant for machine learners by discussing how this information could potentially inform novel neural network architectures for navigation.

- There’s a missing reference on page 3, second last paragraph.

- Figure 5 a,d, and g would be easier to interpret if they all had the same x-range as g (that way, you could see the narrower ranges and wouldn’t have to notice that the scales were different).

- Why have Figure S8 and Fig 3 separate? Aren’t they nearly identical? Fig S8 isn’t that much bigger, why not just make it Fig. 3?
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The authors train RNNs to integrate motion cues, to determine the head direction of a simulated animal. They then investigate the tuning properties of the units in the RNN, and the connectivity between units, and make comparisons to the corresponding systems in the fly and rodent.

Similarities are quite strong: the RNN learns to reproduce the ring attractor seen in the fly, where head direction (HD) cells excite other HD cells with similar direction preference and inhibit those with dissimilar preference. Shifter cells are also observed, that integrate angular velocity cues and "shift" the representation of head direction: these show an intuitive asymmetric connectivity profile. 

I can't quite decide what to make of this paper. One the one hand, the recapitulation of the biological circuit in the RNN is fairly compelling. On the other hand, other models (admittedly with hand-tuned connectivity) yield quite similar findings. So my overall opinion comes down to the question of: how big a deal is it that the RNN learned this connectivity, instead of it being designed by a modeller like Xiao-Jing Wang?

I have a few specific suggestions:

1) It would be useful as a comparison to duplicate the analysis of unit tuning, connectivity, etc., in untrained networks. Recent work (arXiv:1909.10116 [q-bio.NC]) shows that, even in untrained randomly-connected RNNs, some units have (by chance) tuning, and that tuning depends on the connectivity between units in a manner that could somewhat resemble the attractor models.

2) There were important details that I couldn't find. For example, how many of the RNN units receive the external inputs? Is it all of them? Is that realistic?

3) I question the approach (first paragraph of Sec. 3.1) of only analyzing the units  with strong HD and/or AV selectivity. Presumably the mouse shows some units with weak tuning, and investigating the connectivity of the weakly-tuned units in the RNN (e.g., their connectivity to each other, and to HD and AV units) could be informative.

One analysis to consider: ablate the nonselective units after training, and run the RNN with no retraining. Does the head direction tracking function remain intact or degrade? Based on some previous work with feedforward nets (arXiv:1803.06959 [stat.ML]), I'd guess that the function could be seriously degraded by removing nonselective units. That modelling result could make a strong prediction for experiments: inactivation of nonselective neurons (e.g., using archaerhodopsin), would impair function.


review_title: Official Blind Review #2

Meta-review: This paper studies properties that emerge in an RNN trained to report head direction, showing that several properties in natural neural circuits performing that function are detected. 
All reviewers agree that this is quite an interesting paper. While there are some reservations as to the value of letting a property of interest emerge as opposed to simply hand-coding it in, this approach is seen as powerful and valuable by many people, in that it suggests a higher plausibility that the emerging properties are actually useful when optimizing for that function -- a claim which hand-coding would not make possible. Reviewers have also provided valuable suggestions and requests for clarifications, and authors have responded by improving the presentation and providing more insights.
Overall, this is a solid contribution that will be of interest to the part of the ICLR audience that is interested in biological systems.
Decision: Accept (Spotlight)