Paper:
One of the most fundamental organizational principles of the brain is the separation of excitatory (E) and inhibitory (I) neurons. In addition to their opposing effects on post-synaptic neurons, E and I cells tend to differ in their selectivity and connectivity. Although many such differences have been characterized experimentally, it is not clear why they exist in the first place. We studied this question in deep networks equipped with E and I cells. We found that salient distinctions between E and I neurons emerge across various deep convolutional recurrent networks trained to perform standard object classification tasks, and explored the necessary conditions for the networks to develop distinct selectivity and connectivity across cell types. We found that neurons that project to higher-order areas will have greater stimulus selectivity, regardless of whether they are excitatory or not. Sparser connectivity is required for higher selectivity, but only when the recurrent connections are excitatory. These findings demonstrate that the functional and structural differences observed across E and I neurons are not independent, and can be explained using a smaller number of factors.
Deep neural networks have become powerful tools to model the brain (Yamins & DiCarlo, 2016;Kriegeskorte, 2015). They have been used to successfully model various aspects of the sensory (Yamins et al., 2014;Kell et al., 2018), cognitive (Mante et al., 2013;Wang et al., 2018;Yang et al., 2019), and motor system (Sussillo et al., 2015). Deep networks have been particularly effective at capturing neural representations in higher-order visual areas that remain challenging to model with other methods (Yamins et al., 2014;Khaligh-Razavi & Kriegeskorte, 2014).
Demonstrating that these models can mimic the brain in some respects is only a starting point in the process of advancing neuroscience with deep learning. Once some relationship between artificial and biological networks is established, the model can be dissected to better understand how certain biological computations are mechanically implemented (Sussillo & Barak, 2013). Although notoriously difficult to interpret, deep networks are still much more accessible than the brain itself. This is how deep networks can help address the "how" question. On top of that, deep networks can be used in a normative approach to answer the "why" question. Suppose we found that a certain architecture, objective function (dataset), and training algorithm could together lead to a neural network matching some features of the brain. Then we can ask why the brain evolved such features by testing which element of the architecture, dataset, and training is essential for such features to evolve in the artificial networks. For example, Lindsey et al. (2019) showed early layers of a convolutional neural network can recapitulate the center-surround receptive field observed in retina, but only when followed by an information bottleneck similar to the one that exists from retina to cortex. This finding demonstrates how realistic biological constraints can be used to explain the emergence of known properties of the brain.
Despite the many successes of applying deep networks to model the brain, standard architectures deviate from the brain in many important ways. Recently, neural networks that incorporate fundamental structures of the brain are becoming increasingly common (Kar et al., 2019;Miconi et al., 2018;Song et al., 2016). One fundamental structural principle of the brain is the abundance of recurrent connections within any cortical area. Cortical neurons receive a substantial proportion of their inputs from other neurons in the same area (Harris & Shepherd, 2015). Convolutional recurrent networks trained on object classification tasks can perform comparably to purely feedforward networks with similar numbers of parameters, while being able to better explain temporal responses in higher visual areas (Nayebi et al., 2018;Kar et al., 2019).
One of the most fundamental organizational principles of the cortex is the separation of excitatory (E) and inhibitory (I) neurons (Dale, 1935;Eccles et al., 1954). Dale's law states that each neuron expresses a fixed set of neurotransmitters, which results in either excitatory or inhibitory downstream effects. In addition to the difference in their immediate impacts on post-synaptic neurons, E and I neurons differ in several other important ways. There are several times (4-10x) more E neurons than I neurons (Hendry et al., 1987). Neurons that project to other areas, the so-called "principal neurons", are all excitatory in the cortex (Bear et al., 2007). In the mammalian sensory cortex, E neurons are overall more selective to stimuli than I neurons in the same area as extensively reported in mice (Kerlin et al., 2010;Znamenskiy et al., 2018) and to a lesser extent in other species (Wilson et al., 2017). Finally, E neurons are more sparsely connected with each other, with a connection probability of around 10% (Harris & Shepherd, 2015), compared to I neurons, which target almost all neighboring E neurons (Pfeffer et al., 2013).
Although many of these differences across E and I neurons are well-known and routinely incorporated in computational models (Somers et al., 1995;McLaughlin et al., 2000), it remains unclear whether they can emerge in deep neural networks from training. It is also unknown whether these differences are all independent properties, each contributing to the computation, or if some differences are natural results of the others. To answer these questions, we first trained many variants of deep convolutional recurrent neural networks on classical object classification tasks. Across all variants, three structural principles are built in: Dale's law, the abundance of excitatory neurons, and the role of excitatory units as principal neurons. However, other observed properties of biological E-I circuits are not hardwired, and therefore could only evolve under the pressure of performing the task. We found that functional and structural differences across E and I neurons emerged through training. The development of these characteristics allows us to address the "why" question by removing the built-in differences across E and I neurons and monitoring whether specific functional and structural properties still emerge. Our anonymized code is available at https://anonymous.4open.science/repository/d0ae905f-4171-42b0-94b5-abf03d6414aa.

Review:
[Update after rebuttal period]
I would like to thank the authors for the detailed response and for addressing the concerns on such a tight schedule.
Overall my two concerns appear to have been right: 1. The object classification task is not really relevant to elicit the observed behavior and 2. Inhibitory neurons are not essential (at least when training with batch norm). 
Do the conclusions of the paper about understanding of E/I cells in the brain still hold? I am not really sure. I would like to urge the authors to carefully assess their conclusions again in light of the new evidence and perhaps rephrase abstract, intro and discussion where necessary.
Having said that, I still like the paper and its approach. I think it provides valuable insights to the field (e.g. neuro-inspired architecture design, concerns regarding training objectives, etc.) and important steps in the right direction and could be a valuable contribution even if the original claims are not entirely substantiated.
[Original review]
This paper seeks to understand why excitatory neurons in cortex are more stimulus selective than inhibitory cells and why excitatory cells are more sparsely connected to each other. The authors train a recurrent neural network model with a number of biological constraints on CIFAR-10. These constraints include Dale's law, an unequal number of E vs. I cells and only excitatory connections between layers (areas) of the network. I would summarize the main findings as follows:
- Excitatory principal neurons are more selective and more sparsely connected to each other than local inhibitory neurons, consistent with biology
- Stimulus selectivity and performance depend only on the number of excitatory cells, but neither on the E/I ratio nor the number of I cells
- High selectivity appears to be a general property of principal cells, but does not depend on the sign of local connections, while sparse connectivity is mainly linked to the excitatory local connections
Strengths:
+ Architectural decisions are well motivated from a neuroscientific perspective
+ Provides a hypothesis why certain connectivity and selectivity patterns emerge in the brain by incorporating biological knowledge into neural network models trained on a specific task
+ Well written and clear, logic development of the arguments
Weaknesses:
- Unclear whether classification task is necessary to elicit the authors' observations
- Interneurons seem unnecessary, raising concerns about relevance of results
- Also trains on ImageNet, but only some results are shown; most from CIFAR
Overall I like the paper from the perspective of a neuroscientist, as it provides a kind of normative account of why things in the brain might look the way they do. My enthusiasm is somewhat limited, however, because I am not convinced the classification task is actually what drives the authors' observations. I am willing to increase my score and argue more strongly for the paper if the authors can address this concern, detailed in the following:
A few observations lead me to believe the classification task the networks are trained may not be important to elicit the observed behavior. 
1.) The emergent properties in stimulus selectivity show up almost immediately after training starts according to Fig. 5A+B. Performance, on the other hand, takes a few more epochs to pick up according to Fig. 10. 
2.) The connectivity preferences that emerge in Figure 5C appear at already 50 epochs where CIFAR-10 performance is at 50%. To put that into perspective, a linear SVM already achieves 40% on this task, and a one-layer multilayer perceptron with only 100 hidden neurons reaches that performance. 
3.) The results of Fig. 11 suggest that the number of inhibitory channels is not important. Did the authors try a stripped down version with only excitatory cells?
I suggest the authors train on CIFAR-10 with shuffled labels on the training set [Zhang et al. 2016; https://arxiv.org/abs/1611.03530]. Although performance on the test set would be at chance (no object recognition capabilities), it would be interesting to see whether the selectivity and connectivity properties still emerge.
Finally, (related to 3.) I wonder whether it makes sense to draw conclusions about differences between E and I cells from a model trained on a task where the number of inhibitory cells seems irrelevant. Wouldn't one need to have at least both types of neurons to be required for the task in order to draw such conclusions?
Minor Comments:
- I sometimes found the nomenclature of the multiple models you tried a bit confusing and hard to follow — especially in parts of the Appendix
- The type of operations (convolution, element-wise) in equation 1, together with the meaning of nonlinearities like \sigma_c are only defined one page after, making that section a bit hard to follow.