Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Although the cortex can be approximated as a 2-D sheet, different regions of the sheet have been identified as being connected together hierarchically, such as V1 > V2 > V4 ...  The authors develop a stochastically spiking sheet model with lateral connectivity and simple rewiring rules that forms a biologically analogous hierarchical structure--that are mimicked by deep ANNs--in an unsupervised manner.
strengths_and_weaknesses: **Originality:**

 I am unaware of other work that describes how networks of neurons might form hierarchical structures similar to those in the brain from simple 2-D non-hierarchical initial conditions.

**Quality:**

The project has a clearly defined motivating question, methodology to address the question, and analyses that help to address the research question. However, no statistical analyses are done.  Fig 5 in particular appears to only illustrate single runs.  In 5c especially, it appears that the SONS patch might result in a larger wiring cost than the SOM patch.  What happens if these are run longer?  These graphs should be based off of multiple runs of the network, and standard deviations or errors should indicate the range of metrics that result.  Moreover, statistical analyses should be implemented to reveal, after the networks have converged, what resultant differences among the networks are statistically significant.  Minor note: there are a few places, such as L262, where the forward quote is backwards (presumably because of Latex's conventions). Please see the question section as well

**Clarity:**
The authors describe their approach and results clearly. Note: Fig. 4 is difficult to view, in part because of the size, in part because of the pallette chosen (I am red-green color blind, and find these particular red, green choices to be difficult to discriminate).  Additionally, it would be helpful to provide zoomed-in images of regions of interest that are pointed out in the caption and text.  For Fig 7a, please incorporate a colorbar legend.

**Significance:**

At the interface of deep learning and neuroscience, different cortical regions, such as V1/V2/V4/etc., are generally treated as layers in deep network models of such systems as the visual system with increasing amounts of success.  The authors here have asked an important question as to how such hierarchies arise in cortex, given that they structurally can be modeled as lying on a 2-D sheet, a question with interesting implications with respect to the design feedforward ANNs, as the authors point out.  Their analyses seem to provide a first step towards answering this question.  However, the authors to not address the known anatomical connectivity (feedback to apical L1 dendrites, feedforward to basal L4 dendrites) (Q6, below).  The significance of the present work would be made more apparent if the authors address why such known connectivity is insufficient to explain the segregation of cortical regions, and how their model is or might be made to be consistent with this known cortical microcircuit anatomical structure.

questions: 1- Eq 1: How robust are the finding to different a values (not equal to 5)?

2 - Does this require "lateral inhibition", i.e., longer inhibition than excitation, as described in the model?  Such connectivity is not supported by the evidence, and so if it does, this assumption should be listed as a limitation of the current model

3 - L185 describes a hard constraint on the incoming and outgoing connection numbers.  Can this requirement be softened to a more bioplausible one?  E.g., perhaps a range of connections would suffice, and be more realistic

4 - L203: Similarly, might this constraint be softened?

5 - Fig 7b: it appears that around 7-9 layers have formed; is this a function of the distance connectivity of the network + the number of inputs?  If so--especially the latter--the authors should address this aspect explicitly, as an input-dependent number of layers (/cortical regions) would amount to a testable prediction of their model in cortex

6 - How does this model interface with the known anatomical cortical laminae connectivities?  E.g., higher regions such as V4 are known to feed back to lower regions at the L1 apical dendrites that connect to L2/3 and L5 neurons.  Similarly, V1 connects to V2 and higher at L4 basal dendrites. Is such anatomical selectivity not enough to account for the regional distinction that arises in cortex?

limitations: Yes, but some limitations (see Strengths and Weaknesses and Questions above) may not have been addressed yet
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper is based on an original idea that consists in considering a neural network organized on a simple neural layer, and to try to obtain the emergence of a hierarchical organization in this layer like the one observed in classical networks like "LeNet". The paper is properly organized and describes the model from the transformation of an image into a spike pattern  then to the stochastic neuron model, a learning rule of type STDP and the construction of lateral interactions both excitatory and inhibitory. The method is then described on a model with a limited number of spiking neurons, and then shows results for the emergence of a topographic map. This model is compared with a classical self-organized map and shows superior results in this model. This model finally concludes on the emergence of a hierarchy in the map, which is illustrated in figure 7 by the propagation of spikes on the neural map.



strengths_and_weaknesses: 
The main strength of this paper is the originality of the idea of organizing the whole neural network in the same way as the organization of the cortex along a surface. However, this paper is relatively limited by the small size of the network used and also by the simplicity of the task that is performed. The emergence as announced in figure 7 seems too preliminary to be convincing. One can also wonder what is the usefulness of using a spiking neural network in this framework, especially since neural models are based on stochastic firing patterns that potentially destroy the temporal structure of the sensory input.

questions: 
I have several questions in relation to this paper. 
First, have you looked at different settings of the topography constraints in the neural layer to see the emergence of different structures? There must be an optimal trade-off for a given task in order to achieve a hierarchy. 
Second, in section 2.4 you use a synaptic scaling rule that is similar to a homeostatic rule. Have you looked to see if you actually get neuron discharge frequencies that stabilize in the neuronal layer?
Third, it would be interesting to study the emergence of waves of activity in your neuronal layer. This kind of pattern in spontaneous activity seems to be essential in the emergence of the structure of a hierarchical type network and could be an interesting extension to your model

limitations: 
The authors did not specify the limitations and potentially negative societal impacts of this paper.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper presents an unsupervised method that allows the formation of local receptive fields and hierarchical layers in an initially unstructured pool of spiking neurons.
strengths_and_weaknesses: The method is novel and is based on purely local learning rules with lateral inhibition/excitation. This makes it especially valuable for those interested in biologically plausible approaches. The presented method performs similar to self-organizing map but shows better scaling to larger populations of neurons. This, together with the biological plausibility of the method, makes the presented work a strong contribution.

The paper is well positioned using relevant benchmark problems and comparisons to alternatives. It would however be nice to see an additional experiment other than MNSIT.

The review of previous work is very strong. Throughout the paper the authors show a strong familiarity with the relevant literature. This is a strong foundation that makes everything else very convincing. Experimental work is carefully done.

Additional comments:
-L109: "we fist we have"
questions: None
limitations: The authors adequately addressed the limitations of their work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes an interesting model to form visual cortex-like hierarchical layer structures in a self-organization manner. Different from traditional neural networks that ignore the placement of neurons, this study accounts for the location of the neurons in learning with the Self-Organizing Neural Sheet (SONS) model, inspired by the mechanisms of LGN cells (Figures 2 and 3). Experiments with MNIST dataset demonstrate that the SONS model can learn visual cortex-like topographic maps (Figure 4) at a lower cost (wiring cost, Figure 5) compared with SOMs. And the learned structure presents clustered higher-level concepts (Figure 7).
strengths_and_weaknesses: Strength:

The topic is interesting and the idea of taking the placement of neurons into account is of significance. Especially, results demonstrate that the model not only learns visual cortex-like topographic maps but also the layer-wise patterns from low-level patches to the concept of objects is valuable and inspiring. This type of thinking is beneficial for novel network design for a wide area.

Weakness:

The MINIST dataset is too simple to reflect rich information in the network learning process. More image datasets with natural pictures should be evaluated to better support the claims.

questions: 1.	As shown in Figure 4, using spiking SOM can learn similar patterns, only that SONS has a lower wiring cost. Please explain why the wiring cost is a proper metric for evaluation. And what is the advantage of the proposed SONS model? 
2.	In Figure 7b, what does the lower level (near to the input) features like? 

limitations: N/A
review_title: null

Meta-review: metareview: Novel and sound contribution.
confidence: Certain
Decision: Accept