Emergent Orientation Maps —— Mechanisms, Coding Efficiency and Robustness

Authors: Haixin Zhong, Haoyu Wang, Wei P Dai, Yuchao Huang, Mingyi Huang, Rubin Wang, Anna Wang Roe, yuguo yu
Primary area: applications to neuroscience & cognitive science
Keywords: Vision, Energy Efficient Coding, Neural Network, Sensory Coding, Spiking Mechanisms

Abstract:
Extensive experimental studies have shown that in lower mammals, neuronal orientation preference in the primary visual cortex is organized in disordered "salt-and-pepper" organizations. In contrast, higher-order mammals display a continuous variation in orientation preference, forming pinwheel-like structures. Despite these observations, the spiking mechanisms underlying the emergence of these distinct topological structures and their functional roles in visual processing remain poorly understood. To address this, we developed a self-evolving spiking neural network model with Hebbian plasticity, trained using physiological parameters characteristic of rodents, cats, and primates, including retinotopy, neuronal morphology, and connectivity patterns. Our results identify critical factors, such as the degree of input visual field overlap, neuronal connection range, and the balance between localized connectivity and long-range competition, that determine the emergence of either salt-and-pepper or pinwheel-like topologies. Furthermore, we demonstrate that pinwheel structures exhibit lower wiring costs and enhanced sparse coding capabilities compared to salt-and-pepper organizations. They also maintain greater coding robustness against noise in naturalistic visual stimuli. These findings suggest that such topological structures confer significant computational advantages in visual processing and highlight their potential application in the design of brain-inspired deep learning networks and algorithms.

Decision: accept (spotlight)
Reviewer scores: 6, 8, 8

Reviews and discussion:
<thread>
creation date:
1730276475366

thread_type:
official_review

<thread_object>
creator: reviewer_gmwp

<content>
summary:
Authors propose a comprehensive model of the primary visual cortex encompassing anatomical factors, such as natural image inputs, precise and realistic representation of neuronal responses, as well as spiking mechanisms in order to understand the emergence of pinwheel structures. They successfully show that the model can reproduce the emergence of different orientation maps in the visual cortex and that it is compatible with the maps of different species as a function of different factors such a neuron density or RF overlap.

strengths:
The paper is clearly written and presents a novel approach to analyzing the emergence of orientation maps in the visual cortex. The proposed model is well-motivated and supported by mathematical formalization. The authors provide a detailed explanation of the model and its components, which enhances the understanding of the proposed method.

weaknesses:
Stating in the abstract that the subject in "largely unexplored" is a strong claim that needs to be supported by a more extensive literature review. The paper could benefit from a more in-depth discussion of related work to provide context for the proposed approach. Your extensive bibliography provides already a good illustration of the number of works on the subject, and more for instance on the emergence of orientation maps in the visual cortex from haphazard wiring, or from the emergence of both types of maps using pooling mechanisms in a sparse deep predictive coding network, could be useful. More generally, it would be useful to highlight the difference of your work with [Stevens et al., 2013].

questions:
It has been proven multiple times that the anatomy of V1 is efficient for processing visual information. How do you think your model could be used to understand the emergence of orientation maps in the visual cortex? Is the tiling of receptive fields a possible factor ? (see eg https://doi.org/10.1016/j.neuron.2016.07.015 )

The model uses 1/ static images, 2/ images represented on a Cartesian grid. Extensions using dynamic images or images represented on a polar grid could be interesting. The evaluation of the model is limited to the comparison of the orientation maps with experimental data. A more quantitative evaluation of the model's performance on other tasks could provide a more comprehensive assessment of its capabilities.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730307615437

thread_type:
official_review

<thread_object>
creator: reviewer_zfyc

<content>
summary:
Authors study functional organization of the primary visual cortex across animal species. Depending on the species, primary visual cortex is organized in a continuum going between disordered salt-and-pepper and ordered pinwheel structures. Authors model these functional organizations with an E-I spiking neural network that is equipped with two types of Hebbian-like plasticity rules. An important finding is that  the degree of synaptic organization depends crucially on the overlap of feedforward inputs incoming to the network. They also find that once learned, the pinwheel organisation is stable and promotes efficient processing of visual information.

strengths:
Authors presented a rather convincing paper with a clear and timely scope. The paper is technically correct and rigorous in methods. It brings potentially important and new insights into why biological neural networks organise in a continuum of salt-and-pepper and pinwheel structures.

weaknesses:
Major weakness of this paper are the presentation, the lack of justification and elaboration on modelling and parameter choices, and insufficient comment on results. While the presentation is rather transparent, the clarity of the text can be substantially improved. Also, the text contains a number of typos.

1) I recommend to omit all abbreviations besides the very common ones, such as  V1, E-I and potentially FF. The acronym of the network, SESNN, is also an exception and is useful to have. All other acronyms seem counterproductive. While authors have compiled a list of acronyms in the Appendix A.7, frequent use of diverse acronyms throughout the text impedes readers that are less familiar with the topic of self-organising maps to appreciate the results.  

2) On many places, the text is rather difficult to follow, with long and unnecessarily complicated sentences. In some cases, sentences are not grammatically correct. See for example line 275, Figure Caption 3a and h, lines 368-371. On page 9, there are references to Figure 6, but likely authors want to refer to their Figure 4. In line 262, authors refer to the third panel of fig 2a, but it might rather be the first panel?

3) It is not clarified enough what is the difference in effects between the Hebbian Oja's and Correlation Measuring type of learning. The only reasoning I gathered is that Hebbian Oja plasticity rule has the desired property of normalization that prevents too strong E-E connectivity.  What is the drawback of using Hebbian Oja's plasticity for E-I and I-I connectivity? Authors should go deeper into explaining this, as it seems to be crucial for the results they obtain.

4) In Eq. 7, authors define the energy cost of synaptic transmission as inversely proportional to the connection strength. While this is an interesting choice, it is not intuitive why a quantity inversely proportional to the connection strength is used as the metabolic cost. Also, it is not well justified why is the information capacity computed as the entropy of the weight distribution. Could authors elaborate on that and provide references if applicable?

5) The measure of reliability in Eq. 10 is based on the encoding capability of single neurons. Today, however, there is ample evidence suggesting that in the cortex, signal processing is better captured on the level of neural populations.

questions:
1) Seen that Eq. 4 is the one actually used for modelling, it is not clear what is the purpose of Eq. 3? Also, it is unclear what the variable z in Eq. 3 stands for.

2) Why is the "neural connectivity parameter" (max weight) higher for E compared to I neurons? At the same time, authors report that E-I connectivity should be stronger compared to E-E connectivity.

3)  In lines 173-174, authors comment on their choice of hyperparameters and state that "their approach is consistent with empirical findings". It remains unclear how specific hyperparameters of their model relate to empirical findings. Could authors be more specific?

4) How is the coefficient in Eq. 8 different from a well-known measure of the cross-correlogram on neural spike trains (Bair et al., J.Neurosci. 2001)?

5) A recent study showed that efficient encoding of uncorrelated stimulus features with E-I spiking networks does not require E-E connectivity (Koren et al., eLife 2024). However, your study seems to suggest the necessity of E-E connectivity. Could authors comment on this discrepancy?

6) One of the take-home messages of the paper seems to be that the pinwheel structure is more efficient than the salt-and-pepper structure. However, salt-and-pepper organization can be observed in biological brains, and it seems unlikely that it would survive evolution if it was inefficient. Moreover, authors show that salt-and-pepper type of organisation arises when the overlap of inputs to E neurons is small, which is the case in some animal species. All together, it seems to me that there is no single "most efficient" type of structure, but that there are multiple efficient solutions depending on a specific model parameter. Rather than making a simplistic (and not quite correct ?) conclusion and point to one and only best solution, it would seem to me more appealing if these results are interpreted more carefully to give us better insights about biology.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730700051834

thread_type:
official_review

<thread_object>
creator: reviewer_setg

<content>
summary:
The paper investigates how the pattern of orientation map emerges from a recurrent spiking network model with synaptic plasticity, and concludes that the feedforward input overlap is crucial for forming different map patterns, i.e., salt-and-pepper vs. pinwheel structures.

strengths:
1. Biological plausibility of the recurrent spiking network model with plasticity. The model is well supported by neurobiological experiments and is quite similar to the conventional recurrent spiking network used in neuroscience studies.

weaknesses:
1. Although the authors claim they provide a spiking network mechanism of orientation map, the spiking dynamics seem unnecessary, because none of the results rely on spike timing information. All learning rules are based on the firing rate of neurons only. This makes the spiking network model look like a "strawman" a bit. I envision the spiking network model with spike-time-dependent plasticity rule might have different mechanisms from the rate-based Hebbian rules. Anyhow, I think the author can provide some explanations about how much we could gain about the orientation map formation from the proposed spiking network model. 

2. I am still debating about the contribution of this work to ICLR society. That is, how much we could gain from this study to develop the next-generation AI algorithm? Or how the results in the paper can be incorporated into the modern deep learning algorithms? Another thing is although the model is biologically plausible, all learning rules and network model architecture are not new.

3. It is uncommon to see the title of the paper PDF differ from the title shown in the OpenReview. This makes me concerned that the paper might not be well prepared, at least in the submission stage.

### Minor
1. When comparing output orientation map size among different species, the model should scale with the different neuron density, RF size, and magnification factor (in Table 2) across species. For example, the patch size that each neuron perceives is a constant (16 pixel), while for different species it should be scaled with the following term:

    V1 RF size (deg) * magnification factor (mm/deg) * sqrt(neuron density) (neuron / mm)

(so the outcome is the receptive field size scaled with neuron number)

Also, the recurrent wiring size scaled with neuron number in Eq. (5), presumably conservative about 100 ~ 200 um across species, should be scaled with sqrt(neuron density) (neuron / mm) in different species.

questions:
1. What is the $z_j(t)$ in Eq. 3?

2. Could the authors comment on the differences between the current model with Hansel & Vreeswijk, J. Neurosci., 2012?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732303282526

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response (2/3)

comment:
### **General Response 2: Distinguishing SESNN from Other Models by Pinwheel Maps and Functional Advantages**

**Retinotopic Constraints, Overlapping Visual Fields, and Learnable E-E Connections:** Our model emphasizes the biological fidelity of retinotopic topographic constraints, ensuring each receptive field processes only a small local patch from natural images, unlike conventional models that process entire images without retinotopic constraints. Overlapping visual fields play a crucial role in maintaining consistent orientation selectivity among neurons, which is essential for analyzing the influence of feedforward heterogeneity and the resulting orientation map formation (Fig. 2). Another important feature of the SESNN model is its learnable excitatory-excitatory (E-E) connections. Unlike other models that typically focus on excitatory-inhibitory dynamics, our model demonstrates that excitatory neurons can enhance orientation selectivity through local plasticity, effectively aligning adjacent neurons' firing activities. These learnable E-E connections promote orientation similarity and continuous orientation preference, resulting in iso-orientation domains, distiguishing pinwheel and salt-and-pepper patterns.

**Natural Image Encoding & Spike Timing Contribution:** The SESNN model encodes visual information through spike timing, offering a key distinction from traditional rate-based models. We conducted the new experiment using the natural image set to show that the spike timing of pinwheels provides a more complex and efficient encoding of natural images compared to salt-and-pepper structures (**Reviewer seTg, W1.1, Tables a and b**). This temporal precision improves robustness and enhances coding efficiency, supporting the biological plausibility of our network and emphasizing the functional advantages of pinwheel structures.

**Comparative Analysis with Existing Models:** Our SESNN model has several advantages compared to previous models (Hansel & van Vreeswijk, Stevens et al.) (**Reviewer seTg, Q2, Table c & Reviewer GMwp, W1, Table a**). Specifically, our model goes beyond traditional salt-and-pepper organizations, showcasing the emergence of functional pinwheel maps, which provide functionally enhanced noise robustness, computational efficiency with trade-off between wiring costs and representation capabilities. Moreover, the SESNN model incorporates dynamic spiking activity with synaptic plasticity, whereas earlier models often relied on pre-determined connections or random anatomical distances.

### **General Response 3: Integration of SESNN Orientation Maps into Modern Deep Learning Algorithms**

Our SESNN model demonstrates its utility as a front-end encoder for deep learning, particularly in image classification tasks. Compared to Poisson spike encoding, our model's pinwheel organization produces structured spatiotemporal patterns that significantly enhance classification accuracy **(Reviewer seTg, W2.1, Table c)**. Furthermore, our evaluations under noisy conditions reveal that the pinwheel-based encoder exhibits superior robustness, effectively retaining meaningful features even as noise levels increase **(Reviewer seTg, W2.1, Table d)**. These results suggest that integrating our biologically inspired SESNN model into deep learning pipelines can improve both performance and resilience, making it valuable for advancing noise-resistant AI systems.

We appreciate **Reviewer seTg's (W2.1)** feedback, highlighting that integrating our SESNN orientation map into deep learning algorithms could make a meaningful contribution to the AI community.

### **General Response 4: Refinement of Paper Layout, Typographical Errors, Text Clarity; New Additional Experiments**

We appreciate the **Reviewer zfyC**'s detailed comments on **refining** our paper's **layout**, **correcting typographical errors**, and **enhancing text clarity**. We have thoroughly reviewed the manuscript and implemented them in the updated manuscript **using red text**.

In response to the reviewers' comments, we added **new experiments** to validate our findings further: 

**1.** Neuronal response onset latency from pinwheels and salt-and-peppers relates to structural complexity. **(Reviewer seTg, Table a)**

**2.** Spike timing information plays a critical role in decoding visual features. **(Reviewer seTg, Table b)**

**3.** Classification accuracy achieved using our model's spike sequences compares with that of Poisson spike sequences. **(Reviewer seTg , Table c; Reviewer GMwp, Table b)**

**4.** Comparison of classification performance using different spiking sequence representations as inputs. **(Reviewer seTg , Table d; Reviewer GMwp, Table c)**

**5.** Representational stability for both pinwheel and salt-and-pepper organizations. **(Reviewer zfyC, Table a)**
</content>
</thread_object>

</thread>
<thread>
creation date:
1732303330774

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response (1/3)

comment:
We sincerely thank the reviewers for their valuable suggestions and insightful comments, which have significantly improved our work. These inputs have helped us better explain the mechanisms underlying the formation of two distinct functional maps in V1—pinwheel structures and salt-and-peppers—through a spiking neural network model with biologically plausible synaptic plasticity, emphasizing their differences in encoding efficiency and robustness. Throughout the rebuttal-discussion period, we have addressed each point raised and clarified the presentation of our work. **The revised manuscript is now available.**

We have outlined our timeline for submitting the final revised manuscript (please refer **table below**). The revisions address your valuable feedback and aim to strengthen the clarity and quality of our work. Should you find these improvements satisfactory, we would appreciate your consideration in updating your reviewer score and summary accordingly. We sincerely thank you for your time and thorough review.

| Date       | Task  |
|------------|-------|
| 11/23      | First Revision by Authors *(the **PDF** button (top-right))*|
| 11/23-25   | Feedback from reviewers |
| 11/26      | Final Revision by Authors, merged into `SESNN_final_manuscript.pdf`|
| 11/27      | Final update of reviewer scores and summaries |
                                                               

### **General Response 1: Orientation Map Formation in Our Proposed Spiking Network Model (Updated in Revised manuscript, New Section A.6)**

**Role of HO Rule: Learning Receptive Fields.** This SESNN model uses a combination of Hebbian learning (through the Hebbian-Oja (HO) rule) and anti-correlation (through the Correlation Measuring (CM) rule from the E-I-E loop) to develop biologically realistic visual maps in the cortex. The HO rule is used to teach the spiking neural network model how to form receptive fields. Receptive fields are the regions of the visual field that individual neurons respond to. The model learns these receptive fields from visual images using a learning rule based on the correlation between the input and the activity of neurons.

**Retinotopic Topographic Constraints.** The model also respects the retinotopic organization of the visual cortex. In biology, retinotopy means that adjacent neurons in the cortex correspond to adjacent regions in the visual field. The spiking model enforces this constraint by ensuring that each neuron in the network is only exposed to a localized patch of the input image, rather than the entire visual field. 

**High Overlap Enhances Local Excitatory Connections for Map Continuity.** The excitatory-excitatory connections between neurons are learned in a way that strengthens the connections between neurons with overlapping receptive fields. This means that neurons that "see" similar parts of the visual field form strong local connections with each other. This concept of overlapping receptive fields is critical because it ensures that neurons next to each other have similar orientation preferences, leading to a smoothly varying iso-orientation domain.

**Role of CM Rule: Anti-Correlation via E-I-E loops Maintains Diversity and Full Map Coverage.** Meanwhile, the CM rule is used to promote diversity and coverage in the orientation map. Detailedly, the CM rule ensures that excitatory neurons do not become overly correlated through E-I-E loops, promoting diversity and full coverage of orientation preferences across the cortical map.

**Pinwheel Map Formation.** Together, these mechanisms help in forming biologically plausible pinwheel structures and iso-orientation domains, which are crucial for understanding how the visual cortex processes complex visual stimuli.

We appreciate the feedback from all reviewers, and are eager to provide insights into the formation of the pinwheel orientation map from our SESNN perspective. Detailed explanations can be found in our responses to **Reviewer seTg (W1.2)** and **Reviewer GMwp (Q1.1)**. Additionally, the discussion on the functional roles of HO and CM learning rules may also help address **Reviewer zfyC's (W3)** question.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732509223811

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Quick Response

comment:
We sincerely appreciate all the reviewers' valuable suggestions. Many of these have been incorporated into the revised manuscript (accessible via **the PDF button at the top-right**), including improvements in **layout**, **error correction**, **enhanced text clarity**, and **reducing acronyms** to improve readability. To further clarify **the rationale behind using the two Hebbian-like HO and CM rules** (see revised manuscript, lines 143-153), **the function of Eqs. 3 and 4** (lines 180-183), and **the process of orientation map formation in our proposed spiking network model**, we have added a new Section A.6 in revised manuscript, along with other minor revisions. All changes are highlighted **in red**.

We gratefully await your valuable feedback to make our paper even stronger!
</content>
</thread_object>

</thread>
<thread>
creation date:
1732680199354

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Gratitude for Score Revisions

comment:
Dear Reviewers seTg, zfyC, and GMwp, we are thrilled to see your **scores raised from 5 to 6 and from 6 to 8**. Thank you once again for your invaluable feedback and support in enhancing our paper. We truly appreciate your constructive insights.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733109120198

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response (3/3)：A New Additional Experiment

comment:
In response to Reviewer seTg's insightful comments, we conducted an additional experiment to further validate our findings. Specifically, we compared the spiking sequences generated by our Pinwheel Model with firing rate models derived through two distinct methods (**Reviewer seTg, Table g**). These comparisons highlight the critical role of spike timing in encoding visual information effectively.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper explores the computational impact of structured topological maps in neural processing. The authors present a model of sensory cortex wherein Hebbian plasticity and physiological parameters are combined to shape the network connections. This results in topological maps that resemble those seen in cortex: random "salt-and-pepper" for small networks, large, "pinwheel" structures for big networks. The authors claim that critical factors, such as the degree of input visual field overlap, neuronal density, and the balance between localized versus and long-range connectivity competition determine the which type of topolgy emerges. Furthermore, the authors claim that the "large mammal" like pinwheel structures exhibit lower wiring cost and enhanced sparse coding capabilities compared to salt-and-pepper organizations, and that they maintain greater coding robustness against noise.

The strengths of this paper are that it well-connected to experimental data and well-motivated by it, and it brings potentially important insights into why real brains exhibit a continuum between salt-and-pepper and pinwheel structures. It largely supports its empirical claims as well through its experiments. The weaknesses are that the background literature review was limited and its interest for the ICLR community specifically is not very clear (it is really more of a computational neuroscience paper).

On balance, given these considerations and the scores (average of 7.33), a decision of accept (poster) was reached.