Paper: Adaptive deep spiking neural network with global-local learning via balanced excitatory and inhibitory mechanism (ICLR-2024)

Reviewer comment (Novelty):
The contribution of this work is not very clear at the current stage. In the reviewer's understanding, this work aims at improving the current hybrid learning algorithms which "have some theoretical and practical shortcomings and need further improvement". However, there is no theoretical (mathematical) results in the paper and the biological plausibility of the proposed hybrid method is inadequately clarified. Furthermore, the practical performance is relatively unsatisfactory: (i) The improvement based on the global learning rule STBP is minimal. In essence, STBP represents a special case of the proposed EIHL with specifically chosen hyperparameters. Consequently, EIHL can consistently yield slightly superior results to STBP through randomness and careful hyperparameter tuning. (ii) The performance notably lags behind the latest research. Especially, the SOTA results of DVS_CIFAR10 is 20% better than the proposed method. (iii) There is no comparison between this work and other hybrid methods regarding accuracy and biological plausibility. (iv) No experiments on large-scale datasets. In summary, the motivation and contribution of the paper remain ambiguous. The reviewer perceives this work as merely a combination of two existing methods without convincing reasons.

Author response:
We appreciate the feedback from the reviewer. We hope our subsequent responses have adequately addressed their concerns and questions regarding the paper.

* **The contribution of this work is not very clear at the current stage. In the reviewer's understanding, this work aims at improving the current hybrid learning algorithms which "have some theoretical and practical shortcomings and need further improvement". However, there is no theoretical (mathematical) results in the paper and the biological plausibility of the proposed hybrid method is not inadequately clarified. Furthermore, the practical preformance is relatively unsatisfactory:**

We apologize for the confusion, and then according to the reviewer’s suggestion, we re-summarized the contributions of this work:

1. We propose an excitation-inhibition mechanism-assisted hybrid learning(called EIHL), which can combine the high accuracy of global learning and the low energy consumption of local learning and showed excellent accuracy and sparsity in the experiments on three datasets and two models.
2. We obtained inspiration from neuroscience and adopted the excitation-inhibition mechanism to solve the problem of how to reasonably integrate hybrid learning. The experimental results showed that EIHL also achieved sparsity advantages, which made it possible to deploy on hardware.
3. We used the neural excitation-inhibition mechanism to achieve the integration of global and local learning and implemented it by adjusting the weights. This not only provides a new perspective for the field of SNN training methods but also prepares for the generalization of EIHL to the ANN domain.

* **(i). The improvement based on the global learning rule STBP is minimal. In essence, STBP represents a special case of the proposed EIHL with specifically chosen hyperparameters. Consequently, EIHL can consistently yield slightly superior results to STBP through randomness and careful hyperparameter tuning.**

We appreciate the reviewer’s point, but first of all, our motivation is to propose a hybrid learning method that can well integrate the high-precision advantage of global learning and the low-energy consumption advantage of local learning. According to the experimental results analysis, our proposed EIHL achieved the established goal, with higher accuracy than global learning and higher sparsity than local learning. Instead of simply pursuing the improvement of accuracy.

Secondly, in the neural excitation-inhibition mechanism, it is the minority of inhibitory neurons that inhibit the majority of excitatory neurons, and the inhibitory effect is crucial[1-3]. Therefore, we think that the minority of inhibited local learning in EIHL is also crucial.

Finally, in the Tab.1, especially on the DVS dataset, the accuracy of EIHL is 2.7% and 4.35% higher than STBP on the two models, respectively, which is not a gap that can be achieved by simply tuning parameters. In Tab.3, EIHL has higher sparsity than STBP by 10% or even 30% under higher accuracy than STBP, which is obviously not achievable by simply tuning parameters.

* **(ii). The performance notably lags behind the latest research. Especially, the SOTA result of DVS_CIFAR10 is 20% better than the proposed method.**

We agree with the reviewer’s comments and apologize for that, and we will use more advanced models to achieve higher baseline accuracy in future work. The model used in this paper is simple, which directly affects the quality of the results. In addition, there are many parameters that can be adjusted in a network, such as the optimizer, the number of iterations, the learning rate, the learning rate decay rate, etc. There are too many factors that affect the accuracy of the network. Therefore, we also accept the reviewer’s suggestions, and in future work, we will conduct experiments with a better baseline model and parameters.

> [1] Simeone, & Rho, J. M. (2009). Antiepileptic Drugs: Antiepileptic Drug Mechanisms. In Encyclopedia of Basic Epilepsy Research (pp. 59–66). Elsevier Inc. https://doi.org/10.1016/B978-012373961-2.00160-0

> [2] Shea, Thomas. (2021). An Overview of Studies Demonstrating that ex vivo Neuronal Networks Display Multiple Complex Behaviors: Emergent Properties of Nearest-Neighbor Interactions of Excitatory and Inhibitory Neurons. The Open Neurology Journal. 15. 3-15. 10.2174/1874205X02115010003

> [3] Wang X J. Macroscopic gradients of synaptic excitation and inhibition in the neocortex[J]. Nature Reviews Neuroscience, 2020, 21(3): 169-178.

Reviewer comment (Writing):
This paper is not well-written. Several sentences lack coherence, making the overall presentation disjointed. The presentation of equations is arbitrary and non-standard. The resolution of Fig. 1 is low.

Author response:
* **This paper is not well-written. Several sentences lack coherence, making the overall presentation disjointed. The presentation of equations is arbitrary and non-standard. The resolution of Fig. 1 is low.**

We have carefully checked and polished the whole paper, and we have tried our best to make the paper more readable and coherent. Then, we also re-optimized our figures and formulas to present our work more clearly.

*We hope that these revisions can help the reviewers to have a clearer understanding of our paper. If the reviewers have any questions or suggestions, please feel free to tell us, and we will improve them as soon as possible.*

Reviewer comment (Writing):
Could the authors provide a more comprehensive explanation of the excitation-inhibition mechanism and how it is used in this work? Although the authors keep mentioning it, the reviewer cannot understand how excitatory and inhibitory synapses are handled differently and how they are balanced.

Author response:
* **Could the authors provide a more comprehensive explanation of the excitation-inhibition mechanism and how it is used in this work? Although the authors keep mentioning it, the reviewer cannot understand how excitatory and inhibitory synapses are handled differently and how they are balanced.**

*Theoretical analysis:*

Firstly, the challenge of hybrid learning lies in the integration of global and local learning. While current hybrid learning methods have achieved excellent results in terms of accuracy[1], they could do slightly better in integrating the low energy consumption advantage of local learning. Therefore, based on the exciting Long-Term Potentiation (LTP) results of global learning and the inhibitory Long-Term Depression (LTD) results of local learning, an excitatory-inhibitory mechanism is adopted to balance the two.

Secondly, in the cerebral cortex, the excitatory mechanism can enhance synaptic strength[2], while the inhibitory mechanism can weaken it[3,4]. Unlike previous works[5,6] that directly distinguish between excitation and inhibition at the synaptic and neuronal levels, we use the network connection status to differentiate between excitatory and inhibitory states.

Finally, according to the excitatory-inhibitory mechanism, excitation and inhibition are automatically balanced. That is, when the network is overly excited, it should be inhibited, and when it is in an overly inhibited state, it should be excited. We propose an Excitatory-Inhibitory Hybrid Learning (EIHL) method, which better integrates the advantages of local and global learning, resulting in a model with high accuracy and low power consumption.

> [1] Wu Y, Zhao R, Zhu J, et al. Brain-inspired global-local learning incorporated with neuromorphic computing[J]. Nature Communications, 2022, 13(1): 65.

> [2] Malenka R C, Nicoll R A. Long-term potentiation--a decade of progress?[J]. Science, 1999, 285(5435): 1870-1874.

> [3] Abraham W C, Bear M F. Metaplasticity: the plasticity of synaptic plasticity[J]. Trends in neurosciences, 1996, 19(4): 126-130.

> [4] Li X, Steffens D C, Potter G G, et al. Decreased between‐hemisphere connectivity strength and network efficiency in geriatric depression[J]. Human brain mapping, 2017, 38(1): 53-67.

> [5] Kern F B, Chao Z C. Short-term neuronal and synaptic plasticity act in synergy for deviance detection in spiking networks[J]. PLOS Computational Biology, 2023, 19(10): e1011554.

> [6] Zhu G, Zhang Z, Zhang X Y, et al. Diverse Neuron Type Selection for Convolutional Neural Networks[C]//IJCAI. 2017: 3560-3566.

Reviewer comment (Presentation):
Is the EIHL method only applied to the convolutional layer? If not, I hope it can be reflected in the figure.

Author response:
We apologize for the reviewer’s concern about Figure 1, we agree with the reviewer’s comments, and we have optimized Figure 1.

Fig.1 is a description of the EIHL learning process. From the figure, it can be seen that after local learning, the synaptic connections weaken or even disconnect. And in global learning, the disconnected synapses can also reconnect and have the opportunity to learn again. Then, answer the reviewer’s questions about Fig.1: The EIHL method is applied to all layers that need to be trained. Because the main layer is the convolutional layer, only the convolutional layer is marked in the graph. But we thank the reviewer for pointing this out, and accept the reviewer’s opinion to refine Fig.1 to better present the EIHL algorithm.

We are very grateful to the reviewers for their precise comments on this paper. We accept the reviewers’ suggestions and polish the paper accordingly, hoping to eliminate the reviewers’ concerns.

Reviewer comment (Presentation):
I suggest updating and optimizing Fig.1.

Author response:
We apologize for the reviewer’s concern about Figure 1, we agree with the reviewer’s comments, and we have optimized Figure 1.

Fig.1 is a description of the EIHL learning process. From the figure, it can be seen that after local learning, the synaptic connections weaken or even disconnect. And in global learning, the disconnected synapses can also reconnect and have the opportunity to learn again. Then, answer the reviewer’s questions about Fig.1: The EIHL method is applied to all layers that need to be trained. Because the main layer is the convolutional layer, only the convolutional layer is marked in the graph. But we thank the reviewer for pointing this out, and accept the reviewer’s opinion to refine Fig.1 to better present the EIHL algorithm.

We are very grateful to the reviewers for their precise comments on this paper. We accept the reviewers’ suggestions and polish the paper accordingly, hoping to eliminate the reviewers’ concerns.

Reviewer comment (Writing):
The authors should elucidate their contributions more explicitly in order to provide a comprehensive understanding of the research.

Author response:
We appreciate the feedback from the reviewer. We hope our subsequent responses have adequately addressed their concerns and questions regarding the paper.

**Weakness Ⅰ:**

* **The authors should elucidate their contributions more explicitly in order to provide a comprehensive understanding of the research.**

We apologize for the confusion, and then according to the reviewer’s suggestion, we re-summarized the contributions of this work:

1. We propose an excitation-inhibition mechanism-assisted hybrid learning(called EIHL), which can combine the high accuracy of global learning and the low energy consumption of local learning and showed excellent accuracy and sparsity in the experiments on three datasets and two models.
2. We obtained inspiration from neuroscience and adopted the excitation-inhibition mechanism to solve the problem of how to reasonably integrate hybrid learning. The experimental results showed that EIHL also achieved sparsity advantages, which made it possible to deploy on hardware.
3. We used the neural excitation-inhibition mechanism to achieve the integration of global and local learning and implemented it by adjusting the weights. This not only provides a new perspective for the field of SNN training methods but also prepares for the generalization of EIHL to the ANN domain.

Reviewer comment (Theory):
In the context of hybrid learning, 'STDP' process employs a contraction curve to facilitate Long-Term Depression. Nevertheless, the authors have not adequately expounded upon the association between LTD and STDP, and the proposed method do not have a dependence of the spike timing. It's not clear why excitation should be like STBP and depression should be like STDP.

Author response:
We appreciate the feedback from the reviewer. We hope our subsequent responses have adequately addressed their concerns and questions regarding the paper.

**Weakness Ⅱ:**

* **In the context of hybrid learning, 'STDP' process employs a contraction curve to facilitate Long-Term Depression. Nevertheless, the authors have not adequately expounded upon the association between LTD and STDP,**

Spike-Timing Dependent Plasticity (STDP)[1] is a synaptic plasticity mechanism that depends on the firing sequence of pre- and post-synaptic neurons. In STDP, if the pre-synaptic neuron fires before the post-synaptic neuron, synaptic strength increases; if the pre-synaptic neuron fires after the post-synaptic neuron, synaptic strength decreases. Long-Term Potentiation (LTP) and Long-Term Depression (LTD) are the two components that constitute STDP[2]. However, significant LTP only occurs at synapses with relatively low initial strength, whereas the extent of LTD does not show an obvious dependence on the initial synaptic strength[3]. Therefore, the overall effect of STDP tends to exhibit LTD.

* **and the proposed method do not have a dependence of the spike timing.**

Our approach does not rely on spiking timing, yet the original intention of Spiking Neural Networks (SNN) is to emulate brain systems, distinguishing it from Artificial Neural Networks (ANN). Therefore, our process of addressing the hybrid learning problem through biological inspiration aligns with the SNN field, and there is no necessity to utilize spiking timing. Lastly, it is precisely because we do not depend on spiking timing that EIHL can be conveniently extended to the ANN field.

* **It's not clear why excitation should be like STBP and depression should be like STDP.**

Due to BP’s powerful learning capabilities, most weights increase after STBP learning, which corresponds to the excitation of LTP[4]. STDP, on the other hand, exhibits an overall LTD effect, corresponding to inhibitory LTD[4].

Reviewer comment (Writing):
The accuracy of references should be ensured. For example, the paper states, "Spike-Timing Dependent Plasticity (STDP) was proposed based on these rules by Caporale & Dan (2008)." However, the discovery of STDP predates 2008. Caporale & Dan (2008) is a review paper.

Author response:
We appreciate the feedback from the reviewer. We hope our subsequent responses have adequately addressed their concerns and questions regarding the paper.

**Weakness Ⅲ:**

* **The accuracy of references should be ensured. For example, the paper states, "Spike-Timing Dependent Plasticity (STDP) was proposed based on these rules by Caporale & Dan (2008)." However, the discovery of STDP predates 2008. Caporale & Dan (2008) is a review paper.**

First of all, we would like to thank the reviewer for pointing out the mistake about the STDP reference, and we apologize for this mistake. We have corrected the STDP reference in section 2.1 to “Based on these rules, spike-timing dependent plasticity (STDP) was proposed by Song et al. (2000)[1].” We appreciate the reviewer’s correction.

*We hope that these revisions can help the reviewers to have a deeper understanding and a clearer understanding of the contributions and innovations of our paper. If the reviewers have any questions or suggestions, please feel free to tell us, and we will improve them as soon as possible.*

> [1] Song S, Miller K D, Abbott L F. Competitive Hebbian learning through spike-timing-dependent synaptic plasticity[J]. Nature neuroscience, 2000, 3(9): 919-926.

> [2] Zenke F, Gerstner W, Ganguli S. The temporal paradox of Hebbian learning and homeostatic plasticity[J]. Current opinion in neurobiology, 2017, 43: 166-176.

> [3] Bi G, Poo M. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type[J]. Journal of neuroscience, 1998, 18(24): 10464-10472.

> [4] Abraham W C, Bear M F. Metaplasticity: the plasticity of synaptic plasticity[J]. Trends in neurosciences, 1996, 19(4): 126-130.

Reviewer comment (Theory):
In the context of the hybrid learning rule, what is the significance of excitatory and inhibitory synapses, given that STDP and STBP do not appear to rely on the distinction between these synapse types? Furthermore, it seems that excitatory and inhibitory synapses are not typically delineated in deep spiking neural networks.

Author response:
We appreciate the feedback from the reviewer. We hope our subsequent responses have adequately addressed their concerns and questions regarding the paper.

**Question Ⅰ:**

* **In the context of the hybrid learning rule, what is the significance of excitatory and inhibitory synapses, given that STDP and STBP do not appear to rely on the distinction between these synapse types?**

Firstly, the challenge of hybrid learning lies in the integration of global and local learning. While current hybrid learning methods have achieved excellent results in terms of accuracy[1], they could do slightly better in integrating the low energy consumption advantage of local learning. Therefore, based on the exciting Long-Term Potentiation (LTP) results of Spike-Timing-Dependent Plasticity (STDP) and the inhibitory Long-Term Depression (LTD) results, an excitatory-inhibitory mechanism is adopted to balance the two.

Secondly, in the cerebral cortex, the excitatory mechanism can enhance synaptic strength[2], while the inhibitory mechanism can weaken it[3,4]. Unlike previous works[5,6] that directly distinguish between excitation and inhibition at the synaptic and neuronal levels, we use the network connection status to differentiate between excitatory and inhibitory states.

Finally, according to the excitatory-inhibitory mechanism, excitation and inhibition are automatically balanced. That is, when the network is overly excited, it should be inhibited, and when it is in an overly inhibited state, it should be excited. We propose an Excitatory-Inhibitory Hybrid Learning (EIHL) method, which better integrates the advantages of local and global learning, resulting in a model with high accuracy and low power consumption.

* **Furthermore, it seems that excitatory and inhibitory synapses are not typically delineated in deep spiking neural networks**

For this reason, this paper does not adopt the approach of explicitly distinguishing between excitatory and inhibitory synapses in the network. Instead, it differentiates between excitation and inhibition based on the connection status of the network.

> [1] Wu Y, Zhao R, Zhu J, et al. Brain-inspired global-local learning incorporated with neuromorphic computing[J]. Nature Communications, 2022, 13(1): 65.

> [2] Malenka R C, Nicoll R A. Long-term potentiation--a decade of progress?[J]. Science, 1999, 285(5435): 1870-1874.

> [3] Abraham W C, Bear M F. Metaplasticity: the plasticity of synaptic plasticity[J]. Trends in neurosciences, 1996, 19(4): 126-130.

> [4] Li X, Steffens D C, Potter G G, et al. Decreased between‐hemisphere connectivity strength and network efficiency in geriatric depression[J]. Human brain mapping, 2017, 38(1): 53-67.

> [5] Kern F B, Chao Z C. Short-term neuronal and synaptic plasticity act in synergy for deviance detection in spiking networks[J]. PLOS Computational Biology, 2023, 19(10): e1011554.

> [6] Zhu G, Zhang Z, Zhang X Y, et al. Diverse Neuron Type Selection for Convolutional Neural Networks[C]//IJCAI. 2017: 3560-3566.

Reviewer comment (Novelty):
Weight pruning is a technique employed in deep learning to increase network sparsity by eliminating the smallest weights. Please elucidate the distinctions between the 'STDP' process in EIHL and weight pruning techniques in deep learning.

Author response:
We appreciate the feedback from the reviewer. We hope our subsequent responses have adequately addressed their concerns and questions regarding the paper.

**Question Ⅱ:**

* **Weight pruning is a technique employed in deep learning to increase network sparsity by eliminating the smallest weights. Please elucidate the distinctions between the 'STDP' process in EIHL and weight pruning techniques in deep learning.**

We apologize for the confusion regarding the distinctions between the ‘STDP’ process in EIHL and weight pruning techniques in deep learning.

Firstly, weight pruning techniques in deep learning are more of an engineering technique, aimed at reducing the complexity and computation of the model. However, the ‘STDP’ process in EIHL is part of the hybrid training method, aimed at mimicking LTD to suppress network excitation.

Secondly, weight pruning techniques in deep learning do not fully simulate the working principle of biological neural systems, but the ‘STDP’ process in EIHL is a form of synaptic plasticity. It is just that the concept of pruning has some similarities with synaptic plasticity.

Finally, in terms of the specific operation, the ‘STDP’ process in EIHL and weight pruning techniques in deep learning have some similarities. Similarity: They are both implemented by removing the smallest weights. Difference: Pruning is to directly set the weights that are less than a certain value or a certain percentage of the weight distribution to zero. The ‘STDP’ process in EIHL is to fix the update direction of the weights that are less than H(x) according to Eq.7, and then set the weights in the zero domain to zero. Moreover, the proportion of weights that need to be shrunk in the ‘STDP’ process in EIHL is automatically adjusted according to the increase of the target sparsity b, not artificially specified.

Reviewer comment (Writing):
Kindly review the terminology and references utilized in the manuscript to ensure a more precise and coherent presentation of the research study.

Author response:
We appreciate the feedback from the reviewer. We hope our subsequent responses have adequately addressed their concerns and questions regarding the paper.

**Question Ⅲ:**

* **Kindly review the terminology and references utilized in the manuscript to ensure a more precise and coherent presentation of the research study.**

We are very grateful for the reviewer’s comments, and we have carefully checked and polished our paper, correcting the citation errors and incoherent parts. We have tried our best to express the paper more accurately and coherently.

*We hope that these revisions can help the reviewers to have a clearer understanding of our paper. If the reviewers have any questions or suggestions, please feel free to tell us, and we will improve them as soon as possible.*