Adaptive deep spiking neural network with global-local learning via balanced excitatory and inhibitory mechanism
Authors: Tingting Jiang
Qi Xu
Xuming Ran
Jiangrong Shen
Pan Lv
Qiang Zhang
Gang Pan
Venue: ICLR 2024 poster (2024)
Keywords: Spiking neural networks
excitation inhibition mechanism
Long-Term Potentiation
Long-Term Depression
Primary area: applications to neuroscience & cognitive science
TL;DR: Inspired by the excitation-inhibition mechanism, we balance global and local learning by network connectivity to achieve both high accuracy and low energy consumption.

Abstract: The training method of Spiking Neural Networks (SNNs) is an essential problem, and how to integrate local and global learning is a worthy research interest. However, the current integration methods do not consider the network conditions suitable for local and global learning, and thus fail to balance their advantages. In this paper, we propose an Excitation-Inhibition Mechanism-assisted Hybrid Learning(EIHL) algorithm that adjusts the network connectivity by using the excitation-inhibition mechanism and then switches between local and global learning according to the network connectivity. The experimental results on CIFAR10/100 and DVS-CIFAR10 demonstrate that the EIHL not only has better accuracy performance than other methods but also has excellent sparsity advantage. Especially, the Spiking VGG11 is trained by EIHL, STBP, and STDP on DVS_CIFAR10, respectively. The accuracy of the Spiking VGG11 model on EIHL is 62.45%, which is 4.35% higher than STBP and 11.40% higher than STDP, and the sparsity is 18.74%, which is 18.74% higher than the other two methods. Moreover, the excitation-inhibition mechanism used in our method also offers a new perspective on the field of SNN learning.

Published as a conference paper at ICLR 2024
ADAPTIVE DEEP SPIKING NEURAL NETWORK WITH GLOBAL-LOCAL LEARNING VIA BALANCED EXCITA- TORY AND INHIBITORY MECHANISM
Tingting Jiang', Qi Xu!; Xuming
Shen', Pan Lv², Qiang Zhang', Gang Pan?
"School of Computer Science and Technology, Dalian University of Technology
"Coleeee of Computer Science and Technology, Zhejiang University
fjiangtt, xuqi, 2hq/edlut. edu. en, =aaxuming@ggaill com,
(jrshen, lvp, gpan]0zju www.juueducccoo edu.cn cn
ABSTRACT
The training method of Spiking Neural Networks (SNNs) is an essential problem, and how to integrate local and global learning is a worthy research interest. How- ever, the current integration methods do not consider the network conditions suit- able for local and global learning and thus fail to balance their advantages. In this paper, we propose an cistatiin-Inhibitiion Mechanism- (EIHL) algorithm that adjusts the network connectivity by using the excitation- inhibition mechanism and then switches between local and global learning accord- ing to the network connectivity. The experimental results on CIFAR 10/100 and DVS-CIFAR 10 demonstrate that the EIHL not only obtains better accuracy perfor- mance than other methods but also has excellent sparsity advantage. Especially, the Spiking VGG11 is trained by EIHL,, STBP, and STDP on DVS_CIFARY 10, re- spectively. The accuracy of the Spiking VGG11 model with EIHL is 62.45 %, which is 4.35% % higher than STBP and 11.40% higher than STDP. Furthermore. the sparsity achieves 18.74%, which is quite higher than the above two non-sparse methods. Moreover, the
Hybrid Learning
mechanism used in our method also offers a new perspective the field of SNN learning.
INTRODUCTION
Spiking neural networks (SNNs) Maass (997) are a type neural networks that more closely
mimics biological neural systems, with efficiency and energy-saving advantages Xu et al. (2023 Liu
ct al. 2020) Xu et al. 2020 Xu et al. 2024). The learning algorithm of SNNs is vital to their performance and application, which
mainly divided into two categories: local learning Hebb
(2005) Song et al. (2000) Lu & Sengupta (2023) and global learning Zenke & Ganguli (2018) Wu
learning method, suitable for sparse networks and low-power hardware but difficult to handle complex tasks and deep networks. Global learning is loss-driven learning method, suitable for dense networks solve complex tasks, but requires lot of computational energy consumption. Therefore, integrating local and global learning so that the learning algorithm has both the mnergy-saving advantages of local learning and the nigh-performance advantages of global learning is a significant research interest in the field of SNN.
let al. (0188. Local learning is a correlation-drive
The biological three-factor learning rules Gerstner et al. (2018) Bailey al. (2000) provide biologi- cal insights for the new way of integrating local and global learning. The three-factor rule is that the adjustment of synaptic weights depends not only on the activity of the pre-synaptic neuron and the post-synaptic neuron but also on third factor (e.g., neurotransmitter. [w et al. (2022) also pro- posed hybrid synergic learning algorithm, which uses two sets of weights to update the weights of local learning and global learning separately and then directly adds the two sets of weights, achiev- ing comparable performance. However, the current hybrid learning algorithm needs to trade-ofi between the accuracy advantage of global learning and the low energy consumption advantage of
"Corresponding author; Qi Xu, suqi@dlut.eduu..

---

Published as a conference paper at ICLR 2024
local learning. Moreover, the type of hybrid learning algorithms still needs reasonable biolog- ical plausibility. Hence, the current hybrid learning algorithms have some theoretical and practical shortcomings and need further improvement and optimization via the nism.
mecha-
In this paper, propose an Excitation-Inhiiition Mechanism assisted hybrid Learning(EIHL) al- mechanism of biological neural networks Rosenberg
gorithm. Inspired by the
al. (2023) Sadeh & Clopath (2021) Wang et al. (2022) Xue et al. (2014) Shea (2021) Simeone & Rho (2009) Wang (2020), We choose to distinguish between excitation and inhibition states accord- ing to the overall connection state of the network. When the neural network is in an over-excited state, it switches to an inhibited state. When the neural network is in an over-inhibited state, it excitation and inhibition the neural network, we achieve dynamic switching between global learning and local learning.
switches an excited state. Based on the dynamic balance of
The main contributions of this paper are as follows:
We propose an excitation-inhibeee which can combine the high accuracy of global learning and the low energy consump- tion of local learning and showed excellent accuracy and sparsity in the experiments on three datasets and two models.
mechanism-assisted hybrid learning(called EIHL),
We obtained inspiration from neuroscience and adopted the excitation-inhibitio mecha- nism to solve the problem of how to reasonably integrate hybrid learning. The experimen- tal results showed that EIHL. also achieved sparsity advantages, which made it possible to deploy on hardware.
We used the neural cecitatioo-inhiiitiin mechanism achieve the integration of global and local learning and implemented it by adjusting perspective for the field of SNN training methods but also prepares for the generalization of EIHL. to the ANN domain.
the weights. This not only provides new
RELATED WORKS
2.1 LEARNING ALGORITHMS FOR SPIKING NEURAL NETWORKS
Local learning: The Hebb learning rule Hebb 2005 states that the pre-synaptic neuron must be activated before the post-synaptic neuron. Based on these rules, spike-timing dependent plasticity (STDP) was proposed by [ong et al. 22000). STDP a local learning algorithm that adjusts synaptic weights based on the pulse timing interval between pre-synaptic and post-synaptic neurons. Deep- SuDP & Sengupta 2023) a local learning algorithm based on STDP, which uses lseudo-labels obtained through clustering for loss backpropagation and updating of synaptic weights, significantly improving performance. However, Deep-STDP also increases computational complexity and energy consumption.
Global learning: The Backpropagation (BP) BP 1990) algorithm performs well in artificial neural networks (ANNs) (ANNs)) Beatonovic-KKsstiie Beresford 22000, but cannot be directly applied to spiking neural networks (SNNs) Maass (1997). So, the SuperSpike Zenke & Ganguli (2018) algorithm em- ploys gradient approximation method but does not consider the unique spatiotemporal information of SNNs. The Temporll Backpropagation (STBP) Wu et al. (2018) algorithm compensates for the insufficient SuperSpike algorithm. STBP modified variant of the backpropagation al- gorithm specifically designed for training SNNs. STBP considers the temporal relationships and connectivity between spiking neurons, emphasizing the temporal nature of spikes. While STBP performs excellently in task recognition accuracy, it also brings greater computational complexity.
In practice, Wu et al. (2022)rooposed hybrid plasticity (HP), which uses two sets of weights to update local and global learning weights, respectively. This hybrid learning method integrates local and global learning, resulting in higher accuracy. HP inspired our method to fuse local and global learning from the weights perspective. In biology, the three-factor learning rules Gerstner et al. (2018) Bailey et al.] (2000) indicate that another factor affects synaptic strength besides the activity between presynaptic and postsynaptic neurons, such neurotranmmitters.
2

---

Published as a conference paper at ICLR 2024
2.2 EXCITATORY AND INHIBITORY MECHANISMS IN NEURAL SYSTEMS
Neuronal excitation and inhibition mechanisms have pathological and functional significanceeRosen berg et al. (2023) Sadch & Clopath (2021) Wang et al. (2022) Xue et al. (2014). From a pathological perspective, excitation -inaiiiiiiniiiiioon not only regulates the emotions of impulse hesitation but also induces or reduces the frequency and duration of epileptic seizures, as well as improves cognitive and memory functions Rooeeberg et al. (2023). From functional perspective, networks with exci- tation and inhibition have stronger learning abilities than networks with only excitation, as shown by Wang al. 2022) who positive weights excitation and negative weights as inhibition in an ANN with monotonic activation function. They proved that networks with only excitation can achieve monotonic functions and can not implement the XOR function.
PRELIMINARY
3.1 INTEGRATE-AND-FEE MODEL
This paper uses the integrate-and-fire (IF) model as a single computational unit in SNNs, which biological neuron model that describes how neurons produce action potentials. The membrane potential of an IF neuron at a certain time step is the decay of the membrane potential from the previous time step plus the external stimulus at this moment. When the membrane potential exceeds a threshold, neuron triggers to firing a spike at this time step. The calculation formula of IF as follows:
M(t) = V(t - 1) + I(t),
(1)
where M(t) and V(t - 1) is the membrane potential of the neuron at time step t and - 1 without spike firing, respectivley. I(t) is the external stimulus the neuron receives time step t.
s(t) = {o
if M(!) 2 Vin M(t) < Vih
(tt) is the marker of whether the neuron fires a spike at time step t, and Vin the threshold for spike firing. If the membrane potential is greater than the Vnnr it fires; if the membrane potential is less than the Vhn does not fire.
V(I) M(1) - S(1) Van
(3)
(()) the membrane potential after spike firing. It is obtained by subtracting Vhr from M(I) there is spike firing.
3.2
PLASTICITY
Spike-Timme Miike-Tiiingga Dependent Plasticity (STDP) Song et al. 2000) is a synaptic plasticity mechanism that depends on the firing sequence of pre- and post-synaptic neurons. In STDP, the pre-synaptic neu- ron fires before the post-synaptic neuron, synaptic strength increases; if the pre-synaptic neuron fires after the post-synaptic neuron, synaptic strength decreases. Long- Term Potentiation (LTP) Malenka et al. (1999a) and Long-Term Depression (LTD) Ito (1989) are the two components that constitute STDP Zenke et al. (2017). However, significant LTP only occurs at synapses with relatively low ini- tial strength, whereas the extent of L.TD) does not show an obvious dependence the initial synaptic strength Bi & Pool (1998). Therefore, the overall effect of STDP tends to exhibit the LTD.
A A-exp
I
if Lj, A>0 0
Awi
T+
B exp
if > ty, <<0
where A and are the maximum values of synaptic weight change, T4 and r_ are time constants. STDP determines the sign of weight change Awiy based the order of spike firing of pre-neuron i and post-neuron j. and the magnitude of Awiz based on the time interval between pre-spike and post-spike firing.
3

---

Published as a conference paper at ICLR 2024
3.3 SPATIO-TEMPORAL BACKPROPAGATION
Spatio-temporal backpropagation (STBP) Wu et al 2018) combines the elements of STDP and backpropagation (BP) for training SNNs. In STBP, the STDP rule is usually used to adjust the synaptic weights between neurons, simulating the biological learning rule. Then, the BP algorithm used to adjust the parameters in the artificial neural network. Due to BP's powerful learning capabilities, most weights increase after STBP learning, which corresponds to the excitation of LTP. Eq. simply introduces the BP loss function L in STBP.
G
L
1 29
Σ g=1
y
7 ΣΣ ON
(5)
where G is the number of samples in a batch, D/g is the label of the y sample in batch, T is the size of the time window. O is the model output of the g sample at time step i, where N is the number of classes. It represents the prediction of the model for the 9 sample at time step t.
4 EXCITATION-INHIIITOON MECHANISM-ASSISE HYBRID LEARNING
Global learning
Pichuee
Excited strte
Conversion condition
Curr_s Pro_s:
Local leaming
Curr_a Turget, Lk:
Esample: T
Inhibitedd
FI
FC
2001
network
Conv
Sprking
Sakig
Forward
sequence
Figure 1: The illustration of the training processing SNNs via EIHL. The solid lines represent the connections that exist, and the dashed lines represent the connections that are broken. In global learning, the LTP effect of excitation is synaptic enhancement, which makes the neurons in the net- work more active to fire spikes. In local learning, the LTD effect of inhibition is synaptic shrinkage or even disconnection, which makes the neurons in the network more passive to fire spikes.
First of all, the challenge of hybrid learning lies in the integration of global and local learning. While current hybrid learning methods have achieved excellent results in terms of accuracy Wu lt al. (2022). they could do slightly better in integrating the low energy consumption advantage of local learning. Therefore, based on the exciting Long-Term Potentiation (LTP) Malenka et al.) (999a)) results of global learning and the inhibitory Long-Term Depression (LTD) Ito (9899 results of local learning, an excitatoryiinhiiioryy mechanism is adopted to balance the two. Secondly, in the cerebral cortex, the excitatory mechanism can enhance synaptic strength Malenka al. (1999b), while the inhibitory mechanism can weaken it Abraham & Bear (1996) Li et al. (017b)) Unlike previous works Kern & Chao (2023) Zhu et al. (2017) that directly distinguish between excitation and inhibition at the synaptic and neuronal levels from microscopic perspective, we use the net- work connection status to differentiate between excitatory and inhibitory states from a macroscopic perspective. Finally, according to the excilatory-nnhibioory mechanism, excitation and inhibition are automatically balanced. As shown in Fig. [ when the network is overly excited, it should be inhibited, and when it is in an overly inhibited state, it should be excited. We propose Excitatory- Inhibitory Hybrid Learning (EIHL) method, which better integrates the advantages of local and global learning, resulting in model with high accuracy and low power consumption. We use the
4

---

Published as a conference paper at ICLR 2024
Algorithm 1: Excitation-Inhibiie Mechanism assisted Fusion learning method.
Require: current sparsity Curr S, previous sparsity Pre. S, current target sparsity b, the increment step of parameter step.
Ensure: SNN model with EIHI.
for i=1 to epoch do
if Curr >= Pre.S and Curr S b then
Running STDP;
Disconnect the weak connections of excitation or inhibition according to the LTD
principle using Eq. and Eq.
else
Running STBP and x undergoes self-decay in Eq. 6
if just flipped to STBP then b + step
end if end if end for
degree of network connectivity to distinguish between excitation and inhibition, so we set a target sparsity b as the threshold for the network to be in an over-inhibited state. In local learning, due to the effect of LTD, the network sparsity gradually increases. When it exceeds the target sparsity b, the network is considered to be in over-inhibited state, and the network will switch to excited global learning. The contraction formula Eq. accelerates the process of the network gradually becoming sparse local learning.
H(x) = thresh(()) x a
(6)
Actually, H(xx) is weight value, and contraction implies that weights less than the boundary value H(x) will shrink until they disconnect. Moreover, the target sparsity b is percentage, not a weight value, thresh(b) is the mapping of b on the weight distribution. In simpler terms, if the weights that are less than (hresh(b) in each layer are set to O, the current sparsity will directly reach the target sparsity b. The network will immediately exhibit an over-inhibited state and will switch to excited global learning. However, defining the range and setting it to 0 directly is too crude and will cause a lot of unnecessary losses. Therefore, x to give H(x)) a slow expansion process from O. first divides the thresh(b) into multiple scales, and as gradually increases, H(x) will also slowly increase until it equals thresh(b).
However, we believe that simply setting the weights below the boundary value H(x) to 0 is still crude. Therefore, we fix the update direction of the weights at the boundary value H(x) to move only towards 0 and then set the weights near zero to O, to accelerate LTD in relatively smooth manner.
W = W -r VW,
W denotes the weight that has not been updated, W denotes the weight that has been updated, VW is the weight update amount of STDP and Ir denotes the learning rate.
Summarize the contraction operation of EIHL on local learning, that is, The update direction of the weights within the H(x) boundary can only tend to O, and the weight area close to 0 can be directly set to 0. Then x + + Hx) gradually expands, and the final set to 0 area reaches the thesh(b) area, the current sparsity reaches the target sparsity b, and switches to the excited global lcarning.
We set the threshold for over-inhibition as 6, but we did not specify a value for the threshold of over-excitation. The criterion for judging the state of over-excitation is "Curr_S > Pre_S". We assume that in global learning, the network is over-excited when the sparsity stops decreasing, and then switches to inhibited global learning. Meanwhile, in order to better demonstrate the tow-power advantage of local learning, the threshold for ineibiiiiit b is incremented by a step size every time it switches.
5

---

Published as a conference paper at ICLR 2024
5 EXPERIMENTAL SETTINGS
5.1 DATASETS
In this section, we conduct experiments with two models: Spiking Resnet res Han et al. (2020) and Spiking VGG11 [Sengupta et al. (2019b), on three datasets: CIFARIO, 10, CIFAR 1000 Krizhevsky et al. (2009), and DVS-CIFARIO Li et al. (2117a). We compare our methods with local learning methods (STDP) and global learning methods (STBP) demonstrate their superiority in accuracy, robustness, and network sparsity.
CIFARIO small RGB dataset with 10 classes. Each category has 6000 images, of which 5000 are used for training and 1000 are used for testing. The shape of each image is 32x32x3, It has some representativeness and universality. It can verify whether the obtained model has general classification ability.
CIFARI00 an extension of CIFARIO, with 20 superclasses, each containing 5 classes. Each category has 600 images, of which 500 are used for training and 100 are used for testing. The shape of each image also x2x32x3. It is very challenging classification task.
DVS-CIFAR10 is an event-based dataset for object classification, which is obtained by scanning the CIFARIO dataset with Dynamic Vision Sensor (DVS) Gallego et (2022). It has total of 10,000 event streams, of which 8,000 are used for training and 2,000 are used for testing. The shape of each event stream is x28x x28x2, where the last dimension represents the timestamp and polarity. is more suitable for the training and testing of Spiking Neural Networks.
5.2 PATTERN RECOGNITION TASK
We conducted four experiments to evaluate our method. First, tested our method on CI- FAR 10/100 and DVS DVS_CIFAS ,CIFARIO,, and compared the accuracy and sparsity with local learning STDP and global learning STBP. verify the essential performance of the model obtained by EIHL. Sec- ondly, we supplemented an ablation study experiment on the increment step of parameter b in the contraction curve Eq. o that directly affects the sparsity the EIHL method on CIFARIO, and controlled the sparsity of the network by the step size, to test the sparsity advantage of the model obtained by EIHL. Thirdly, we did a random pruning experiment on CIFARIO. 100, which is to randomly prune 20%, 40%, and 60% of the network connections, and test the capability of the network to clas- sify spatiotemporal patterns under the EIHL method. Finally, we compare the performance of EIHL, and other hybrid learning methods CIFARIO, 10, verify the ability of EIHL. Furthermore, for all the experiments, we used Spiking Resnet18 and Spiking VGG11 to verify the generalization ability of our method. In these experiments, the sparsity is additional advantage for our EIHL method. The sparsity evaluation metric computes the proportion of weights that are 0 among all the weight parameters. The accuracy evaluation metric equals the proportion of samples correctly classified in the test set.
5.3 IMPLEMENT DETAILS
We conduct experiments on server equipped with 16-core Intel(R) Xeon(R) Xeon(R) Gold 6330 2. 2.80GHz CPU and 20 NVidia GeForce RTX 3090 Ti GPUs. these experiments, we used the SpikingJelly framework Fang al. (220)) to simulate the whole training process, the parameters of STDP layer were updated by the SGD optimizer Mandt et al. (2017), and the global information was updated by the Adam optimizer Zhang (2018). The two optimizers shared a learning rate. The learning rates of Resnet18 on CIFARIO, 10, CIFAR100 and DVS datasets are 2e-3, 5e-4 and 2e-3 respectively. The learning rates of VGG11 1 three datasets 2e-4, 2e-4 and le-5 respectively. Besides, the increment step of parameter in the experiments of Tab. and Tab. 3 is 0.5. The number of epochs in the four experiments is 200.
EXPERIMENTAL RESULTS
Firstly, we evaluate the sparsity and accuracy of EIHL's experimental results as shown in Tab. [ EIHL. achieved certain accuracy advantages and ensured advantageous sparsity. Secondly,

---

Published as a conference paper at ICLR 2024
investigate the influence of the contraction boundary value in EIHL's experimental results as shown in Tab. 2 We found that, overall, the step size is proportional to the sparsity and inversely propor- tional to the accuracy. Thirdly, we compare the performance of the model on different disconnect degrees as shown in Tab. 3 EIHL. still maintained a certain accuracy and high sparsity under differ- ent degrees of random pruning. Finally,
compare the performance of the model with other hybrid learning methods as shown in Tab, 4 The experimental results show that the EIHL method still has superior accuracy and unique sparsity advantages.
6.1 EVALUATION THE SPARSITY AND ACCURACY OF EIHL
We first tested the spiking r 8 model on three datasets, and it outperformed STBP and STDP in terms of accuracy as well as sparsity. For example, on the CIFAR 100 dataset. EIHL's accuracy is 0.15% higher than STBP and 25, 2,797% higher than STDP. Moreover, EIHL's sparsity is 31.75% higher than the other two pure global or local methods. Then, we used another model framework Spiking VGG11 to evaluate the generalization ability of EIHL.. The results show that the test accuracy all datasets is higher than that of STBP and STDP alone. And it also retains a certain sparsity. As shown in Fig. 2 EIHL's accuracy was almost always surpass STBP in the later about 25 epoches stages. It has been shown that when the current sparsity reaches the target sparsity in the EIHL approach, the target sparsity increases by one step when entering STBP, whereas the current sparsity steadily decreases. When the current sparseness no longer shifts, it will switch to STDP during learning, and the current sparsity will steadily grow until it approaches the desired sparsity. Although the EIHI. accuracy is approximately equal to STBP on the VGG model and CIFARIO 10 dataset, it is because the increment step of is 0.5 here. When the increment step of b is Tab. [2] the EIHL method has slightly higher accuracy than STBP, and also 28.42% higher sparsity. Moreover, when the increment step of is O. 125 in Tab. 2 the EIHL accuracy is 0.3% higher than STBP, and also 4.75% higher in sparsity.
EHL
0.522
0.54-
Figure 2: This is the curve variation of the precision and sparseness of the final 25 epochs. Left:: The comparison of the accuracy of three methods CIFAR 100 using the VGG11 model. It can be that the accuracy of EIHL is significantly higher than that of STDP and STBP. Right: The change in current sparsity and target sparsity in EIHL. It can be seen that in the EIHL method, after the current sparsity reaches the target sparsity, the target sparsity will increase by one step when entering STBP, and the current sparsity will gradually decrease. When the current sparsity no longer changes, it will flip to STDP for learning, and the current sparsity will gradually increase until it reaches the target sparsity this time.
Table 1: Evaluation of sparsity and accuracy CIFAR10/100 and the EIHL and other methods.
10 datasets between
Spiking Resnet 8 Hu al. (202
CIFARIO
CIFAR100
DVS CIFARIO
Model
(9arsity(S6
Accuracy(S5)
Sparsity:(9)
Accurayy
Sparsity
Accuracy
STDP STBP
0.00
76.71
0.00
32. 66
0.00
36,00
0.00
89.53
0.00
48
0.00
60.2
ETHI.
17.17
90.25
31.75
58.63
11.22
62.9
Spiking VGGL Sengupta et | 11P
2019b
CIFARIO
DWS,CIFARID
STDP
0.00
78.29
0.00
29.17
0.00
51.05
STBP
0.00
85.76
0.00
$3.75
0.00
58.10
EIHL
13.10
85.75
14.12
55.33
18.74
62/45
7

---

Published as a conference paper at ICLR 2024
Therefore, the proposed hybrid learning methods of EIHL achieves superior performance accu- racy and sparsity compared with pure local or global learning mechanisms for SNNs models. At the same time, compared with the hybrid plasticity (HP) method proposed in Wu et al. (2022). with sparsity of our EIHL method has a certain sparsity advantage.
6.2 THE INFLUENCE OF THE CONTRACTION BOUNDARY VALUE IN EIHL
As shown in the right Fig. the current sparsity increases with the gradual increase of b. Therefore, in the second experiment, we set different increment steps of b to control the sparsity of the network. The experimental results are shown Tab. 2 When step = 0.125, the accuracy of Resnet18 is 90.00%, and the accuracy of VGG11 is 86.06% which is still higher than the accuracy of STBP and STDP. However, because the step size is small, the final spansity of the network is 6.65% and 4.75%.
When slep 4, the accuracy of the two models is 74.53% and 84.66%, and the sparsity is 80.77% and 48.68% Too large a step size will cause some damage to the accuracy of the model. It can be seen that the step size of b basically proportional the sparsity and inversely proportional the accuracy.
Although the accuracy the highest for the Resnet18 model and CIFARIO dataset when the incre- ment step b is 0.5, the accuracy changes are not significant between the steps of 0.125, 0.25, and 0.5, and the change does not exceed 0.25%. Therefore, from an overall perspective, the increment step of b is still proportional to the sparsity and inversely proportional to the accuracy. Moreover, this experiment also shows that the neural excitation-inhibic mechanism also bring an absolute sparsity advantage to EIHI.
Table Evaluation of sparsity and accuracy on CIFARIO between difference increment step of b.
SNN Model
Spiking Resnet18
Spiking VGG11
Value of b Sparsity.(%)
Ascuracy.
Sparsity.(9)
Accuracy
0.125
6.65
90.00
4.75
86.06
0.25
11.58
90.02
R.31
85.49
0.5
17.17
90.25
13.10
85.75
1,000
30.31
89.61
20.28
85
2,000
44.62
89.08
28.42
85.77
4,000
80.77
74.53
84.66
According to the experimental results, EIHL can control the sparsity of the network structure by the increment step of and has stronger sparsity advantage than the separate local learning STDP and the separate global learning STBP.
6.3 COMPARISON THE PERFORMANCE OF THE MODEL ON THE DIFFERENT DISCONNECT DEGREE
In the third experiment, we further verified the robustness and sparsity advantage of EIHL by ran- domly pruming the network with different disconnect degrees, For example, the 20% random pruning experiment is randomly select 20% of the weights in each layer and set them to 0 at the initial stage. And keep them and their gradients as O in the subsequent learning. As illustrated in Tab under 20% and 40% pruning proportion, the accuracy of EIHL still exceeds that of STBP and STDP. For example, under 40% pruning, the accuracy of the Resnet18 model on EIHI. is 0.36% higher than STBP and 11. .999% higher than STDP, and the sparsity 29.73% higher than the other two methods. Although under the Resnet18 model, when pruning 60%, the accuracy of EIHL is no longer higher than STBP, that is because the sparsity of EIHL has reached 87.26%, and the accuracy only drops by 0.26% compared with STBP with 60% sparsity.
This experimental result shows that, under the same pruning level, EIHL performs better than the separate local learning STDP and the separate global learning STBP in the aspects of both accuracy and sparsity. It also proves that EIHL has certain robustness, as the disconnect degrees increase, the EIHL. keeps higher accuracies than the other separate methods.
8

---

Published as a conference paper at ICLR 2024
Table Evaluation of sparsity and accuracy on CIFARIO between different disconnect degrees.
Spiking Resnet
20%
40%
60%
(parsity:5e)
Accuracy
Spansity:(5))
(ccuracy(55)
SParsity.
Accuracy
STDP EIHL
20.00
72.
45.24
89.24
69.73
87.79
87,26
83.79
Spiking VGG11
20%
40%
60%
STDP EIHL
20.00
74.72
40.00
72.25
60.00
69. 14
33.64
85:62
50:10
84.36
74.67
82.93
6.4 COMPARISON THE PERFORMANCE OF MODEL WITH OTHER HYBRID LEARNING METHODS
In the last experiment, we conducted a performance comparison experiment between EIHL and Axcitatory-Inhiiitor Cooperative Iterative Learning (EICIL) (Shao et al. (2024). EICIL is hybrid training method that simulates the excitatory and inhibitory behaviors of biological neurons and seamlessly integrates them into the training process of Spiking Neural Networks (SNNs). It also has performance. EICIL proposes two training methods: the iteration using the Surrogate Gradient Method method and STDP-BW(whi)) incorporates the backpropagation technique into the STDP model) method as GSI, and the iteration using the GS method and STDP-BW-GS method as GSGI.
Table 4: Evaluation of sparsity and accuracy on CIFAR1O between the EIHL and EICIL.
SNN Model
Spiking Resnet18
Spiking VGGI1
Value of b Sparsity.(S))
Accuracy.
Sparsity.(9)
Accuracy:
GSI GSGI FTH..
0.00
89.32
0.00
85.63
0.00
88.95
0.00
85 36.66
17.17
90.25
13.10
85.75
As shown Tab. 4 EIHL's accuracy is higher than GSI and GSGI under both models. Especially on the Spiking resnet18 model, EIHL's accuracy is 0.93% higher than GSI and 1.3% higher than GSGI. And compared to GSI and GSGI, our EIHL shows a unique sparsity advantage.
7 CONCLUSION
We propose Excitement-Inhibitionn Mechanism assisted Hybrid learning method (EIHL) for spik- ing neural networks (SNNs). Benefitting from the automatic harmonization of the excitement- inhibition mechanism, the hybrid learning method with local and global learning is designed to extend the learning scenario of SNNs, which can automatically adjust the network connection sta- according to the LTP and LTD principles and then switches between local learning and global learning modes according to different network connection states. Hence, the proposed EIHL method overcomes the drawback of current hybrid learning methods that fail take advantage of the low energy consumption of local learning and the high accuracy of global learning. Experimental re- sults demonstrated that the SNNs model trained by the proposed EIHI method has better accuracy, certain sparsity, and strong robustness compared with other common-used methods.
As to the future work, the proposed method needs to be evaluated on large-scale datasets Rus- sakovsky al. (2015) and other network architectures (Sengupta et al. (2019a) Mostafa (2017) on GPU and hardware platforms, as EIHL can achieve strong sparsity and have significant potential once deployed on hardware platforms Roy al. (2019). What's 's more, other variants of local and global learning methods could be explored to improve the performance EIHL further Zhang et al. 2021
Lu & Sengupta (2023).

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGE
This work is supported by National Key R&D Program of China (2021ZD0109803) National Natural Science Foundation of China (NSFC No.62206037, 62306274), National Science Fund for Distinguished Young Scholars (NSFC No.61925603). The Open Project of Anhui Provincial Key Laboratory Multimodal Cognitive Computation, Anhui University, No. MMC202 104, un- der Grant No. GML-KF-22-1 and the Fundamental Research Funds for the Central Universities (DUT2IRC())91).
REFERENCES
Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78(10): 1550-1560, 1990.
Wickliffe Abraham and Mark Bear. Metaplasticity; the plasticity of synaptic plasticity. Trends in neurosciences, 1964):12--130, 1996.
Agatonovic- Kustrin and Rosemary Beresford. Basic concepts of artificial neural network (ann) modeling and its application in pharmaceutical research. Journal of pharmaceutical and biomed- ical analysis, 22(5):717-727, 2000.
Craig H Bailey, Maurizio Giustetto, Yan- You Huang, Robert D Hawkins, and Eric R Kandel. Is het- erosynaptic modulation essential for stabilizing hebbian plasiticity and memory. Nature Reviews Neuroscience, 1(1): 2000.
Guo-qiang Bi and Mu-ming Poo. Synaptic modifications in cultured hippocampal neurons: depen- dence on spike timing. synaptic strength, and postsynaptic cell type. Journal of neuroscience, 18 124):10464-10472, 1998.
Wei Fang. Yanqi Chen, Jianhao Ding, Zhaofei Yu, Timothée Masquelier, Ding Chen, Liwei Huang, Huihui Zhou, Guoqi Li, Yonghong Tian, al. Spikingjelly. 2020. Accessed: YYYY-MM-DD.
com/
Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew Davison, Jörg Conradt, Kostas Daniilidis, and Davide Scaramuzza. Event-based vision: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence, 44(1):154-180, 2022. doi: 10. 10//PPAMI.2220030044111
Wulfram Gerstner, Marco Lehmann, Vasiliki Liakoni, Dane Corneil, and Johanni Brea. Eligibility traces and plasticity on behavioral time scales: experimental support of neohebbian three-factor learning rules. Frontiers in neural circuits, 12:53, 2018.
Bing Han, Gopalakrishnan Srinivasan, and Kaushik Roy. Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network. Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 13558-13567, 2020.
Donald Olding Hebb. The organization of behavior: neuropsychologica theory. Psychology press, 2005.
Yangfan Hu, Huajin Tang, and Gang Pan. Spiking deep residual networks. IEEE Transactions on Neural Networks and Learning Systems, 2021.
Masao Ito. Long-term depression. Annual review of neuroscience, 12(1):85-102, 1989.
Felix Benjamin Kern and Zenas Chao. Short-term neuronal and synaptic plasticity act in synergy for deviance detection in spiking networks. PLOS Computational Biology. 19(10):1011554, 2023.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
10

---

Published as a conference paper at ICLR 2024
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: event-stream dataset for object classification. Frontiers in neuroscience, 11:309, 2017a.
Xuesong Li, David Steffens, Guy Potter, Hua Guo, Sen Song, and Lihong Wang. Decreased Setween-hemisphere connectivity strength and network efficiency in geriatric depression. Human brain mapping, 38(1):33-66 67, 2017b.
Qianhui Liu, Gang Pan, Haibo Ruan, Dong Xing, Qi Xu, and Huajin Tang. Unsupervised aer object recognition based multiscale spatio-temporal features and spiking neurons. IEEE transactions on neural networks and learning systems, 31 31 (12):5300-3311 2020.
Sen Lu and Abhronil Sengupta. Deep unsupervised learning using spike- spike-timing liming-dependend plastic- ity. arXiv preprint 4rX1v:2307.04055, 2023.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models.
Neural networks, 10(9):1659-1671, 1997.
Robert C Malenka, Nicoll, and Roger A. Long-term potentiation-a decade of progress? Science, 285(5435):887-18777 1999a.
Robert C Malenka, Nicoll, and Roger A. Long-term potentiation-a a decade of progress? Science, 28565435) 1999b.
Stephan Mandt, Matthew Hoffman, and David Blei. Stochastic gradient descent as approximate bayesian inference. arXiv preprint arXiv:1104.0422: 2017.
Hesham Mostafa. Supervised learning based on temporal coding in spiking neural networks. IEEE transactions on neural networks and learning systems, 29(7):327-3325, 2017.
Evan C Rosenberg. Simon Chamberland, Michael Bazelot, Erica R Nebet, Xiaohan Wang, Sam McKenzie. Swati Jain, Stuart Greenhill, Max Wilson, Nicole Marley, et al. Cannabidiol mod- ulates excitatory-inhibitoy ratio to counter hippocampal hyperactivity. Neuron, 11828):1282- 1300, 2023.
Kaushik Roy. Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575(7784)607--617, 2019.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei- Fei. Imagenet large scale visual recognition challenge. Int. Comput. Vision 115(3): 211-252, dec 2015. doi: 10. 10.1007/s11288 s -11263-015-888
URL https:////// htt.diioorggo. 10 1007/
Sadra Sadeh and Claudia Clopath. Inhibitory stabilization and cortical computation. Nature Reviews Neuroscience, 22(1):21-37, 2021.
Abhronil Sengupta, Yuting Ye, Robert Wang. Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in neuroscience, 13:95, 2019a.
Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in neuroscience, 13:95, 2019b.
Zihang Shao, Xuanye Fang, Yaxin Li, Chaoran Feng, Jiangrong Shen, and Qi Xu. Eicil: Joint excitatory inhibitory cycle iteration learning for deep spiking neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Thomas B Shea. An overview of studies demonstrating that neuronal networks display multiple complex behaviors: Emergent properties of neirest-neighhor interactions of excitatory and in- hibitory neurons. The Open Neurology Journal, 15(1), 2021.
Timothy A Simeone and JM Rho. Antiepileptic drugs: antiepileptic drug mechanisms. In Encyclo- pedia of basic epilepsy research, pp. 59-66. Elsevier Inc., 2009.
Sen Song, Kenneth D Miller, and Larry Abbott. Competitive hebbian learning through spike- timing-dependent synaptic plasticity. Nature neuroscience, 3(9):919-928 2000.

---

Published as a conference paper at ICLR 2024
Qingyang Wang, Michael A Powell, Ali Geisa, Eric Bridgeford, and Joshua T Vogelstein. Why do networks have inhibitory/negaatiee connections? arXiv preprint arXiv:2208.03211, 2022.
Xiao-Jing Wang. Macroscopic gradients of synaptic excitation and inhibition in the neocortex.
Nature Reviews Neuroscience, 2163):169-178, 2020.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience, 12:331, 2018.
Yujie Wu, Rong Zhao, Jun Zhu, Feng Chen, Mingkun Xu, Guoqi Li, Sen Song, Lei Deng, Guanrui Wang, Hao Zheng, et al. Brain-inspired global-local learning incorporated with neuromorphic computing. Nature Communications, 13(1):65, 2022.
Qi Xu, Jianxin Peng, Jiangrong Shen, Huajin Tang, and Gang Pan. Deep covdensesnn: A hier- archical event-driven dynamic framework with spiking neurons in noisy environment. Neural Networks, 121:512-519, 2020.
Qi Xu, Yaxin Li, Xuanye Fang, Jiangrong Shen, Jian Liu, Huajin Tang, and Gang Pan. Biologi- cally inspired structure learning with reverse knowledge distillation for spiking neural networks. arXiv preprint arXiv:2304.099: 09500, 2023.
Qi Xu, Yuyuan Gao, Jiangrong Shen, Yaxin Li, Xuming Ran, Huajin Tang, and Gang Pan. Enhanc- ing adaptive history reserving by spiking convolutional block attention module in recurrent neural networks. arXiv preprini urXiv:2401.037/1 2024.
Mingshan Xue, Bassam V Atallah, and Massimo Scanziani. Equalizing excitation-inhibition ratios across visual cortical neurons. Nature, 511(7511):596 600, 2014.
Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural computation, 3066):111411411, 2018.
Friedemann Zenke, Wulfram Gerstner, and Surya Ganguli. The temporal paradox of hebbian learn- ing and homeostatic plasticity. Current opinion in neurobiology, 16666,6 2017.
Tielin Zhang. Shuncheng Jia, Xiang Cheng, and Bo Xu. Tuning convolutional spiking neural net- work with biologically plausible reward propagation. IEEE Transactions on Neural Networks and Learning Systems, 33(12):7621-7631 2021.
Zijun Zhang. Improved adam optimizer for deep neural networks. In 2018 IEEE/ACM 26th inter- national symposium on quality of service (IWQoS), pp. 1-2. leee, 2018.
Guibo Zhu, Zhaoxiang Zhang, Xu-Yas Zhang, and Cheng-Lin Liu. Diverse neuron type selection for convolutional neural networks. In IJCAI, pp. 3566-3366, 2017.
12


Reviews:
[{"review_id": "ibz9oC318r", "reviewer": "Reviewer_iXWZ", "summary": "The authors proposed an Excitation-Inhibition Mechanism-assisted Hybrid Learning (EIHL) algorithm for training Spiking Neural Networks (SNNs), a learning algorithm that hybrid local learning rule and global learning rule. \nExperiments on CIFAR10/100 and DVS-CIFAR10 showed that EIHL outperforms other methods in terms of accuracy and sparsity.", "questions": "1. In the context of the hybrid learning rule, what is the significance of excitatory and inhibitory synapses, given that STDP and STBP do not appear to rely on the distinction between these synapse types? Furthermore, it seems that excitatory and inhibitory synapses are not typically delineated in deep spiking neural networks.\n\n2. Weight pruning is a technique employed in deep learning to increase network sparsity by eliminating the smallest weights. Please elucidate the distinctions between the 'STDP' process in EIHL and weight pruning techniques in deep learning.\n\n3. Kindly review the terminology and references utilized in the manuscript to ensure a more precise and coherent presentation of the research study.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Fusion of Global and Local Learning Rules: According to the authors, the integration of both global and local learning paradigms could potentially pave the way for attaining enhanced performance and energy efficiency in neural networks.\n\n2. Performance on CIFAR Benchmark: The authors successfully demonstrated an improvement in performance on the CIFAR dataset when compared to the traditional backpropagation technique.", "weaknesses": "1. The authors should elucidate their contributions more explicitly in order to provide a comprehensive understanding of the research.\n\n2. In the context of hybrid learning, 'STDP' process employs a contraction curve to facilitate Long-Term Depression. Nevertheless, the authors have not adequately expounded upon the association between LTD and STDP, and the proposed method do not have a dependence of the spike timing. It's not clear why excitation should be like STBP and depression should be like STDP.\n\n3. The accuracy of references should be ensured. For example, the paper states, \"Spike-Timing Dependent Plasticity (STDP) was proposed based on these rules by Caporale & Dan (2008).\" However, the discovery of STDP predates 2008. Caporale & Dan (2008) is a review paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8uwyFPWoCP", "reviewer": "Reviewer_UpZe", "summary": "This paper proposes an Excitation-Inhibition Mechanism assisted hybrid Learning (EIHL) algorithm. Inspired by the biological neural excitation-inhibition mechanism, it achieves adaptive adjustment of spiking neural network connectivity, and automatically alternates between global and local learning according to the growth or decay of synaptic strength which depends on the excitation-inhibition mechanism. It also conducts three experiments to demonstrate that this method has higher accuracy than global learning, and higher sparsity than local learning.", "questions": "I would like to ask the author, is the random pruning at different levels applied to the whole network or to a specific layer? And will the synapses that are randomly cut off at the beginning be restored in the later learning or remain disconnected? Maybe it should be explained in the third experiment.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper proposes a new hybrid method of local and global learning, which is regulated by the biological neural excitation-inhibition mechanism. The paper also argues that the excitation-inhibition mechanism leads to sparse results for neural networks, which is an innovative perspective. Moreover, the language and logic of the introduction and method description are clear and smooth. Finally, the paper presents EIHL as a new point in the field of SNN training methods, and also provides new insights for ANN training methods, which has some significance.", "weaknesses": "The paper conducted three experiments to verify the performance advantages of the method, but the details of the third experiment are less described. The third experiment should specify which layer or the whole network is randomly pruned at different levels, and also explain the specific operation of random pruning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kVMENvJBbF", "reviewer": "Reviewer_K674", "summary": "This paper proposes a hybrid learning method that uses the neural excitation and inhibition mechanism to assist local learning and global learning (called EIHL), by simulating the biological neural excitation and inhibition mechanism to adjust the network connection state, thus integrating local learning and global learning. The experimental results also show that this method has advantages in accuracy and sparsity compared to separate local learning and global learning.", "questions": "Is the EIHL method only applied to the convolutional layer? If not, I hope it can be reflected in the figure. I suggest updating and optimizing Fig.1.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper is inspired by the biological neural excitation-inhibition mechanism and proposes a new hybrid learning method for SNN, which is more brain-like than previous methods and has originality. Moreover, this paper has some practical significance from both the biological perspective and the accuracy and sparsity of the experimental results. Furthermore, the whole paper is logically coherent and fluent, and the language is concise and clear.", "weaknesses": "The Fig.1 in this paper that explains the EIHL algorithm is too simple and not detailed enough, only showing the connection processing between the convolutional layer and the IF neuron layer. This figure could consider adding another layer to show the specific operation of the algorithm more finely.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "awQ6KH0Dlp", "reviewer": "Reviewer_p6P6", "summary": "This paper proposes the Excitation-Inhibition Mechanism-assisted Hybrid Learning (EIHL) algorithm for training spiking neural networks. The algorithm combines the global learning and local learning together, and achieves better results than individual learning rules, as demonstrated through the experiments.", "questions": "Could the authors provide a more comprehensive explanation of the excitation-inhibition mechanism and how it is used in this work? Although the authors keep mentioning it, the reviewer cannot understand how excitatory and inhibitory synapses are handled differently and how they are balanced.", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "The method is inspired by biological mechanisms. Researchers in the SNN community should be encouraged to seek inspiration from neuroscience.", "weaknesses": "1.The authors do not provide a clear and comprehensive description of their method in section 4. The review raises several concerns, outlined as follows:\n\n (i). What does the notation $x$ mean? Is it a global parameter for the entire network? How does it \"gradually increase\" during the STDP period and \"decay by itself\" during the STBP period?\n\n (ii). What is the precise formula for the operation $thresh(\\cdot)$? It appears that $b$ remains unchanged during the STDP period based on Alg. 1. Does this imply that $thresh(b)$ remains constant in each STDP period?\n\n (iii). Could the authors provide further elaboration on Eq. 7? Why is gradient descent necessary for updating the weight in the STDP period? How is dL/dx computed, given that L does not seem to be differentiable with respect to x? Also, why do x and W share the same dimension? The reviewer thinks that it should be a scalar. Additionally, why is the \"contraction factor\" $a$ needed?\n\n (iv). Regarding alg. 1, how is the \"current sparsity\" calculated? Why is \"Curr S >= Pre S\" required in the algorithm?\n\n\n2.The contribution of this work is not very clear at the current stage. In the reviewer's understanding, this work aims at improving the current hybrid learning algorithms which \"have some theoretical and practical shortcomings and need further improvement\". However, there is no theoretical (mathematical) results in the paper and the biological plausibility of the proposed hybrid method is not inadequately clarified. Furthermore, the practical preformance is relatively unsatisfactory: \n\n(i). The improvement based on the global learning rule STBP is minimal. In essence, STBP represents a special case of the proposed EIHL with specifically chosen hyperparameters. Consequently, EIHL can consistently yield slightly superior results to STBP through randomness and careful hyperparameter tuning.\n\n(ii). The performance notably lags behind the latest research. Especially, the SOTA results of DVS_CIFAR10 is 20% better than the proposed method.\n\n(iii). There is no comparison between this work and other hybrid methods regarding accuracy and biological plausibility.\n\n(iv). No experiments on large-scale datasets.\n\nIn summary, the motivation and contribution of the paper remain ambiguous. The reviewer perceives this work as merely a combination of two exsiting methods without convincing reasons. \n\n3.This paper is not well-written. Several sentences lack coherence, making the overall presentation disjointed. The presentation of equations is arbitrary and non-standard. The resolution of Fig. 1 is low.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)