GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks
Authors: Xingting Yao
Fanrong Li
Zitao Mo
Jian Cheng
Venue: NeurIPS 2022 Accept (2022)
Keywords: Spiking Neural Networks
Leaky Integrate-and-Fire
Unified Spiking Neuron
Biological Features

Abstract: Spiking Neural Networks (SNNs) have been studied over decades to incorporate their biological plausibility and leverage their promising energy efficiency. Throughout existing SNNs, the leaky integrate-and-fire (LIF) model is commonly adopted to formulate the spiking neuron and evolves into numerous variants with different biological features. However, most LIF-based neurons support only single biological feature in different neuronal behaviors, limiting their expressiveness and neuronal dynamic diversity. In this paper, we propose GLIF, a unified spiking neuron, to fuse different bio-features in different neuronal behaviors, enlarging the representation space of spiking neurons. In GLIF, gating factors, which are exploited to determine the proportion of the fused bio-features, are learnable during training. Combining all learnable membrane-related parameters, our method can make spiking neurons different and constantly changing, thus increasing the heterogeneity and adaptivity of spiking neurons. Extensive experiments on a variety of datasets demonstrate that our method obtains superior performance compared with other SNNs by simply changing their neuronal formulations to GLIF. In particular, we train a spiking ResNet-19 with GLIF and achieve $77.35\%$ top-1 accuracy with six time steps on CIFAR-100, which has advanced the state-of-the-art. Codes are available at https://github.com/Ikarosy/Gated-LIF.

GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks
Xingting Yao!.2, Fanrong Li Zitao Mo', Jian Cheng 1.3+
Institute of Automation, Chinese Academy of Sciences SSchoo of Future Technology, University of Chinese Academy of Sciences $CAS Center for Excellence in Brain Science and Intelligence Technology fyaoxingting2020, lifanrong2017, mozitao2017, jian. cheng) ac.cn,
Abstract
Spiking Neural Networks (SNNs) have been studied over decades to incorpo- rate their biological plausibility and leverage their promising energy efficiency. Throughout existing SNNs, the leaky integrate-and-fir (LIF) model is commonly adopted to formulate the spiking neuron and evolves into numerous variants with different biological features. However, most LIF-based neurons support only single biological feature different neuronal behaviors, limiting their expressiveness and neuronal dynamic diversity. In this paper, propose GLIF, unified spiking neuron, to fuse different bio-features in different neuronal behaviors, enlarging the representation space of spiking neurons. In GLIF, gating factors, which are exploited to determine the proportion of the fused bio-features, are learnable during training. Combining all learnable membrane-relaee make spiking neurons different and constantly changing. thus increasing the hetero- geneity and adaptivity of spiking neurons. Extensive experiments on a variety of datasets demonstrate that our method obtains superior performance compared with other SNNs by simply changing their neuronal formulations GLIF. In particular, we train a spiking ResNet-19 with GLIF and achieve 77.35% top-1 accuracy with six time steps on CIFAR-I00, 100, which has advanced the state-of-the-art. Codes are available at httph:tt/ thug. com/
parameters, our method can
1 Introduction
Spiking neural network (SNN) is considered the third generation of the neural network, and the bionic modeling of its activation unit is called the spiking neuron, which brings about the spike-based communication between layers [ 2. Due to the high sparsity and multiplicatoon-fee operation in information processing, such spike-based communication can improve energy efficiency 3 4 Moreover, spiking neuron effectively capture temporal information and has good performance in processing neuromorphic data [5] and anti-noise [6]. Together with artificial neural network (ANN) colleagues, SNN further shows great potential for general intelligence M Thus, SNN has aroused heated research interests recently.
The commonly used spiking neuron in different SNNs 6
is the leaky integrate- and-fire (LIF)) model, named vanilla LIF. As the name suggests, can implement three different neuronal behaviors: membrane potential leakage, integration accumulation, and spike initiation. To better model neurons, prior works [33 14 [5] [6] [7] have improved the vanilla LIF from the above three aspects, and proposed many LIF variants with different biological features. For all these vanilla and variant LIF models (referred to as simplex LIF), any of the three neuronal behaviors supports
Corresponniig author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

S7-1
L'
F
1
U
U--1
Fine
U
60x
C²
(a)
(c)
Figure 1: Illustration of the GLIF model. (a) Discrete GLIF neuron. The membrane potential Ure and the output spike 5 are updated over discrete time steps. Gating units G.. G3. and G- fuse dual bio-features in membrane potential leakage, integration accumulation, and spike initiation, respectively, (b) Gating unit. (c) Gating factor.
only single biological feature. For instance, the vanilla LIF model only utilizes exponential decay in membrane potential leakage, while [5] only supports linear decay. However, different neurons in the cerebral cortex have different response characteristics [18] 199Thhis naturally raises an issue: could the higher neuronal dynamic diversity of spiking neurons help SNNs do better? In this paper, we investigate the hybrid bio-features in the LIF model answer this question.
We propose the gated LIF model (GLIF) that fuses different bio-features the aforementioned response characteristics. As illustrated in Fig. GLIF
three neuronal behaviors to possess controls the fusion different bio-features through gating units Ga. G3. and G., for those three neuronal behaviors that membrane potential leakage, integration accumulation, and spike initiation, respectively. each gating unit, gating factor is computed from a Sigmoid function axx over learnable gating parameter determine the proportion of each bio-feature, thus guiding the fusion of different bio-features. Therefore, GLIF can simultaneously contain different bio-features, possessing more response characteristics. In addition, when the gating factor is 0 or 1, GLIF can also support single bio-feature. As result, GLIF can cover other different LIF models and be viewed as super spiking neuron, greatly enlarging the representation space of spiking neurons.
Furthermore, we introduce the channel-wiss parametric method to GLIF. This method makes all membrane-related parameters in GLIF learnable and shares the same GLIF parameters channel- wisely in SNNs. Combining with Icarnable gating factors in GLIF, on the one hand, this method makes different channels in SNNs have completely different spiking neurons, leveraging the larger representation space of GL.F neurons to increase the neuronal dynamic diversity of spiking neurons. Meanwhile, the heterogeneity of spiking neurons and the expressive ability of SNNs are also increased. On the other hand, the spike neurons in SNNs are constantly changing during training, which is thus the adaptivity of spiking
similar the neuronal maturation during development (20 21 22 neurons being enhanced.
Our contributions are as follows:
We propose the GLIF model, unified formulation of the spiking neuron that fuses multiple biological features through gating units, enlarging the representation space of spiking neurons.
We exploit the channel-wise parametric method to leverage the larger representation space of GLIF, thus increasing the neuronal dynamic diversity in SNNs.
Our experiments demonstrate the effectiveness of the GLIF model on both static datasets and the neuromorphic dataset. For example, our model can achieve the ofate-of-the-art 75.48% top-1 accuracy CIFAR-10 with only two time steps.

---

2 Related Work
Variant LIF. Variant LIF models with different biological features are proposed in previous arts, and those models differ in one or more of the three different neuronal behaviors: membrane potential leakage, integration accumulation, and spike initiation. Firstly, membrane potential leakage represents the potential decay mechanism. Exponential decay is first exploited in the vanilla LIF model [23 and it brings neuronal dynamics with stable convergence, shrinking the potential to the resting potential steadily. But, exponential decay is not efficient enough in massive neural simulations due to its multiplication operation. Therefore, linear decay replaces exponential decay in [3 24 25 26, 27). and brings neuronal dynamies with the flexibility, making it possible for configurable spiking rates 11]. Although the two types of decay behave differently in neuronal dynamics, both of them are proven effective in SNNs 6 28). Secondly, integration accumulation represents the potential accumulation induced by presynaptic spikes, Commonly, spike intensities of different time steps are considered equally high. Consequently, the discrepancies among the spikes at different time steps are ignored. To address this issue, many prior works propose to assign different weights to the input spikes at different time steps. These methods are implemented by manual settings (29 30 31]
or adaptive settings [32] 33 [4] to improve biological plausibility, SNN performance, or inference efficiency. Lastly, spike initiation functions as the resetting mechanism triggered by spike firing.
There are two common resetting strategies: hard reset 335 and soft reset [5] 36 When a spike is triggered, the hard reset directly sets the potential to the resting potential, while the soft reset subtracts the potential by configurable value. Therefore, hard reset is more stable as strict reset mechanism, while soft reset is more flexible, making configurable spiking rates feasible [15. Compared with the LIF models above, our GLIF incorporates different neuronal behavioral mechanisms to increase the representation space of LIF models and improve the performance of SNNs.
Parametric spiking neurons. Early existing spiking neurons often require manual tuning of the membrane-related parameters to set their neuronal dynamics, and these parameters are often shared by all neurons, which limits the diversity of spiking neurons and the expressive ability of SNNs. In recent years, parametric spiking neurons (37 38
10 28 have been proposed, which set one or more
membrane-related parameters to be learnable, and update these parameters during the training of
SNNs. 37 and 38 set the membrane leak and potential threshold learnable, while [10] and [28]
choose the membrane leak only. FS-neuron [39 is more aggressive, setting all membrane-relane
parameters in the spiking neuron learnable to strictly fit the neuronal dynamics to the target activation function. Compared with the above methods, GI.IF adopts the channel-wise parametric method to
fully parameterize the spiking neuron, including learnable exponential decay, linear decay, potential threshold, resetting voltage, input conductance, and gating factors.
Supervised direct learning of SNNs. Supervised direct learning of SNNs follows the idea of backpropagation (BP) in ANN [40]. Bohte et al. [41] first adopt the surrogate gradient (SG) to empirically solve the non-differentiable term of spiking neurons by approximation with the local derivative of potential. Based on the observation that the proper steepness of the SG curve matters more than the shape in SNN learning 42). Li al. [1] conduct pre-estimation before each epoch to find the SG curve with the optimal smoothness. Besides, many techniques are developed to make deep SNNs converge faster, such as NeuNorm (8) tdBN (9. SEW block (43, TET [12, MS residual block [44], etc. In this way, the problem of end-to-end training of deep SNNs can be solved. In this paper, we choose supervised direct learning of SNNs to estimate the effectiveness of our GLIF
3 Methodology
3.1 Revisiting LIF models
Vanilla LIF The Vanilla LIF model is commonly adopted as spiking neuron model in SNNs, which can be discretely formulated as follows:
U (10 (1 - S(t-110)) + c(tD)
(1)
((t.)) W S(
(2)
((t.)) = ((())))) VaN)-
(3)
3

---

Here, @ denotes the clement-wise multiplication. U(!!) is the membrane potential vector of the layer/ I at time-step t and can be updated in discrete way through Eq. where C(t,) denotes the input vector and can be obtained by the dot-product between the synaptic weight matrix W and the output spike vector of the previous layer ((1.-1), as described in Eq. The output spike vector g(C.)) is given by the Heaviside step function HI(() in Eq. indicating that a spike is fired when the membrane potential exceeds the potential threshold ViA- When updating U(t.0) in Eq.I the three neuronal behaviors are considered. Specifically, the time constant T acts as the exponential decay coefficient on the term 71) implement exponential decay, C(t.)) is integrated over time to U(4) and the term (1 (()) hard resets the membrane potential to 0 if a spike is fired the previous time step.
Bio-features & Primitives. Bio-features represent the characteristies of the three critical neuronal behavior. For the instance of vanilla LIF, its bio-features can be described as exponential decay, uniform-codii coding scheme, and hard reset. And, the three bio-features qualitatively describe the neuronal behavior of membrane potential leakage, integration accumulation, and spike imitiation, respectively. Since each behavior is actually implemented by the corresponding term formulas, for explicimesss we refer to those neuronal behavior-relater parameters as primitives in this paper. In this way. bio- features can be quantitatively described by primitives. For example, exponential decay is realized by the term 7 TU(-1111) in Eq.(1. T is primitive and determines the amplitude of exponential decay. Thus, by choosing different primitives for each behavior, we can formulate different LIF variants. For instance, with the linear decay primitive Thi and the soft reset primitive Tre. we have: U(O U(t-1d)) Tlin Vr C(t,)) which formulates the spiking neuron in [5]
3.2 Gated LIF Model
Although variant LIF offers a different choice in modeling the spiking neuron, it still remains simplex, i... the bio featuressse of the neuronal dynamic are unitary. Once formulated, hard-reset LIF neuron cannot perform the soft-reset operation. As a result, these simplex LIF models make the spiking neurons in SNNs lack dynamic diversity. Thus, we propose the gated LIF (GLIF), unified spiking neuron, to fuse different bio-features in the three neuronal behaviors. GLIF is illustrated in Fig. and further described follows:
U(t.)) = (t.)) + (1.) + ((t.)) @ (t-1.)
(4)
((+))) W. S(
(5)
S((.)) = (U(()) Vin),
(6)
(L. = G. G. (U(; Min: Temp);
(7)
(0.)) Gg(C(t); 1),
(8)
F(t.)) = (I Vres 0).
(9)
In Eq. (t.l) denotes the membrane potential vector after membrane potential leakage, (t.l) denotes the vector of the incremental potential caused by integration accumulation, and F((.1) denotes the vector of the potential reduction caused by spike initiation. Obviously, the above three vectors reflect the consequences of the three neuronal behaviors. In Eq. (8))) the gating units (-), G.).), and (.).) are proposed to formulate the three neuronal behaviors, thus computing the consequent vectors.
Computing ((.)), In computing L (+.1), the two primitives Taan and Tezp are considered. Tinn is the linear decay primitive, and Teap is the exponential decay primitive. As their names indicate, The and Teap respectively play the roles of the linear decay and exponential decay on U(t-1,1), Through the gating unit G.).), the two primitives are fused together to perform membrane potential leakage. Following is the formulation Go (-):
G. (Ult- Thin, Teap) = [1 - a(1 Teapp) ( - (1 (1- a))
(10)
where or is the gating factor of G. (.), and its value is bounded within (0, 1). When a approaches
[1 a(1 Te and (1 11--ap a) tend to 1, which means GLIF performs linear decay. As increases

---

from to 1, (1 Tim decreases from Tin to O, while [1 aa(1 Tezp] decreases from to Tezp which means GLIF is shifting from linear decay exponential decay. Therefore, Eq.(TO), the two primitives Thin and Texp are dually fused with the help of the gating factor which can balance the amplitudes of linear decay and exponential decay. Note that the primitives in the same gating unit are referred to as the dual primitives in the following content, and correspondingly. the bio-features the dual primitives describe are referred to as the dual bio-features. In the extreme case of a 0 or 1, Ga preserves only single primitive, which means GLIF covers the simplex formulations of membrane potential leakage.
Computing (t..) In computing (tt.), the dual primitives g' and are considered to perform integration accumulation. g' denotes the conductance primitive, namely, the time-dependent synaptic weight, which can be coded with different patterns, such as rank-order based patterns [29] and phase-coding based patterns [31] So, we can view it the flexible-coding primitive. While, the constant / means g' = 1. representing the uniform uniform-coding primitive, which is the dominant paradigm and proven doing well in the direct learning of SNNs 45 12 (.):
Following is the formulation of
Gs(C 1)
11
(11)
where 3, bounded within (0, 1), is the gating factor of Gg(+). Similarly, as 3 O, the uniform-coding pattern is preferred rather than the dexible-coding, while 3 - 1 is the opposite case. When 3 = 0 or 1, GLIF supports single spike coding pattern in integration accumulation.
Computing F(t.), In computing F(r.1), the soft-reset primitive Vre and the hard-reset primitive 0 are coupled as the dual primitives to perform spike initiation. Following is the formulation of G, (-):
G. (t.0, exp
(1.)) Vre, 0) = (0 + L harp
(1 Vre
(12)
where 7, bounded within (0, 1), is the gating factor of G-(-). The term (1 > is the actual potential decrease caused by soft reset, while the term (0 + Leap is the actual potential decrease caused by hard reset. Note that (t.d) is given by [1-a[] a(1 Temp)] U in Eq. 10). which means GLIF only performs hard reset on the aanonential-deccy-relaeee part of the membrane potential. In a similar way, y - 0, soft reset is incrementally biased, while - 1, hard reset is incrementally biased. When y = 0 or 1, GLIF supports the simplex reset formulation of spike initiation.
Therefore, with gating units, the GLIF-based spiking neurons can fuse the dual bio-features in each one of the three neuronal behaviors. When gating factors are binary, GLIF transforms into the corresponding simplex LIFs. For example, in the case of a, 8, y 1, 0, 1, GLIF is equivalent to the vanilla LIF. In a nutshell, GLIF unifies different simplex LIFs by incorporating dual bio-features and exploits gating factors to balance the dual bio-features.
Channel- wise parametric method. The parametric method is applied to all the membrane-related parameters in our GLIF, including primitives (Tim, Teaps Vre. g², and Via) and gating factors (a, 3, and 7). Given x is one of these membrane-related parameters, we can obtain æ by the Sigmoid function:
1+expl ) where xp (-oc, +oo) is a learnable parameter that changes at each training iteration. With all these membrane-related parameters trainable, the adaptivity of the spiking neurons is increased. Such that, GLIF neurons can adjust their neuronal response characteristics in wide range during training.
Furthermore, prior SNNs commonly adopt the same membrane-relan parameters for all spiking neurons throughout the network, totally neglecting the heterogeneity. While in GLIF, the channel- wise sharing scheme, where spiking neurons in the same channel share unique setting of the membrane-related parameters but inter-channel spiking neurons don't, is employed.
With the channel-wise sharing scheme and the parametric method combined, we can greatly increase the heterogeneity of spiking neurons after training. The heterogeneity over channels and layers can be observed as we visualize the membrane-related parameters in the Appendix.
Training framework. In terms of the training framework, we utilize backpropagation through time (BPTT) the training algorithm for our GLIF SNNs. As for the non-differentiable Heaviside step
5

---

0.4
Patential InV
60
0.4-
62
p+ a.11
0.0
GUPINOI s. C.-00.0 f.y. 2...o..Va22....9. 2.2.99.5
guriLa =-0: 50.00.05
GUFIFOI a-05
SUrIED 2=1
20
25
30
25
(a)
(b)
GUTHIR) 1 s.d. -13,05
GUTtRRI V.-05 4,p.y-L11
Membrane Povertial my
0.4
GUrISNI Ny-000 y-d
GUPIONI -0-0
GURORR Ma-da
GURYEN Na-07
cUr)AR) y=2: K
6.3-
6
cunyii vC
0.5
0.2
05
63-
63-
0.5
20
25
25
5
15 20 25 (d)
(c)
Figure 2: Visualization of neuronal dynamics. (a) The potential trace with a being the variable to bridge the linear and exponential decay. (b) Upper: the potential trace with 3 being the variable to harness the amplitude of the cosine coding pattern. Lower: the cosine oscillation of conductance g.. (c) The potential trace with Vre being the variable to display possible scenarios of soft reset in comparison with the hard reset. (d) The potential trace with y being the variable to bridge the soft and hard reset. Note that ND represents don-y, LD linear decay. ED exponential decay, FD fused decay, SR soft reset, HR hard reset, and FR fused reset.
function in (), we solve it by the surrogate gradient defined as: dHl(z)
HI.0. - |x:]).
(13)
da
Thus, GLIF SNNs can be trained end to--dd.
3.3 Dynamic Analysis
GLIF contains dual bio-features, which brings great diversity to the neuronal dynamics. Here we use simple case to analyze the dynamic properties of GLIF. Fig. (2) illustrates the response of the spiking neurons to the constant input that spikes each time step until the membrane potential reaches the threshold. As shown in the figure, GLIF neurons with different parameter configurations have completely different dynamic responses to the same input.
Bridging the gap between exponential decay and linear decay. As shown in Fig. 2 a). compared with non-decay, on the one hand, exponential decay brings the neuronal dynamic with stability and astringency, making it increasingly difficult for the membrane potential climb over the threshold. However, the membrane potential falls into the saturation zone and fails to reach the potential threshold, which indicates exponential decay may impair the spiking responsiveness. On the other hand, linear decay operates equally at each time step to avoid saturation, thus maintaining the high spiking responsiveness. However, if there are no inputs, linear decay leads the potential decreasing in an unlimited way, which impairs the stability. While in GLIF, the above two types of decay are the extreme cases, i.e., GLIF only acts exponential decay a and only serves linear decay = O. In most cases where E (0. 1). GLIF is eligible for exponential decay's 's astringency and benefits from the high spiking responsiveness linear decay brings about. To some extent, the gating factor makes the two types of decay complementary and determines the proportion of each of the two decay, bridging the gap between the neuronal dynamics of exponential decay and linear decay and enlarging the representation space of membrane potential leakage. Furthermore, since Teap and Tün are learnable GLIF, the gap berween exponential and linear decay becomes more flexible. Therefore, the actual representation space given by Tezps Thn and a could be even larger.

---

Incorporating time-dependent synaptic weights. In GLIF, we assign the time-dependent synaptic weights g to the spike train to differentiate spike stimulation levels at different time steps. Since g' is learnable, the flexible-coding patterns of different GLIF neurons may differ significantly. For the convenience of analysis, we add constraints to g". assuming that g is in the form of cosine oscillation, where B controls the amplitude. As shown in Fig. b when B = 0, the uniform-codlig pattern takes effect. Each time the spiking neuron receives an imput spike, the membrane potential increases by the same amount. Our GLIF is different because the flexible-coding is also incorporated. When E (0, 1). each time the GLIF receives an input spike, the membrane potential responds differently depending on the partieular stimulation level of the current input spike. On the one hand, considering that there is no constraint on g' in practice, GLIF is more flexible, and the representation space is further enlarged. On the other hand, because the stimulus of input spikes at different times considered differently, incorporating time-dependent synaptic weights may also help to better process temporal features.
Bridging the gap between hard reset and soft reset. Fig. 2 compares the neuronal dynamics between soft reset and hard reset. As we can see, soft reset reduces the current membrane potential by Vre. Since Vr is not related to the current membrane potential, soft reset may lead to over-resetting or under-resetting of membrane potential, which is flexible for the spiking neurons. Hard reset reduces the current membrane potential to an empirical value, e.g., O, restarting the potential trace all over, providing the neuronal dynamies with stability. While GLIP, we consider the potential reduction caused by spike initiation to be both related to Vre and the current membrane potential. Because Y E (0, 1) balances the amplitudes of the two resets, GLIF can reset the membrane potential to any possible levels between the consequent potentials of hard reset and soft reset as shown in Fig. d), enlarging the representation space spike initiation.
Experiments
In this section, we demonstrate the effectiveness of our proposed GLIF model. We first compare results with other existing state-of- the-arttttt arts, then carry out ablation studies to evaluate different aspects of our proposed method.
4.1 Experimental Settings
We perform experiments on two kinds of datasets. The first is image classification datasets, including CIFAR [46) and ImageNet [47). The other one is the event-based dataset, i.e., CIFAR10-DVE [48), which is converted from popular image recognition dataset CIFAR- 10 using DVS camera. For fair comparisons with other methods, we choose the commonly used network architectures in existing works for experiments, including ResNet-family 49 9 44 7B-wideNet 43). and CIFARNet
42 g 50) and apply tdBN (9. More implementation details of training those SNNs are revealed in the Appendix.
4.2 Comparison with Existing Works
CIFAR. We test GLIF on CIFAR-10 and CIFAR-100 datasets and run three times for each experiment to report the "mean + std". As shown in Table T our GLIF SNNs can outperform existing works. Specifically, on CIFAR-10, we first test GLIF a smaller network CIFARNet and achieve the superior accuracy over other methods 42 g 50 even with fewer time steps. Since tdBN is employed in our GLIF SNNs, then compare GLIF with STBP-tdBN [1]] on ResNet- 19. By simply changing the neurons in STBP-tdBN to GLIF, our SNNs can achieve significant improvement across all the tested time-steps. Compared with the state-of-the-art TET [12], method still obtain better accuracy, and our ResNet-19 19 with time-step 4 even outperforms theirs with time-step by 0.35%. On CIFAR-100. our method can also achieve much better performance than STBP-tdBN and TET. We also compare GLIF ResNet-18 with the highly handcrafted ResNet- 18 of Dspike [M. As listed in Table our GLIF ResNet-18 can outperform Dspike ResNet- by at least 0.63% and 2.92% absolute accuracy on CIFAR-10 and CIFAR-100, respectively.
ImageNet. We apply GLIF to ResNet-34 and MS-ResNet-18 18 441 Table 2 shows the results on the validation set. We first compare our method with STBP-tdBN, which can be viewed our
7

---

Table 1: Comparisons with existing works on CIFAR datasets.
CIFARIO
CIFARI00
Method
Architecture
TimeStep
Accuracy
TimeStep
Accuracy
STBP 42
CIFARNet
12
89.83
STBP NeuNom
CIFARNet
12
90.53
TSSL-BP 50
CIFARNet
5
91.41
6
93.16
71.12::0.57
STBP-1dBN 9
ResNet- 19
92.34 4
92.92
70.86+00.22
69.41+00.08
6
94.50+H0.07
74.72+00.28
TET 12
ResNet-19
94.44:10.08 4
74.47+t0.15
94.16+t0.03 2
72.87+0.110
6
94.25
74.24
Dspike
ResNet-18*
93.66:+0.05
73.35+t0.14
93.13 2
2
71.68:00.12
CIFARNet
93.28 5
6
95.03
77.35+0.077
ResNet-19
4
94.85100007
77.05=10.14
GLIF
94.44110110 2
75.48
6
94.88+10.15
77.28+10.14
ResNet-18
4
94.67:10.05
76.42:10.06
2
94.15:10.04
74.60-10.24
* denotes highly handcrafted network.
Table 2: Comparisons ImageNet.
Table 3: Comparisons on CIFARIO-DVS.
Method
Architecture
T
Acc.
Method
Architecture
T
Acc.
TET
[12
ResNet-34
6
64.79
STBP-tdBN 99]
ResNet-19 19
40
67.80
Dspike
11
ResNet-34
6
68.19
STBP-ddNN
9
ResNet-34
6
63.72
LIAF (28)
LIAF-Net
10
70.40
SEW+PLIF
43
SEW-ResNet-34
4
67.04
LIAF+TA [33
TA-SNN-Net
10
72.00
MS-ResNet
44
MS-ResNet-18
6
63.10
PLIF [0
PLIF-Net
20
74.80
ResNet-34
6
69.09
SEW+PLIF 43
7B-wideNet
16
74.40
GLIF
ResNet-34
4
67.52
GLIF
7B-wideNet
16
78.10
MS-ResNet-ll
6
68.11
T denotes the time step.
T denotes the time step.
baseline, ResNet-34, and observe that our GLIF SNNs achieve better performance (69.09% v.s. 63.72%) Since parametric technology is employed our GLIF, we also compare GLIF with the parametric spiking neuron PLIF Without changing the model structure, our method can still achieve higher accuracy than PLIF on ResNet-34 (67.52% v.s. 67.04%). Moreover, compared with MS-ResNet 44 whose handcrafted residual block achieves the state-of-the- art performance, we apply our GLIF to MS-Res ASS-ResNet-1 8 and can also significantly increase the accuracy (68. 11% v.s. 60.10%) These results demonstrate the effectiveness of our proposed GLIF model. In addition. compared with other SNNs in TET [12 and Dspike [T]] GLIF SNNs can also obtain higher accuracy.
44
To demonstrate the effectiveness of GLIF in processing spatio-temporal features, we conduct experiments on the event-based dataset CIFARIO-DVS. Since SEW+PLIF 43 is the closest to our work for it also exploits parametric methods and tdBN, we use the same network architecture Ne-wideNet SEW+PLIF. As shown in Table 3 GLIF outperforms SEW-PLIF along
CIFARIO-DVS.
with other methods PLIF [10. STBP-tdBN [9] and LIAF 28 which demonstrates the effectiveness of the proposed GLIF on the event- based dataset.

---

Table 4: Comparisons with different simplex LIFs.
Model
000
001
010
011
100
101
110
111
GLIF
ResNet-19
76.29
76.59
73.89
74.71
77.15
76.26
74.39
74.68
77.22
ResNet-18
75.77
76.05
74.27
74.59
75.54
75.51
73.36
74.49
76.39
Table Ablation study of gating factors.
Table 6: Comparisons with Layer-wise GLIF,
Model
GLIF s
GLIF f GLIF
Model
T
Jayer-Wise
GLIF
76.89
71.26
77.22
4
75.71
77.22
ResNet-19
ResNet-19
75.27
72.14
75.54
2
74.19
75.54
76.22
72.60
76.39
4
75.29
76.39
ResNet-18
ResNet-18 18
74.24
72.10
74.85
2
73.40
74.85
T denotes the time step.
T denates the time step.
4.3 Ablation Study
Effectiveness of different bio-features. To demonstrate the effectiveness of different bio-features that are incorporated in our GL.F, we evaluate the performance of GLIF SNNs extreme cases where GLIF only acts single bio-feature in each neuronal behavior. Since the gating factors control bio-features, we can obtain eight simplex LIFs by combining different gated biometrics, all which have static binary gating factors. We name those simplex LIFs with the values of their gating factors, e.g., 101 is the GLIF with a, 3,7 = 1, that only acts exponential decay, the uniform-coding pattern, and hard reset. On CIFAR-100, we use ResNet-18 and ResNet-19 19 with time-step 4 for experiments. As listed in Table 4 our proposed GLIF outperforms all of those simplex LIFs, which demonstrates the effectiveness of fusing different bio-features.
Importance of learnable gating factors. To demonstrate the effectiveness our proposed learn- able gating factors in fusing bio-features, compare GLIF with other two GLIF variants: GLIF and GL.F_ where GLIF_s keeps gating factors static after initialization, GLIF _ removes gating- factors and simply fuses two different bio-features by addition. On CIFAR-100,
ResNet- -8 and ResNet-19 with time-step 2 and 4. Table 5 lists the experimental results. On the one hand, compared with GLIF S. our method can get higher accuracy, which demonstrates that learnable gating factors can make GLIF better. On the other hand, compared with GLIF f, GLIF can also achieve better performance, this because learnable gating factors can balance dual bio-features in each neuronal behavior. As result, learnable gating factors are important in GLIF.
Channel-wise v.s. Layer-wise. Besides the investigations above, we conduct an ablation study on GLIF with different parameter-sharing schemes, which are layer-wise sharing and channel-wise sharing. We also use ResNet-18 and ResNet- with time-step 2 and 4 on CIFAR-100. listed in Table 6 GLIF, which leverages channel-wise sharing scheme, consistently improves the performance over the layer-wise sharing scheme.
Conclusion
In this paper, we propose GLIF, a unified spiking neuron, that fuses the bio-fcatures in each neuronal behavior. In GLIF, our method exploits the gating factor to bridge the gap between the dual bio- features, enlarging the representation space. Furthermore, we adopt the channel-wise parametric method make all membrane-relaee related parameters learnable, and train those parameters through BPTT, increasing the neuronal adaptivity of spiking neurons and enhancing the neuronal dynamic diversity of spiking neurons. the sate-of-the-are performance compared with other methods. Therefore, we think GLIF could be positive answer to the issue: could the higher neuronal dynamic diversity of spiking neurons help SNNs do better?
validate our GLIF SNNs on various benchmark datasets, showing

---

Additionally, the distributions of learned parameters are interesting as we visualize them in the Appendix. The initially identical parameters learn into different bell-shaped distributions layer-wisely, We think this could be further studied to mine the biological plausibility of SNNs. How to apply GLIF in other training frameworks, e.g., ANN2SNN conversion, also remains a worth-studying problem.
Acknowledgments and Disclosure of Funding
This work was supported in part by the National Key Research and Development Program of China (Grant No. 2021ZD0201504), the Strategic Priority Research Program of Chinese Academy of Science (N0.XDB32050000) Jiangsu Key Research and Development Plan (No.BE2021012-2),
References
[1] Wolfgang Maass. Networks of spiking neurons: the third generation neural network models. Neural networks, 1099):655-1661, 1997.
[2] Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575(7784):607-617, 2019.
[3] Mike Davies, Narayan Srinivasa, Tsung- Han Lin, Gautham Chinya, Yonggiang Cao, Sri Harsha Choday. Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, al. Loihi: neuromorphic manycore processor with on-chip learning. leee Micro, 38(1):82-99) 2018.
[4] Ziru Li, Bonan Yan, and Hai Li. Resipe: Reram-based linle-spiking processing in memory engine. In 2020 57th ACM/IEFF Design Automation Conference (DAC), pages 1-6, IEFE, 2020.
[5] Weihua Hc, YuJie Wu, Lei Deng, Guoqi Li, Haoyu Wang, Yang Tian, Wei Ding, Wenhui Wang, and Yuan Xie. Comparing sans and mns on neuromorphic vision datasets: Similarities and differences. Neurul Networks, 132: 108- 2020.
[6] Xiang Cheng, Yunzhe Hao, Jiaming Xu, and Bo Xu. Lisun: Improving spiking neural networks with lateral interactions for robust object recognition. In IJCAI, pages 1519-1525 2020.
[7] Jing Pei, Lei Deng, Sen Song, Mingguo Zhao, Youhui Zhang, Shuang Wu, Guanrui Wang, Zhe Zou, Zhenzhi Wu, Wei He, al. Towards artificial general intelligence with hybrid tianjic chip architecture. Nature, 572(7767): 111, 2019.
[8] Yujie Wu, Lei Deng, Guoqi Li, Jun 7hu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI Conference on Artifciia Intelligence, volume 33, pages 1311-13188 2019.
[9] Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with trainnd larger spiking neural networks. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pages 11062 11070, 2021.
[10] Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. Incorporating leamable membrane time constant to enhance learning of spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2661-2671, 2021.
[11] Yuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng, Yongqing Hai, and Shi Gu. Differentiable spike: Rethinking gradient-desse for training spiking neural networks. Advances in Neural Information Processing Systems, 34, 2021.
[12] Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu. Temporal efficient training of spiking neural network via gradient reiihighti In International Conference on Learning Representations, 2021.
[13] Andres Upegui, Carlos Andrés Pena-Reyes, and Eduardo Sanchez. An fpga platform for on-line topology exploration of spiking neural networks. Microprocessors and microsystems, 29(5):21 11-233, 2005.
[14] Benjamin Schrauwen, Michiel D' Haene, David Verstraeten, and Jan Van Campenhout. Compact hardware liquid state machines on fpga for real-time speech recognition. Neural networks, 21(2-3):51--22 2008.
[15] Peter U Diehl, Bruno Pedroni, Andrew Cassidy, Paul Merolla, Emre Neftci, and Guido Zarrella. Truchappiness: Neuromorphic emotion recognition on truenorth. In 2016 international joint conference on neural networks (tjenn), pages 4278 4285. IEEE, 2016.
10

---

[16] Peter U Diehl, Guido Zarrella, Andrew Cassidy, Bruno U Pedroni, and Emre Nefici. Conversion of artificial recurrent neural networks spiking neural networks for low-power neuromorphic hardware. In 2016 IEEE International Conference on Rebooting Computing (ICRC), pages 1-8. IEEE, 2016.
[17] Scongsik Park, Seijoon Kim, Byunggook Na, and Sungroh Yoon. T2fsmn: deep spiking neural networks with time-tofirst-spikee coding. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1-6. IEFE, 2020.
[18] Eric R Kandel, James H Schwartz, Thomas M Jessell, Steven Siegelbaum, A James Hudspeth, Sarah Mack, ct al. Principles of neural science, volume 4. McGraw- hill New York, 2000.
[19] Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski. Neuronal dynamics: From single neurons to networks and models of cognition. Cambridge University Press, 2014.
[20] Verónica C Piatti, M Georgina Davies Davies-Sala, M Soledad Espósito, Lucas A Mongiat, Mariela F Trinchero, and Alejandro F Schinder. The timing for ncuronal maturation in the adult hippocampus is modulated by local network activity. Journal of Neuroscience, 31(21):7715 7728, 2011.
[21] Zhisong He and Qianhui Yu. Identification and characterization of functional modules reflecting transcrip- tome transition during human neuron maturation. BMC genomicx, 1911111-11, 2018.
[22] Frank Bradke, Simone Di Giovanni, and James Fawcett. Neuronal maturation: Challenges and opportunities in nascent field. Trends in Neurosciences, 43(6):360-360 2020.
[23] Peter Dayan, Laurence Abbott, et al. Theoretical neuroscience: computational and mathematical modeling of neural systems. Journal of Cognitive Neuroscience, 15(1):154-155. 2003.
[24] Andrew Nere, Atif Hashmi. Mikko Lipasti, and Giulio Tononi. Bridging the semantic gap: Emulating biological neuronal behaviors with simple digital neurons. In 2013 IEEE 19th International Symposiun on High Performance Computer Architecture (HPCA), pages 472-483. TERF, 2013.
[25] Paul A Merolla, John V Arthur, Rodrigo Alvarez-Ieaza, Andrew S Cassidy, Jun Sawada, Filipp Akopyan, Bryan L Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, al. A million spiking-neuron integrated circuit with scalable communication network and interface. Science, 345(6197))668-677 2014.
[26] James E. Smith. Efficient digital neurons for large scale cortical architectures. ACM SIGARCH Computer Architecture News, 42(3):229224,, 2014.
[27] Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-Icaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al. Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE transactions on computer-aie aided design of integrated circuits and systems, 34(10):1537 1557,
[28] Zhenzhi Wu, Hchui Zhang, Yihan Lin, Guoqi Li, Meng Wang, and Ye Tang. Liaf-net: Leaky integrate and analog fire network for lightweight and efficient spatiotemporal information processing. IEEE Transactions on Neural Networks and Learning Systems, 2021.
[29] Simon Thorpe and Jacques Gautrais. Rank order coding. Computational neuroscience, Springer, 1998.
pages pages 113-118.
[30] Christoph Kayser, Marcelo A Montemurro, Nikos K Logothetis, and Stefano Panzeri. Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns. Neuron, 644:596776 2009.
[31] Jachyun Kim, Heesu Kim, Subin Huh, Jinho Lec, and Kiyoung Choi. Deep neural networks with weighted spikes. Neurccopurting, 311:373-386, 2018
[32] Seongsik Park, Seijoon Kim, Hyeokjun Choe, and Sungroh Yoon. Fast and efficient information trans- mission with burst spikes in deep spiking neural networks. In 2019 56th ACM/IEEE Design Automation Conference (DAC), pages 1 1-6. JEEE, 2019.
[33] Man Yao, Huanhuan Gao, Guangshe Zhao, Dingheng Wang, Yihan Lin, Zhaoxu Yang. and Guoqi Li. Temporal attention spiking neural networks for event streams classification. Proceedings of the IEEÉ/CVF International Conference on Computer Vision, pages 10221-10234 2021.
[34] Yukun Yang, Wenrui Zhang, and Peng Li. Backpropagated neighborhood aggregation for accurate training of spiking neural networks. In International Conference on Machine Learning, pages 11852 11652-11863. 11862. PMLR, 2021.
11

---

[35] Peter U Diehl, Daniel Neil. Jonathan Binas, Matthew Cook. Shih-Chii Liu, and Michael Pfeiffer. Fast- classitying, high-aceuracy spiking deep networks through weight and threshold balancing. In 2015 International joint conference on neurul networks (IJCNN), pages 1-8. ieee, 2015.
[36] Bodo Rueckauer, Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Conversion of continuous-valiue deep networks efficient event-driven networks for image classification. Frontiers in neuroscience, 11:682, 2017.
[37] Romain Zimmer, Thomas Pellegrini. Srisht Fatch Singh, and Timothée Masquelier. Technical report: su- pervised training of convolutional spiking neural networks with pytorch. arXiv preprint 1r1.v:191..1012, 2019.
[38] Nitin Rathi and Kaushik Roy. Diet-snm: Direct input encoding with leakage and threshold optimization in deep spiking neural networks. arXiv preprint arXiv:2008. 03658, 2020.
[39] Christoph Stöckl and Wolfgang Maass. Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes. Nature Machine Intelligence. 3(3):230- 2233002-8,,00 2021.
[40] David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back- propagating errors. nature, 323(688) 1986.
[41] Sander M Bohte, Joost N Kok, and Johannes A La Poutré. Spikeprop: backpropagation for networks of spiking neurons. In ESANN, volume 48, pages 419 4194424. Bruges, 2000.
[42] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training perforerfomm spiking neural networks. Frontiers in newroscience, 12:331, 2018
[43] Wei Fang. Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems, 34, 2021.
[44] Yifan Hu, Yujie Wu, Lei Deng, and Guoqi Li. Advancing residual learning towards powerful deep spiking neural networks. arXiv preprint arXiv:21 2021.
[45] Wenzhe Guo, Mohammed E Fouda, Ahmed M Eltawil, and Khaled Nabil Salama. Neural coding in spiking neural networks: A comparative study for robust neuromorphic systems. Frontiers in Neuroscience, 15:212, 2021.
[46] Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
[47] Jin Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages leee, 2009.
[48] Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in neuroxcience, 11:309, 2017.
[49] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[50] Wenrui Zhang and Peng Li. Temporal spike sequence learning via backpropagation for deep spiking neural networks. Advances in Neural Information Processing Systems, 33:1 13:222222203, 2020.
[S1] Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in neuroscience, 13:95, 2019.
12


Reviews:
[{"review_id": "ePDByLj59hv", "reviewer": "Reviewer_i4xG", "summary": "In this work, the authors propose a mixture type of neuron model called GLIF to introduce additional flexibility to the neurons. They demonstrate that the proposed model can achieve SOTA performances on a bunch of classification tasks. ", "questions": "1. How does the distribution of $\\alpha$, $\\beta$, and $\\gamma$ look like? Does it display heterogeneity over layers, especially for the phenotypes defined in Figure 2?\n2. Since each neuron here contains three branches, it makes me feel that its computational cost is actually equivalent to three times the regular one, where the advantage of the current approach is not as high as it appears to be. \n3. Considering the model complexity, it would also be interesting to compare with the model ensembled with three different configurations for a single neuron type in order to support the claim that the heterogeneity in the same model makes a difference. ", "limitations": "NA", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is written clearly. \n2. The model achieved SOTA performances.\nWeakness:\n1. There could be more analyses on why the proposed method works beyond the comparison to other methods. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Gwf0q0h-nPf", "reviewer": "Reviewer_ZaJt", "summary": "The authors propose Gated LIF, a heterogeneous spiking neuron, to fuse different bio-features in different neuronal behaviors, enlarging the representation space of spiking neurons. They use gating factors to determine the proportion of the fused bio-features, which are learnable during training. The authors show that the combination of different learnable membrane-related parameters and methods can make spiking neurons constantly changing, thus increasing the heterogeneity and adaptivity of spiking neurons. ", "questions": "As GLIF neurons add more gated functionalities to the standard LIF neurons, it would be great to see the computational cost and latency change due to the added computational overload.\n\nThe results shown in Tables 4-6 are very confusing - \n1. The results do not have any confidence interval - does that indicate that this is the best performance observed?\n2. The results shown in Table 1 and Table 4 does not match - table 4 says the average accuracies are much higher than the accuracies on the same architecture and dataset shown in Table 4\n3. Why did the authors choose T=4 for Table 4,5,6 when Table 1 clearly shows that T=6 consistently gives better results.\n\nThe authors in the ablation studies show the \"Effectiveness of different bio-features.\" and \"Importance of learnable gating factors\" as separate sections. It would be great if the authors could discuss the difference between these two ablation studies and why they are essential.\n\nIn the ablation studies, the parameter sharing scheme is a bit hard to follow, and it would be great if the authors could add a bit more context and/or some references for the same.", "limitations": "The authors did not discuss the limitations of their work or the potential negative social impact of their work in the paper. The authors can discuss how the use of GLIF can make the SNN more biologically plausible, and by understanding the nature these SNN works, we can make more energy-efficient neural network models.\n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n GLIF controls the fusion of different bio-features using different gating units for those three neuronal behaviors. GLIF can simultaneously contain different neuronal dynamic features, increasing the neuronal dynamic diversity of LIF models. \n\nThe GLIF model consistently improves the performance of all the datasets and the models. An interesting thing to note is that GLIF consistently achieves 100% accuracy with a 0% error margin on the DVS Gesture dataset, which is impressive.\n\n\n\n\nWeakness:\nI felt the paper is generally a bit hard to read and very dense. Also, the structure of the paper's organization makes it confusing for readers unfamiliar with spiking neural networks. As such, it would be greatly appreciated if the authors could elaborate on the idea and make the contributions of the work a bit more explicit.\n\nThe author uses some terminology without explaining what it means, as it might be new for people from different backgrounds. For example, the main premise of the paper is based on the hypothesis that GLIF fuses different \"bio-features\" in different neuronal behaviors, expanding the representation space of spiking neurons. However, it is difficult to understand what the authors mean by \"bio-features\" as it has not been defined anywhere in the manuscript.\nThe author discusses the variation of the neuronal dynamics using Fig. 2 and Section 3.3, which seemed a bit redundant and can be moved to the Appendix. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GVb6UPnlR5g", "reviewer": "Reviewer_GThD", "summary": "The authors proposed three attention schemes that work on three different aspects of the LIF model: membrane potential leakage, integration accumulation, and membrane potential rest. The authors aim to enhance the LIF model's representation space with the proposed attention schemes. The key to the proposed attention schemes is three learnable parameters. They balance the pre-defined LIF behaviors, i.e., linear decay vs. exponential decay, uniform-coding vs. flexible-coding, and hard reset vs. soft reset rest. \n\nTo show the effectiveness of the proposed method, the authors mainly leveraged ResNet-based architectures on classification tasks. ", "questions": "writing issues:\n1. line 41: \"units $G_\\alpha$, $G_\\beta$, and $G_\\gamma$ for those three neuronal behaviors\" It is unclear to me which one is which. \n2. lines 58-59. Poor writing\n", "limitations": "From my perspective, the proposed method is incremental, not exciting. More importantly, the learnable parameters did not really solve the \"single biological feature\" problem. The proposed method added three dimensions to the LIF model but is still a static SNN model. \n\nThe validation is weak from my perspective. Comparing with SRM is necessary, and the validation without tbBN is necessary. The experimental results related to the spiking threshold should also be reported and analyzed.   ", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+1. Enhancing SNNs' capability in encoding different bio-inspired behavior is an exciting direction. \n\nWeaknesses:\n-1. I did not see the value of adding many learnable parameters into LIF. Essentially, they are still fixed during inferences and \"support only a single biological feature in different neuronal behaviors.\" In addition, the SRM model has already offered much more parameters, which can be learned directly. As such, the authors should provide a comparison with SRM-based models.\n\n-2. I think the key to SNNs research should be on the dynamic side. Dynamically dealing with different conditions is more appreciated. However, the proposed method did not introduce any dynamic schemes. \n\n-3. The authors applied tdBN to all their models. Did other competing models have tdBN? In addition, what's the performance without tbBN. Basically, we do not know whether the tbBN gives more contribution to the performance or the proposed method.\n\n-4. How does the spiking threshold impact the classification performance? In theory, all the proposed attention schemes can be replaced by applying different spiking threshold to each SNN neuron. Therefore, it is necessary to show the experimental results related to different spiking thresholds. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9l9e_N2utF4", "reviewer": "Reviewer_jY7N", "summary": "This paper proposes the GLIF neuron model, which has rich neuronal dynamics with learnable gates. The GLIF neuron achieves SOTA accuracy on CIFAR10, CIFAR100 and ImageNet datasets.", "questions": "Does g^t in line 153 represent G_beta in eq.11? Or it is a learnable parameter? \nWhy beta=1 in eq.11 means single spike coding pattern of integration accumulation? In my opinion, when beta=1, G_beta = g^t * C, which behaves like (1 – tau) * C in the LIF neuron (V = (1-tau) * V + tau * C).\n\nIn line 167, “GLIF only performs hard reset on the exponential-decay-related part of the membrane potential”. Can the author explain why use such a hard reset? In most cases, hard reset means the whole V, rather than a part of V, is set to V_reset .", "limitations": "I find from codes that some parameters of GLIF are not reused across different time-steps because they have shape [T, *] where T is the total time-steps. I am afraid that this design violates the parameter reuse rule in SNNs.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The complex spiking neuron model has not been well studied in deep SNNs. Most deep SNNs use the simple IF or LIF neuron. As far as I know, this is one of the first papers that focuses on the complex spiking neuron.\n\nThe design of the GLIF neuron is reasonable. And it gets SOTA accuracy on most datasets.\n\nI notice that the proposed method gets 100% accuracy on DVS Gesture, which is hard to achieve. I check the codes and find that there is a bug at line 241 in data_builder.py: \n\ntest_dataset = DVS128Gesture(dataset_dir, train=True\n\nIt seems that the author takes train set as the test set and gets 100% accuracy. But I think it is an honest mistake.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept