GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks
Abstract
Spiking Neural Networks (SNNs) have been studied over decades to incorporate their biological plausibility and leverage their promising energy efficiency.
Throughout existing SNNs, the leaky integrate-and-fire (LIF) model is commonlyadopted to formulate the spiking neuron and evolves into numerous variants withdifferent biological features.
However, most LIF-based neurons support only singlebiological feature in different neuronal behaviors, limiting their expressivenessand neuronal dynamic diversity.
In this paper, we propose GLIF, a unified spikingneuron, to fuse different bio-features in different neuronal behaviors, enlargingthe representation space of spiking neurons.
In GLIF, gating factors, which areexploited to determine the proportion of the fused bio-features, are learnable duringtraining.
Combining all learnable membrane-related parameters, our method canmake spiking neurons different and constantly changing, thus increasing the heterogeneity and adaptivity of spiking neurons.
Extensive experiments on a variety ofdatasets demonstrate that our method obtains superior performance compared withother SNNs by simply changing their neuronal formulations to GLIF.
In particular,we train a spiking ResNet-19 with GLIF and achieve 77 .
35% top-1 accuracy withsix time steps on CIFAR-100, which has advanced the state-of-the-art.
Introduction
Spiking neural network (SNN) is considered the third generation of the neural network, and thebionic modeling of its activation unit is called the spiking neuron, which brings about the spike-basedcommunication between layers[1, 2].
Due to the high sparsity and multiplication-free operationin information processing, such spike-based communication can improve energy efficiency[3, 4].
Moreover, spiking neuron can effectively capture temporal information and has good performance inprocessing neuromorphic data[5] and anti-noise[6].
Together with artificial neural network (ANN)colleagues, SNN further shows great potential for general intelligence[7].
Thus, SNN has arousedheated research interests recently.
The commonly used spiking neuron in different SNNs[6, 8, 9, 10, 11, 12] is the leaky integrateand-fire (LIF) model, named vanilla LIF.
As the name suggests, it can implement three differentneuronal behaviors: membrane potential leakage, integration accumulation, and spike initiation.
Tobetter model neurons, prior works [13, 14, 15, 16, 17] have improved the LIF model from the abovethree aspects, and proposed many LIF variants with different biological features.
For all the above
LIF models (referred to as simplex LIF), any of the three neuronal behaviors supports only singlebiological feature.
For instance, the vanilla LIF model only exploits exponential decay in membrane τ exp
S t −
U t − τ lin
V re + ×
S t
U t g t
C t +
U t
S t (a) (b) (c) σ ( x )  × ×−/
𝔾 γ
𝔾 α 𝔾 β
L t
F t
I t potential leakage, while [15] only supports linear decay. However, different neurons in the cerebralcortex have different response characteristics[18, 19], especially for neurons from different brainregions. This naturally raises an issue: could higher neuronal dynamic diversity of LIF models help SNNs do better?
In this paper, we investigate the hybrid bio-features in the LIF model to answer thisquestion.
We propose the gated LIF model (GLIF) that fuses different bio-features in the three neuronalbehaviors.
As illustrated in Fig.
1, GLIF controls the fusion of different bio-features through gatingunits G α , G β , and G γ for those three neuronal behaviors, respectively.
In each gating unit, a gatingfactor is computed from a sigmoid function σ ( x ) over a learnable gating parameter x to determinethe proportion of each bio-feature, thus guiding the fusion of different bio-features.
Therefore, GLIFcan simultaneously contain different neuronal dynamic features, increasing the neuronal dynamicdiversity of LIF models.
In addition, when the gating factor is 0 or 1 , GLIF can also support onlysingle neuronal dynamic feature.
As a result, GLIF can cover other different LIF models and can beviewed as a super spiking neuron, greatly enlarging the representation space of spiking neurons.
Furthermore, we introduce the channel-wise parametric method to GLIF.
This method makes allmembrane-related parameters in GLIF learnable and shares the same GLIF parameters channel-wiselyin SNNs.
Combining with learnable gating factors in GLIF, on the one hand, this method makesdifferent channels in SNNs have completely different spiking neurons, increasing the heterogeneityof spiking neurons and the expressive ability of SNNs.
On the other hand, during training, thespike neurons in SNNs are constantly changing, which is similar to the neuronal maturation duringdevelopment[20, 21, 22], thus the adaptivity of spiking neurons being enhanced.
We demonstrate the effectiveness of GLIF on both static datasets and the neuromorphic dataset. Simply replacing the spiking neuron model with our GLIF in multiple networks, our model canachieve solid improvement on different tasks.
For example, our model can achieve the state-of-the-art75 .
48% top-1 accuracy on CIFAR-100 with only 2 time steps, which achieves the state-of-the-artresults compared with other methods.
Related Work
Variant LIF .
Variant LIF models with different biological features are proposed in previous arts, andthose models differ in one or more of the three different neuronal behaviors: membrane potentialleakage, integration accumulation, and spike initiation.
Firstly, membrane potential leakage representsthe potential decay mechanism.
Exponential decay is first exploited in the vanilla LIF model[23], and it brings neuronal dynamics with stable convergence, shrinking the potential to the resting potentialsteadily.
But, exponential decay is not efficient enough in massive neural simulations due to itsmultiplication operation.
Therefore, linear decay replaces exponential decay in [13, 24, 25, 26, 27],and brings neuronal dynamics with the flexibility, making it possible for configurable spiking rates[15].
Although the two decays behave differently in neuronal dynamics, both of them are proveneffective in SNNs[6, 28].
Secondly, integration accumulation represents the potential accumulationinduced by presynaptic spikes.
Commonly, spike intensities of different time steps are consideredequally high.
Consequently, the discrepancies among the spikes at different time steps are ignored.
To address this issue, many prior works propose to assign different weights to the input spikes atdifferent time steps.
These methods are implemented by manual settings[29, 30, 31] or adaptivesettings[32, 33, 34] to improve biological plausibility, SNN performance, or inference efficiency.
Lastly, spike initiation functions as the resetting mechanism triggered by spike firing.
There are twocommon resetting strategies: hard reset[35] and soft reset[15, 36].
When a spike is triggered, thehard reset directly sets the potential to the resting potential, while the soft reset subtracts the potentialby a configurable value.
Therefore, hard reset is more stable as a strict reset mechanism, while softreset is more flexible making configurable spiking rates feasible[15].
Compared with the LIF modelsabove, our GLIF incorporates different neuronal dynamic features to increase the neuronal dynamicdiversity of LIF models and improve performance of SNNs.
Parametric spiking neurons .
Early existing spiking neurons often require manual tuning of themembrane-related parameters to set their neuronal dynamics, and these parameters are often sharedby all neurons, which limits the diversity of spiking neurons and the expressive ability of SNNs.
Inrecent years, Parametric spiking neurons[37, 38, 10, 28] have been proposed, which set one or moremembrane-related parameters to be learnable, and update these parameters during the training of
SNNs.
[37] and [38] set the membrane leak and potential threshold learnable, while [10] and [28]choose the membrane leak only.
FS-neuron[39] is more aggressive, setting all membrane-relatedparameters in the spiking neuron learnable to strictly fit the neuronal dynamics to the target activationfunction.
Compared with the above methods, GLIF adopts the channel-wise parametric method tofully parameterize the spiking neuron, including learnable exponential decay, linear decay, potentialthreshold, resetting voltage, input conductance, and gating factors.
Supervised direct learning of SNNs.
Supervised direct learning of SNNs follows the idea ofbackpropagation (BP) in ANN[40].
Bohte et al.
[41] first adopt the surrogate gradient (SG) toempirically solve the non-differentiable term of spiking neurons by approximation with the localderivative of potential.
Based on the observation that the proper steepness of the SG curve mattersmore than the shape in SNN learning[42], Li et al.
[11] conduct a pre-estimation before each epoch tofind the SG curve with the optimal smoothness.
Besides, many techniques are developed to makedeep SNNs converge faster, such as NeuNorm[8], tdBN[9], SEW block[43], TET[12], MS residualblock[44], etc.
In this way, the problem of end-to-end training of deep SNNs can be solved.
In thispaper, we choose supervised direct learning of SNNs to estimate the effectiveness of our GLIF.
Methodology
3.1 Revisiting LIF models
Vanilla LIF.
The Vanilla LIF model is commonly adopted as a spiking neuron model in SNNs,which can be discretely formulated as follows:
[Equation]
Here, ⊙ denotes the element-wise multiplication.
U ( t,l ) is the membrane potential vector of thelayer l at time-step t and can be updated in a discrete way through Eq.(1), where C ( t,l ) denotes theinput vector and can be obtained by the dot-product between the synaptic weight matrix W and the output spike vector of the previous layer S ( t,l − 1) , as described in Eq.(2).
The output spike vector
S ( t,l ) is given by the Heaviside step function H ( · ) in Eq.(3), indicating that a spike is fired when themembrane potential exceeds the potential threshold V
th .
When updating U ( t,l ) in Eq.(1), the threeneuronal behaviors are considered.
Specifically, the time constant τ acts as the exponential decaycoefficient on the term τ U ( t − 1 ,l ) to implement exponential decay, C ( t,l ) is integrated over time to
U ( t,l ) , and the term (1 − S ( t − 1 ,l ) ) hard resets the membrane potential to 0 if a spike is fired at theprevious time step.
Notably, with all the three neuronal behaviors influencing the membrane potential,the membrane potential traces over time can explicitly depict the neuronal dynamics.
Primitives.
Since each behavior is implemented by the corresponding term in formulas, for explicitness, we refer to those neural behavior related parameters as primitives in this paper.
In this way,bio-features can be quantitatively described by primitives.
For example, exponential decay is realizedby the term τ U ( t − 1 ,l ) in Eq.(1), so τ is a primitive and determines the amplitude of exponentialdecay.
Thus, by choosing different primitives for each behavior, we can formulate different LIFvariants.
For instance, with the linear decay primitive τ lin and the soft reset primitive τ re , we have:
U ( t,l ) =
U ( t − 1 ,l ) − τ lin − V re S ( t − 1 ,l ) + C ( t,l ) , which formulates the spiking neuron in [15].
3.2 Gated LIF Model
Although variant LIF offers a different choice in modeling the spiking neuron, it still remains simplex,i.e., the bio-features of the neuronal dynamic are unitary.
Once formulated, a hard-reset LIF neuroncannot perform the soft-reset operation.
As a result, these simplex LIF models make the spikingneurons in SNNs lack dynamic diversity.
Thus, we propose the gated LIF (GLIF), a unified spikingneuron, to fuse different bio-features in the three neuronal behaviors.
GLIF is illustrated in Fig.
1 andfurther described as follows:
[Equation]
In Eq.(4), L ( t,l ) denotes the membrane potential vector after membrane potential leakage, I ( t,l )denotes the vector of the incremental potential caused by integration accumulation, and F ( t,l ) denotesthe vector of the potential reduction caused by spike initiation.
Obviously, the above three vectorsreflect the consequences of the three neuronal behaviors.
In Eq.(7)(8)(9), the gating units G α ( · ) , 135 G β ( · ) , and G γ ( · ) are proposed to formulate the three neuronal behaviors, thus computing theconsequent vectors.
Computing L ( t,l ) .
In computing L ( t,l ) , the two primitives τ lin and τ exp are considered.
τ lin is thelinear decay primitive, and τ exp is the exponential decay primitive.
As their names indicate, τ lin andτ exp play the roles of the linear decay and exponential decay on U ( t − 1 ,l ) .
Through the gating unit G α ( · ) , the two primitives are fused together to perform membrane potential leakage.
Following isthe formulation of G α ( · ) :
[Equation]
where α is the gating factor of G α ( · ) , whose value is bounded within (0 , 1) .
When α approaches 0 ,[1 − α (1 − τ exp )] and (1 − α ) tend to 1 , which means GLIF performs linear decay.
As α increasesfrom 0 to 1 , (1 − α )
τ lin decreases from τ lin to 0 , while [1 − α (1 − τ exp )] decreases from 1 to τ exp ,which means GLIF is shifting from linear decay to exponential decay.
Therefore, in Eq.(10), the twoprimitives τ lin and τ exp are dually fused with the help of the gating factor α , which can also balancethe amplitudes of linear decay and exponential decay.
Note that the primitives in the same gating unit are referred to as the dual primitives in the following content, and correspondingly, the bio-featuresthe dual primitives describe are referred to as the dual bio-features .
In the extreme case of α = 0or 1 , G α preserves only single primitive, which means GLIF covers the simplex formulations ofmembrane potential leakage.
Computing I ( t,l ) .
In computing I ( t,l ) , the dual primitives g t and 1 are considered to performintegration accumulation.
g t denotes the conductance primitive, namely, the time-dependent synapticweight, which can be coded with different patterns, such as rank-order based patterns[29] and phasecoding based patterns[31].
So, we can view it as the flexible-coding primitive.
While, the constant1 means g t ≡ 1 , representing the uniform-coding primitive, which is the dominant paradigm andproven doing well in the direct learning of SNNs[45, 12, 11].
Following is the formulation of G β ( · ) :
[Equation]
11) where β , bounded within (0 , 1) , is the gating factor of G β ( · ) .
Similarly, as β → 0 , the uniform-codingpattern is preferred rather than the flexible-coding, while β → 1 is the opposite case.
When β = 0 orβ = 1 , GLIF supports single spike coding pattern of integration accumulation.
Computing F ( t,l ) .
In computing F ( t,l ) , the soft-reset primitive V re and the hard-reset primitive 0are coupled as the dual primitives to perform spike initiation.
Following is the formulation of G γ ( · ) :
[Equation]
(12) where γ bounded within (0 , 1) is the gating factor of G γ ( · ) .
The term (1 − γ ) V re is the actualpotential decrease caused by soft reset, while the term ( 0 + γ ) L ( t,l ) exp is the actual potential decreasecaused by hard reset.
Note that L ( t,l ) exp is given by [1 − α (1 − τ exp )]
U ( t − 1 ,l ) in Eq.(10), which means
GLIF only performs hard reset on the exponential-decay-related part of the membrane potential.
In asimilar way, as γ → 0 , soft reset is incrementally biased, while γ → 1 , hard reset is incrementallybiased.
When γ = 0 or 1 , GLIF supports the simplex reset formulation of spike initiation.
Therefore, with gating units, the GLIF-based spiking neurons can fuse the dual bio-features in eachone of the three neuronal behaviors.
When gating factors are binary, GLIF transforms into thecorresponding simplex LIFs.
For example, in the case of α, β, γ = 1 , 0 , 1 , GLIF is equivalent to thevanilla LIF.
In a nutshell, GLIF unifies different simplex LIFs by incorporating dual bio-features andexploits gating factors to balance the dual bio-features.
Channel-wise parametric method.
Parametric method is applied to the primitives and gatingfactors in our GLIF, including τ lin , τ exp , V re , g t , V th , α , β , and γ .
We make all these parameterslearnable, which makes GLIF neurons adjust their neuronal dynamics during training, increasing theadaptivity of the spiking neurons.
Given x is one of these parameters, we can obtain x by:
[Equation]
(13) where x p ∈ ( −∞ , + ∞ ) is shared channel-wisely in SNNs.
Consequently, each channel shares aunique setting of GLIF neurons.
Thus, the heterogeneity of spiking neurons is significantly increased.
Training framework.
In terms of the training framework, we utilize backpropagation through time(BPTT) as the training algorithm for our GLIF SNNs.
As for the non-differentiable Heaviside stepfunction in Eq.(6), we solve it by the surrogate gradient defined as: [Equation] Thus, GLIF SNNs can be trained end-to-end.
Leak
V th V th
V th
V re V re
V re γ γ γ
Leak
V th V th
V th
V re V re
V re γ γ γ
Leak
Leak α τ lin τ exp
Soft & hard reset
V re (c) (d) (b) (a) α β
Leak
V th V th
V th
V re V re
V re γ γ γ
Leak
V th V th
V th
V re V re
V re γ γ γ
3.3 Dynamic Analysis
GLIF contains dual bio-features, which brings great diversity to the neuronal dynamics.
Here weuse a simple case to analyze the dynamic properties of GLIF.
Fig.
(2) illustrates the response ofthe spiking neurons to the constant input, which spikes each time step until the membrane potentialreaches the threshold.
As shown in the figure, GLIF neurons with different parameter configurationshave completely different dynamic responses to the same input.
Bridging the gap between exponential decay and linear decay.
As shown in Fig.
2(a), comparedwith non-decay, on the one hand, exponential decay brings the neuronal dynamic with stability andastringency, making it increasingly difficult for the membrane potential to climb over the threshold.
However, the membrane potential falls into the saturation zone and fails to reach the potentialthreshold, which indicates exponential decay may impair the spiking responsiveness.
On the otherhand, linear decay operates equally at each time step to avoid saturation, thus maintaining the highspiking responsiveness.
However, if there are no inputs, linear decay leads to the potential decreasingin an unlimited way, which impairs the stability.
While in GLIF, the above two types of decays arethe extreme cases, i.e., GLIF only acts exponential decay if α = 1 and only serves linear decayif α = 0 .
In most cases where α ∈ (0 , 1) , GLIF is eligible for exponential decay’s astringencyand benefits from the high spiking responsiveness linear decay brings about.
To some extent, thegating factor α makes the two decays complementary and determines the proportion of each of thetwo decays, bridging the gap between the neuronal dynamics of exponential decay and linear decayand enlarging the representation space of membrane potential leakage.
Furthermore, since τ exp andτ lin are learnable in GLIF, the gap between exponential and linear decay becomes more flexible.
Therefore, the actual representation space given by τ exp , τ lin , and α could be even larger.
Incorporating time-dependent synaptic weights.
In GLIF, we assign the time-dependent synapticweights g t to the spike train to differentiate spike stimulation levels at different time steps.
Since g t is learnable, the flexible-coding patterns of different GLIF neurons may differ significantly.
Forthe convenience of analysis, we add constraints to g t , assuming that g t is in the form of cosineoscillation, where β controls the amplitude.
As shown in Fig.
2(b), when β = 0 , which means theuniform-coding pattern, each time the spiking neuron receives an input spike, the membrane potentialincreases by the same amount.
Our GLIF is different.
When β ∈ (0 , 1) , each time the GLIF receivesan input spike, the membrane potential responds differently depending on the specific stimulationlevel of the current input spike.
On the one hand, considering that there is no constraint on g t inpractice, GLIF is more flexible, and the neuronal dynamic diversity is further increased.
On the otherhand, because the stimulus of input spikes at different times are considered differently, incorporatingtime-dependent synaptic weights also helps to better process temporal features.
Bridging the gap between hard reset and soft reset.
Fig. 2(c) compares the neuronal dynamicsbetween soft reset and hard reset.
As we can see, soft reset reduces the current membrane potential by
V re .
Since V re is not related to the current membrane potential, soft reset may lead to over-resetting orunder-resetting of membrane potential, which is flexible for the spiking neurons.
Hard reset reducesthe current membrane potential to an empirical value, e.g., 0 , restarting the potential trace all over,providing the neuronal dynamics with stability.
While in GLIF, we consider the potential reductioncaused by spike initiation to be both related to V re and the current membrane potential.
Becauseγ ∈ (0 , 1) balances the amplitudes of the two resets, GLIF can reset the membrane potential to anypossible levels between the consequent potentials of hard reset and soft reset as shown in Fig.
2(d),enlarging the representation space of spike initiation.
Experiments
In this section, we demonstrate the effectiveness of our proposed GLIF model.
We first compareour results with other existing state-of-the-arts, then carry out ablation studies to evaluate differentaspects of our proposed method.
4.1 Experimental Settings
We perform experiments on two kinds of datasets.
The first is image classification datasets, including
CIFAR[46] and ImageNet[47].
The other one is the event dataset, i.e. DVS128 Gesture[48], which isa gesture recognition dataset captured by DVS cameras.
For a fair comparison with other methods,we choose the commonly used network architectures in existing works for experiments, including ResNet-family[49, 9, 44], LIAF-Net[28], 7B-Net[43], and CIFARNet[42, 8, 50], and apply tdBN[9]to all of them.
More implementation details of training those SNNs are revealed in Appendix.
4.2 Comparison with Existing Works
CIFAR.
We test our GLIF on CIFAR-10 and CIFAR-100 datasets and run three times for eachexperiment to report the “mean ± std”.
As shown in Table 1, our GLIF SNNs can outperform existingworks.
Specifically, on CIFAR-10, we first test our GLIF on a smaller network CIFARNet and achievethe superior accuracy over other methods [42, 8, 50], even with fewer time steps.
Since tdBN isemployed in our GLIF SNNs, we then compare GLIF with STBP-tdBN[51] on ResNet-19.
By simplychanging the neurons in STBP-tdBN to GLIF, our SNNs can achieve significant improvement acrossall the tested time-steps.
Compared with the state-of-the-art TET[12], our method can still obtainbetter accuracy, and our ResNet-19 with time-step 4 even outperforms theirs with time-step 6 by0 .
On CIFAR-100, our method can also achieve much better performance than STBP-tdBNand TET.
We also compare GLIF ResNet-18 with the highly handcrafted ResNet-18 of Dspike[11].
As listed in Table 1, our GLIF ResNet-18 can outperform Dspike ResNet-18 by at least 0 .
63% and2 .
92% absolute accuracy on CIFAR-10 and CIFAR-100, respectively.
ImageNet.
We apply GLIF to ResNet-34 and MS-ResNet-18[44].
Table 2 shows the results onthe validation set.
We first compare our method with STBP-tdBN, which can be viewed as our
[Table]
[Table]
[Table]
baseline, on ResNet-
[Table]
and we can observe that our GLIF SNNs achieve better performance ( 69 . 09%v.s.
Since parametric technology is employed in our GLIF [Table] we also compare GLIF withthe parametric spiking neuron PLIF[43].
Without changing the model structure, our method canstill achieve higher accuracy than PLIF on ResNet-34 ( 67 . 52% v.s. 67 .
Moreover, comparedwith MS-ResNet[44], whose handcrafted residual block achieves the state-of-the-art performance,we apply our GLIF to MS-ResNet-18 and can also significantly increase the accuracy ( 68 . 11% v.s.63 .
These results demonstrate the effectiveness of our proposed GLIF model.
In addition,compared with other SNNs in TET[12] and Dspike[11], GLIF SNNs can also obtain higher accuracy.
DVS128 Gesture.
To show the effectiveness of GLIF in processing spatio-temporal features, weconduct experiments on the event dataset DVS128 Gesture and run each experiment three times toreport the “mean ± std”.
Since previous works use different network architectures on this dataset, wecompare with existing methods PLIF[43], STBP-tdBN[9], and LIAF[28] on 7B-Net, ResNet-17, and
LIAF-Net, respectively. As shown in Table 3, our GLIF SNNs can outperform all of those methods,which demonstrates the effectiveness of the proposed GLIF on spatio-temporal event streams.
[Table]
[Table] [Table]
4.3 Ablation Study
Effectiveness of different bio-features.
To demonstrate the effectiveness of different bio-featuresthat are incorporated in our GLIF, we evaluate the performance of GLIF SNNs in extreme caseswhere GLIF only acts single bio-feature in each neuronal behavior.
Since the gating factors controlbio-features, we can obtain eight simplex LIFs by combining different gated biometrics, all of whichhave static binary gating factors.
We name those simplex LIFs with the values of their gating factors,e.g., 101 is the GLIF with α, β, γ = 1 , 0 , 1 that only acts exponential decay, the uniform-codingpattern, and hard reset.
On CIFAR-100, we use ResNet-18 and ResNet-19 with time-step 4 forexperiments.
As shown in Tabel 4, our proposed GLIF outperforms all of those simplex LIFs, whichdemonstrates the effectiveness of fusing different bio-features.
Importance of learnable gating factors To demonstrate the effectiveness of our proposed learnablegating factors in fusing bio-features, we compare GLIF with other two GLIF variants: GLIF_s and
GLIF_f, where GLIF_s keeps gating factors static after initialization, GLIF_f removes gating-factorsand simply fuses two different bio-features by addition.
On CIFAR-100, we also use ResNet-18and ResNet-19 with time-step 4.
Table 5 lists the experimental results.
On the one hand, comparedwith GLIF_s, our method can get higher accuracy, which demonstrates that learnable gating factorscan make GLIF better.
On the other hand, compared with GLIF_f, GLIF can also achieve betterperformance, this is because learnable gating factors can balance dual bio-features in each neuronalbehavior.
As as result, learnable gating factors are important in GLIF.
Channel-wise v.s. Layer-wise.
Besides the investigations above, we conduct an ablation studyon GLIF with different parameter-sharing schemes, which are layer-wise sharing and channel-wisesharing.
We use ResNet-19 and ResNet-18 with time-step 2 and 4 on CIFAR-100.
As listed in Table6, GLIF, which leverages channel-wise sharing scheme, consistently improves the performance overthe layer-wise sharing method.
Conclusion
In this paper, we propose GLIF, a unified spiking neuron, that fuses the bio-features in each neuronalbehavior.
In GLIF, our method utilizes the gating factor to bridge the gap between the dual bio-features,enhancing the neuronal dynamic diversity and enlarging the representation space of spiking neurons.
Furthermore, we adopt the channel-wise parametric method to make all membrane-related parameterslearnable, as well as the gating factors, and train those parameters through BPTT, increasing theneuronal adaptivity of spiking neurons.
We validate our GLIF SNNs on various benchmark datasets,showing the state-of-the-art performance compared with other methods.
Additionally, how to apply
GLIF in other training frameworks