SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks
Authors: Mingqing Xiao
Qingyan Meng
Zongpeng Zhang
Yisen Wang
Zhouchen Lin
Venue: ICLR 2022 Submitted (2022)
Keywords: spiking neural network
equilibrium state
spike-based training method
neuromorphic engineering

Abstract: Spiking neural networks (SNNs) with event-based computation are promising brain-inspired models for energy-efficient applications on neuromorphic hardware. However, most supervised SNN training methods require complex computation or impractical neuron models, which hinders them from spike-based energy-efficient training. Among them, the recently proposed method, implicit differentiation on the equilibrium state (IDE), for training feedback SNNs is a promising way that is possible for generalization to locally spike-based learning with flexible network structures. In this paper, we study spike-based implicit differentiation on the equilibrium state (SPIDE) that extends the IDE method for supervised local learning with spikes, which could be possible for energy-efficient training on neuromorphic hardware. Specifically, we first introduce ternary spiking neuron couples that can realize ternary outputs with the common neuron model, and we prove that implicit differentiation can be solved by spikes based on this design. With this approach, the whole training procedure can be made as event-driven spike computation and weights are updated locally with two-stage average firing rates. Then to reduce the approximation error of spikes due to the finite simulation time steps, we propose to modify the resting membrane potential. Based on it, the average firing rate, when viewed as a stochastic estimator, achieves an unbiased estimation of iterative solution for implicit differentiation and the variance of this estimator is reduced. With these key components, we can train SNNs with either feedback or feedforward structures in a small number of time steps. Further, the firing sparsity during training demonstrates the great potential for energy efficiency. Meanwhile, even with these constraints, our trained models could still achieve competitive results on MNIST, CIFAR-10 and CIFAR-100. Our proposed method demonstrates the great potential for energy-efficient training of SNNs on neuromorphic hardware.

Under review as a conference paper at ICLR 2022
SPIDE: A PURELY SPIKE-BASED METHOD FOR TRAINING FEEDBACK SPIKING NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Spiking neural networks (SNNs) with event-based computation are promising brain-inspired models for energy-efficient applications neuromorphic hard- ware. However, most high-pe performaace supervised SNN training methods in ma- chine learning research, such as conversion from artificial neural networks or di- rect training with surrogate gradients, require complex computation not supported by spiking neurons, which hinders them from spike-based energy-efficient train- ing. Among them, the recently proposed method, implicit differentiation on the equilibrium state (IDE), for training feedback SNNs promising way possi- ble for generalization to spike-based learning with flexible network structures. In this paper, we study spike-based implicit differentiation on the equilibrium state (SPIDE) that extends IDE for supervised local learning with spikes, which could possible for energy-efficient training on neuromorphic hardware. Specifically, we first introduce ternary spiking neuron couples to realize ternary outputs with the common neuron model, and prove that implicit differentiation can solved by spikes based this design. With this approach, the whole training procedure made as event-driven spike computation and weights are updated locally with two-stage average firing rates. Then reduce the approximation error of spikes due to the finite simulation time steps, we propose to modify the resting membrane stochastic estima-
potential. Based it, the average firing rate, when viewed tor, achieves an unbiased estimation of iterative solution for implicit differentia- tion and the variance of this estimator is reduced. With these key components, we can train SNNs with either feedback or feedforward structures in small number of time steps. Further, the firing sparsity during training demonstrates the great po- tential for energy efficiency. Meanwhile, even with these constraints, our trained models can still achieve competitive results on MNIST, CIFAR-10, CIFAR- 1000, and CIFAR IO-DVS, Our proposed method demonstrates the great potential for energy-efficient training of SNNs on neuromorphic hardware.
INTRODUCTION
Spiking neural networks (SNNs) are brain-inspired models that transmit spikes between neurons for event-driven energy-efficient computation. SNNs be implemented with less energy neuro- morphic hardware (Akopyan et al., 2015 Davies et al. 2018 Pei et al. 2019 Roy et al. 2019). which can remedy the defects of large energy consumption of artificial neural networks (ANNs).
Different from ANNs, however, directly supervised training of SNNs is a hard problem due to the complex spiking neuron model which is discontinuous. To solve this problem, converting ANNs to SNNS (Hunsberger & Eliasmith] (2055] Rueckauer et al., 2017 Sengupta et al., 2019 Rathi et al. Deng & Gul
2019
20211 Yan et al. [2021). many other direct SNN training methods (Wu al.
2018
Bellec et al.
2018 Jin et al. 2018. Shrestha & Orchard 2018 Wu et al. 2 Neftci et al.
2019
Zhang Li 2019 Kim et al. 2020, /Zheng et al.] [202]] Bohte et al. 20022 Zhang Li 2020
Kim et al.) 2020; Xiao et al.) (021) have been proposed. While these methods could partly tackle
the problems of unsatisfactory performance or high latency, they require complex computation for gradient calculation approximation, which cannot be implemented by common spiking neurons on neuromorphic hardware. They aim at training SNNs on commonly used computational units, e.g. GPU, and deploying trained models for energy-efficien inference. However they do not consider the training procedure could leverage the same spike-basse based computation for gradient calculation and training to reduce the large energy consumption during training well.

---

Under review as a conference paper at ICLR 2022
Table I: Comparison of different supervised SNN training methods with respect to performance,
latency, structure flexibility, neuron model, spike-based or not, and neuromorphic plausibility.
Method
High Perform. Low Lakeney
Struc. Flexi.
Common Neuron Model Spike-bused
Neuro. Plaus.
ANN-SNN
x
x
x
Low
BPTT with Surrogate Gradients DFA with Spikes
Low
High
SpikeCirad [Thiele et 2019
Medium
IDB (Xiao et al. 2021
Low
SPIDE (ours)
High
A few previous works try to train SNNs with spikes (Guerguiev et al. 2017 Neftci et al. 2017 Samadi et al. 2017; O' O'Connor & Welling 2016, Thiele et al. 2019b a). They either are based require impractical special neuron models (Thiele et al. 2019b(a). Besides, they only focus on feedforward network structures imitated from ANNs, which ignores feedback connections that are ubiquitous in the hu- man brain and enable neural networks to be shallower and more efficient (Kubilius et al.) [219 Xiao al. (02T). Actually, feedback structures suit SNNs more since SNNs will naturally compute with multiple time steps, which could reuse representations and avoid uneconomical costs to unfold along time that ANNs suffer from (Xiao et al., 2021). So training algorithms for feedback SNNs, which may also degrade to feedforward structures by taking feedback as zero, is worth more exploration.
on direct feedback alignment (DFA) (Nokland (016) and performs poorly,
An ideal SNN training method should tackle the common problems, be suitable for flexible struc- tures (feedforward or feedback) and be spike-based with high neuromorphic plausibility. The im- plicit differentiation on the equilibrium state (IDE) method (Xiao et al. (02T). which is recently proposed to train feedback spiking neural networks (FSNNs), is promising method that may gen- cralize to spike- learning for requirement. They derive that the forward computation of FSNNs converges an equilibrium state, which follows fixed-poin equation. Based on it, they propose to train FSNNs by implicit differentiation on this equation, which tackles the common difficulties for SNN training including non-differentiabiiity and large memory costs, and has interesting local update properties. In their method, however, they leverage general root-finding methods to solve implicit differentiation, which requires complex computation on standard computation systems.
In this work, we extend the IDE method to spike-based IDE (SPIDE), which fulfills our require- ments and has great potential for energy-efficient training of SNNs on neuromorphic hardware, by introducing ternary spiking neuron couples and proposing to solve implicit differentiation by spikes based on them. Our method is also applicable to feedforward structures by degrading the feedback connection as zero. In practice, however, it may require long time steps to stabilize the training with spikes due to approximation error for gradients. So we further dive into the approximation error from the statistical perspective, and propose to simply adjust the resting potential of SNNs to achieve an unbiased estimation of gradients and reduce the estimation variance of SNN computa- tion. With these methods, can train our models in a small number of time steps, which could further improve the energy efficiency as well as the latency. Our contributions include:
1.
We propose the SPIDE method that is the first train high-performance SNNs by spikes with common neuron models. Specifically, we propose ternary spiking neuron couples and prove that implicit differentiation for gradient calculation can be solved by spikes based on this design. Our method is applicable to both feedback and feedforward structures.
2. We theoretically analyze the approximation error of solving implicit differentiation by spikes, and propose to modify the resting potential remove the approximation bias and reduce the estimation variance, which enables training in a small number of time steps.
3. Experiments show the low latency and firing sparsity during training, which demonstrates the great potential for energy-efficient training of SNNs on neuromorphic hardware. The performance on MNIST, CIFAR-10, CIFAR-100 and CIFARIO-DVS are also competitive.
RELATED WORK
Early works seek biologically inspired methods to train SNNs, e.g. spike-time dependent plastic- ity (STDP) (Diehl & Cook. 2015) or rise of successful ANNs, several works try convert trained ANNs to SNNs to obtain high perfor- mance (Hunsberger & Eliasmith 2013; Rueckauer et al., 2017| Sengupta et al. 2019 Rathi et al.] 2019
STDP (Legenstin et al., 2008). Since the
Deng & Gu
2021 Yan al. 2021). However, they suffer from extremely large time steps
2

---

Under review as a conference paper at ICLR 2022
and their structures are limited in the scope of ANNs. Others try to directly train SNNs by imitating backpropagation throught time (BPTT) and use surrogate derivative for discontinuous spiking func-
tions (Lee et al. 2016 Wu et al. 2018 Bellee et al. 20188) et al. [2018] Shrestha & Orchard 2018)
et al. 2019 Zhang & Li 2019 Neftci et al. 2019 Zheng et al. 2021 or compute gradient with
respect spiking times (Bohte et al. 2002 Zhang & Lij 20200 Kim al. 2020). However, they suf-
fer from approximation error and large memory costs, Xiao et al. (2021) propose the IDE method to
train feedback spiking neural networks, which decouples the forward and backward procedures and avoids the common SNN training problems. However, all these methods require complex compu- tation during training rather than spike-basee based. A few works focusing on training SNNs with spikes either are based on feedback alignment and limited in simple datasets (Guerguiev et al. 2017 Neftci et al. 2017 Samadi et al. 2017 O'Connor & Welling 2016), or require impractical special neuron models that require consideration of accumulated spikes for spike generation (Thile et al. 22019b a), which is impractical on neuromorphic hardware. And they are only applicable to feedforward archi- tectures. Instead, we are the first to leverage spikes with common neuron models to train SNNs with feedback or feedforward structures. The comparison of different methods is illustrated in Table I
3 PRELIMINARIES
We first introduce preliminaries about spiking neurons and the IDE training method. The basic thought of IDE (Xiao et al., 2021] is identify the underlying equilibrium states of FSNN compu- tation so that gradients can be calculated based on implicit differentiation on the equilibrium state. We will briefly introduce the conclusion of equilíbrium states in Section 3.2 and the IDE method in Section 3.3
For more descriptions about the background please refer to Appendix
3.1 SPIKING NEURAL NETWORK MODELS
Spiking neurons draw inspirations from the human brain to communicate with each other by spikes. Each neuron integrates information from input spike trains by maintaining a membrane potential through a differential equation, and generates an output spike once the membrane potential exceeds a threshold, following which the membrane potential is reset to the resting potential. We consider the commonly used integrate and fire (IF) model and simple current model, whose discretized com- putational form is: uc [t + 0.5] u Σ Wigsylti + +
(1)
+|[+1]== (u: |t + 0.5] - Ven), u,[t + =]= 0.5) (Vah- Urest) )s, 1].
u,
where u,(t] is the membrane potential of neuron at time step f, s, [t] is the binary output spike train of neuron i, Way is the connection weight from neuron j to neuron b is bias, H is the Heaviside step function, Vea is the firing threshold, and Mrest is the resting potential. We use subtraction as the reset operation. Wrest is usually taken as O in previous work, while we will reconsider it in Section 4.3
3.2
EQUILIBRIUM STATES OF FEEDBACK SPIKING NEURAL NETWORKS
Xiao et al. ) derive that the (weighted) average rate of spikes during FSNN computation with common neuron models would converge to an equilibrium state following a fixed-point equation given convergent inputs. We focus on the conclusions with the discrete IF model under both single- layer and multi-layer feedback structures. The single-layer structure has one hidden layer of neurons with feedback connections on this layer. The update equation of membrane potentials is:
(2) where uft] and s(t] are the vectors of membrane potentials and spikes of these neurons, x[f] is the input at time step t, W the feedback weight matrix, and F is the weight matrix from inputs to these neurons. The average input and average firing rate are defined as x = E+1 Et= x(r] and
ult + 1] u({] Ws[[] + Fx[[] + b (Ven "rreat + +1,,
alt] = - s(T), respectively. Define (x)) = min(1, (1, max(0, 2)).
The equilibrium state of the single-layer FSNN is described as (Xiao et al. 2021): inputs converge an equilibrium point X - x*, and there exists then the average firing rates of FSNN with discrete IF model will converge to an equilibrium point
If the average such that | S VVth.
az [t] a", which satisfies the fixed-point equation a = aa Via (Wa* Fx* + b)). Note that they take Urest 0 in this conclusion, if we consider nonzero Urest: the constraint and the fixed- point equation should be ||W||2 < (Ven Urest) and ax = Ven-drant (Wa" + Fx + b)).
3

---

Under review as a conference paper at ICLR 2022
The multi-layer structure incorporates more non- con-linearity into the equilibrium fixed-point equation, which has multiple layers with feedback connections from the last layer to the first layer. The update equations of membrane potentials are expressed as:
[1+11] = (4 [1] + w' [4] + F [ + b² - (Vin Wrest )s + 11,
(3)
+ 1] + b' (Ven. (rest)ss( 1}, 1 1= 2, N.
The equilibrium state of the multi-layer FSNN with Ureet is described as (Xiao et al.
202I): If
the average inputs converge to an equilibrium point x[4 + x", and there exists 7 such that [W' Il2... || ll2 S >(vi urest)"* then the average firing rates of multi layer FSNN with discrete IF model will converge to equilibrium points a - a",, which satisfy the fixed- point equations a I* = -i (fN o.... fu(a="), x)) and "+1* = fi+1(a"),, where fi x) =
Mreat (W a + F'x bb)) and fi(()) a Vin Vin-dreed (F'a+b')).
3.3 IDE TRAINING METHOD
3.2 we can train FSNNs by calculating gradients with implicit differentiation (Xiao al.) 2021 Let fo (a) denote the fxed-poin equation of the equilibrium state which is parameterized by 0, go(ax) fa((x) - a, and let L(xx") denote the objective function with respect the equilibrium state a². The implicit differentiation satisfies 8fa(a") da- 8faaa") db
Based on the equilibrium states in Section
(Bai et al. 2019) (we follow the numerator layout convention for
00
derivatives). Therefore, the differentiation of [(a") for parameters can be calculated as:
al(a²) aC(a²) Oa (m--la-) (fa(aa) 09
(4)
00
where J a is the inverse Jacobian of ge evaluated at ax. The calculation of inverse Jacobian can be avoided by solving an alternative linear system (Bai et al. 2019 2020 Xiao et al.) 202I):
(J (a((a')
0a = 0.
(5)
Note that a readout layer after the last layer of neurons will be constructed for output (Xiao et al., 2021], which is equivalent to a linear transformation on the approximate equilibrium state, i.c. the output would be = a", and the loss will be calculated between and labels y with common criterion such as cross-entropy. Then the gradient on the equilibrium state could be calculated. For the solution of implicit differentiation, Xiao al. (222 follow Bai al. (2019 2020) to leverage root-finding methods, while we will solve it by spike dynamics, as will be derived in Section 4 We treat the forward computation of average firing rates [{T] of FSNNs time step T roughly reach the equilibrium state. Then by substituting a by a<(T) in the above equations, gradients for the parameters can be calculated only with a aT] and the equation, and we calculate them based on spikes. With the gradients, first-order optimization methods such as SGD (Rumelhart et al. 1986) and variants can be applied to update parameters.
SPIKE-BASED IMPLICIT DIFFERENTIATION ON THE EQUILIBRIUM STATE
In this section, we present our SPIDE method that calculates the whole training procedure based on spikes. We first introduce ternary spiking neuron couples in Section 4.1 and how to solve implicit differentiation in Section 4.2 Then we theoretically analyze the approximation error and propose the improvement Section
Finally, a summary of the training pipeline is presented in Section 4.4
4.1 TERNARY SPIKING NEURON COUPLES
The common spiking neuron model only generates spikes when the membrane potential exceeds a positive threshold, which limits the firing rate from representing negative information. To enable approximation of possible negative values for implicit differentiation calculation in Section require negative spikes, whose expression could be:
we
1,
u,(t + 0.5] > V
st + 1] T (uilt 0.5], ViA)
0.
| + 0.5] S Vtn
(6)
- 1,
u.(l 0.5] < VeA
and the reset is the same as usual: u,[t+1] us u +0.5] -Vnn (s Direct realization of such ternary output, however, may be not supported by common neuromorphic hardware for SNNs.
4

---

Under review as a conference paper at ICLR 2022
Input X
We propose to leverage two coupled common neurons realize this computation. As illusttrted in Figure [ the two coupled neu- rons with the common IF model (Eq. ( receive opposite inputs and output opposite spikes, which aim to deal with positíve infor- mation and negative information with spikes, respectively. They should share a reset operation in order to accord with Eq. (6), which can be realized by the connection between them: subtraction as the reset operation, the connection whose weight equals ViA Ureat enables one neuron to reset another equiva- lently. To see how this works, consider the condition that the accumulated membrane potential of neuron reaches Vin, then neuron 1 would generate spike and reset, and the output is this positive spike. At the same time, the membrane potential of neu- ron 2 is VVa and the neuron will not fire and reset, but the spike from neuron 1 will reset it to - -rest, which accords with our de- sired reset for ternary output. Similarly, if the inputs are negative, neuron 2 will generate a spike which will be treated negative as output, and both neurons are reset. For the operation of taking negative, one solution is enable the reverse operation hardware, an- other is to reconnect neuron 2 with other neurons while taking the weight negative to that of neuron 1. Therefore, such kind of coupled neurons with the common IF model could realize ternary output.
x
x
(Vi -eress
1
use
W= (Vi Sreest
spike S1
Spike S2
$1 + (-S2)
output
Figure 1: Illustration of ternary spiking neuron couples.
We note that the SpikeGrad algorithm (Thicle et al. 2019a) also requires neurons for ternary output. However, they do not consider how such kind of operation can be implemented with the common neuron model on neuromorphic hardware, and moreover, they propose another modified ternary model in practice that requires consideration of accumulated spikes for spike generation, which is further impractical on neuromorphic hardware. Differently, our method can be realized with the common neuron model suitable for neuromorphic hardware.
4.2 SOLVING IMPLICIT DIFFERENTIATION WITH SPIKES
Based on the coupled neurons in Section 4.1 we can solve implicit differentiation with spikes. For notation simplicity, we directly use Eq. (6) as a ternary neuron without detailing coupled neurons below. Our main focus on solving Eq. (5) with spikes. The brief outline for the derivation is: we first derive the update equation of membrane potentials in SNN computation, then we derive the equivalent equation of the rate of spikes with eliminating perturbation, finally, we could prove that the rate of spikes converges to the solution of Eq. (5)
We first consider the single-layer condition. Let a [T'F']] denote the average firing rate of these neu- rons after the forward computation with time steps Tr as approximate equilibrium state (we treat
the forward procedure as the first stage), g = 0cx TF)
ac
denote the gradient the loss function
on this approximate equilibrium state, and m o' (a [ F)), M Diag (m) denote a mask indicator
We will have another 1, < <
based on the firing condition in the first stage, where o(xx)
0, else
TE time steps in the second backward stage to calculate implicit differentiation. We set the input to these neurons as g at all time steps, which can be viewed as input currents (Zhang & Lij |2020] Xiao al. 202T). Then along the inverse connections of neurons and with mask neurons or weights and an output rescale, the computation of FSNN with ternary neurons is calculated as:
u(L + 1] = u[(]= + Via Uyest
(MW) Ts[4 + (Vhh - - (lees)s(( + 1],
where Vi,, Urest and Vih. uhest are the threshold and resting potential during the first and second stage, respectively. Define the 'average firing rate" at this second stage as B[|] Et_ s(r]. and u(0] = [ 0, then through summation, we have:
B + 1] Vbb weest (++IV Urest
- (MW) 3[0 +
1
u/l ul(+1]
(8)
i+1
Since there could be at most t spikes during t time steps, B would be bounded in the range of [-1, The membrane potential uz(!] will maintain the exceeded terms, i.e. de-
fine v,[(] = EFI Via Moast (MW)T3() + 8)
we can divide uz(t] as u + uf((], where
5

---

Under review as a conference paper at ICLR 2022
u [t] max [v. -
Vth 0) min (v.[t) + Vih 0) is the exceeded term while u [t] is bounded term (iiao al. 2021) which typically bounded in the range of Vilo Vell- Then, Eq. (8) into:
turns
1
1
u " 11 t+1
g + 1] = o
(MW) B g
1/ Wfest
{ + 1
ViA
Ureat
where (xx) = min(1, max( (1ax(-1,x))) Note that if the input g and weight (MW) are in an appro- priate range, there would be no exceeded term and therefore & will not take effect. Indeed we will rescale the loss to control in an appropriate range, as will be indicated in Section 4.3 With this consideration, we could derive that the average firing rate 3it] converges to the solution of Eq. (
Theorem 1. If there exists 7 < 1 such that (I(W)) ll2 S (Vvn " th uhest). then the average firing rate B[1] will converge to an equilibrium point ((4 - 3r. When Via Whest = 1, and there exists X < 1 such that ([(MW) T S A(en - Urest) and ||gll> 1 - X, then f is the
solution of Eq. (5)
The proof and discussion of assumptions are in Appendix With Theorem T we can solve Eq. ( by simulating this second stage of SNN computation to obtain the 'firing rate' [[TB] as the ap- proximate solution. Plugging this solution to Eq. (4). the gradients can be calculated by: VwC-
Via=dren (BTT VrL
Vía pecd M VbC=
Vie- Myeat M3(Tø].
Note that in practice, even if the data distribution is not properly in the range of ø, 6 as a kind of clipping for improperly large numbers, which could be similar to empirical techniques like "gradient clipping" to stabilize the training.
still view
Then we consider the extension to the multi-layer condition. Let a' [TF),, 1 = 1,2, NN denote
the average firing rate of neurons in layer after the forward computation, g Tox ac
denote the gradient of the loss function on the approximate equilibrium state of the last layer, and m' of (TT]), M' = Diag(m') denote the mask indicators. Similarly, we will have another Tn time steps in the second stage to calculate implicit differentiation. We set the input the last layer as at all time steps. Then along the inverse connections of neurons and with a mask on neurons or weights and an output rescale, the computation of FSNN with ternary neurons is calculated as:
u N it + 1] u [t]
(M'w">
[t] + g (V JT
S
"s (tt 1],
Veh Urest
u' +1 1++1===p [t
(M+
s/ (Vth
uhe the )s + +1,, 1= - 1,
Ven Urest
(10)
The 'average firing rates' B [t]] are similarly defined for each layer, and the equivalent form can be similarly derived as:
1
B 11+
1]
BN + =
(M'W') TB +g
V/ west
t + Ven Urest
t t+
u B it + 11
g [|++1]== 1]
(M++I1+1))
g T +1]
wheat
Ven Urest
t t+1
th
(11)
The convergence of the 'firing rate' at the last layer to the solution of Eq. (5) can be similarly derived as Theorem However, we need to calculate gradients for each parameter as Eq. 4. which is more complex than the single layer condition. Actually, we can derive that the 'fring rates' at each layer converge to equilibrium points, based which the gradients can be casily calculated with information from the adjacent layers. Theorem gives a formal description.
Theorem 2. If there exists < 1 such that [MM"W [M 112.... 1I < rVin - Urest) N (Vh ubest) then the average firing rates (((t) will converge to equilibrium points (L()) + Bt. When Vbb. uhest = 1, and there exists X < 1 such that (M'' T 1loc <
X(ViR Urest); FM Mmx S >(Vih urest);l 2.... ,N and ||g]l>o S 1 - XN, then 3N
the solution of Eq. (5 and 1 = 22.
(Bhw(a 3N" - -1.... ,1, where hi(a>
Ji o..... f2 (Ji(a""* ,x²) ,1 ,1= N.

---

Under review as a conference paper at ICLR 2022
The functions/ fi are defined in Section 3.2
For the proof please refer to Appendix With Theo- rem by plugging the solutions explicitly into Eq. (4). the gradients can be calculated by V
M M Tnaaa [Ty]] [TT]]
,
2. ,. V VILL Vin Ureat
M B [T] VwiL
Ven Great
[T) [T>]", V Ver-trest [TBl
Van
Note that the gradient calculation shares an interesting local property, i.c. it is proportional to the firing rates of the two neurons connected by it: VF,,, _ Ves -uestt 3 During calculation, since we will have the firing rate of the first stage before the second stage, this calculation can also be carried out by event-based calculation triggered by the spikes in the second stage. So it would be plausible on neuromorphic hardware as well.
Also, note that the theorems still hold if we degrade our feedback models to feedforward ones by setting feedback connections as zero. In this setting, the dynamics and equilibriums degrade to direct functional mappings, and the implicit differentiation degrades to the explicit gradient. We can still approximate gradients with this computation.
In the following, we take V -uhest = 1 by default to fulfiil the assumption of theorems (it may take other values if we correspondingly rescale the outputs and we set 1 for simplicity). Other techniques like dropout can also be integrated into the calculation. Please refer to Appendix D for details.
4.3 REDUCING APPROXIMATION ERROR
Section 4.2 derives that we can solve implicit differentiation with spikes, as the average firing rate will gradually converge to the solution. In practice, however, will simulate SNNs for finite time steps, and a smaller number of time steps is better for lower energy consumption. This will introduce approximation error which may hamper training. In this subsection, we theoretically study the approximation error and propose to adjust the resting potential to reduce it. Inspired by the theoretical analysis on quantized gradients (Chen et al. 2020), we will analyze the error from the statistical perspective.
For the 'vverage firing rates' ('( in Eq. (8) and the multilayer counterparts, the approximation error e to the equilibrium states consists of three independent parts Ge. and e;: the first is u u
that the exceeded term due to the limitation of spike number, the second is u [t + 1] which can be viewed as a bounded random variable, and the third is the convergence error of the iterative update scheme without u'(t + 1]. i.e. let b([(] denote the iterative sequences for solving 3 as b' b'[t+1]= I VyA - (M++F'+ " bb't), the convergence error is ||b [|. The second part e, be again decomposed into two independent components e, Ey + eg: Cg is the quantization effect due to the precision of firing rates ( for T time steps) if we first assume the same average inputs at all time steps, and eg is due to the random arrival of spikes rather than the average condition, as there might be unexpected output spikes, e.g. the average input is 0 and the expected output should be O, but two large positive inputs followed by one larger negative input at the last time would generate two positive spikes while only one negative spike. So the error is divided into: e + Eq ee + ei. Since the iterative formulation is certain for é; we focus on é,, Eg and e,,
Firstly, the error Ey due the quantization effect is influenced by the input scale and time steps TB- To enable proper input scale and smaller time steps, we will rescale the loss function by a factor st. since the magnitude of gradients considering the direct cross-entropy loss function is relatively small. We scale the loss to an appropriate range so that information can be propagated by SNNs in smaller time steps, and most signals are in the range of / as analyzed Section 4.2 The base learning rate will be scaled by correspondingly. This is also adopted by Thiele et al. (2019a).
Then given the scale and number of time steps, Ey. and e, can be treated as random variables from statistical perspective, and we view 3 as stochastic estimators for the equilibrium states with e,, For the stochastic optimization algorithms, the expectation and variance of the gradients are important for convergence and convergence rate (Bottou| of gradients and smaller estimation variance. As for e, and e., they depends on the input data and the expectations are E[e=] O, E[es] 0 (the positive and negative parts have the same probability). While for eq. it will depend on our hyperparameters V and uh Since the remaining terms in u
i.e. we hope an unbiased estimation
B [t + 1] caused by the quantization effect is in the range of [ubeect V for positive terms while
v u ceee]] for negative ones, given Vib whest and considering the uniform distribution, only
7

---

Under review as a conference paper at ICLR 2022
when ufeet - the expectation B/ey] O for both positive and negative terms. Therefore, should adjust the resting potential from commonly used 0 (Wu et al. 2018 Sengupta et al. 2019 Xiao et al. 2021 to Ven for unbiased estimation, as described in Proposition
Proposition For fixed Vth. -Keest and uniformly distributed inputs and Eg. only when u -V the [tt] are unbiased estimators for b b'[().
Also, taking uhest = - achieves the smallest estimation variance for the quantization effect Eg. considering the uniform distribution on [ugest: Vikl U - -ueest] 6 are independent of Eg and their variance is certain given inputs, it leads to Proposition 2
Since the effects of eas eg and
Proposition 2. Taking uhest -V reduces the variance of estimators B [t].
With this analysis, we will take VA 0.5, uheer -0.5 in the following to stabilize the training. For Vea and Urest during the first forward stage, we will also take Urest
Vvh-
4.4 DETAILS AND TRAINING PIPELINE
The original IDE method (Xiao et al. 2021) leverages other training techniques including modi- fied batch normalization (BN) and restriction on weight spectral norm. Since the batch statistical information might be hard to obtain for calculation on neuromorphic hardware and we seek for algo- rithms that could be possible on it, we drop the BN component in our SPIDE method. The restriction on the weight norm, however, is necessary for the convergence of feedback models, as indicated in theorems. We will adjust for a more friendly calculation, please refer to Appendix D for details.
We summarize our training pipeline as follows (we also provide detailed pseudocodes in Ap- pendix There are two stages for forward and backward procedures respectively. the first stage, SNNs will receive inputs and perform the calculation as Eq. 2 for TF time steps, after which we get the output from the readout layer, and save the average inputs as well as the average firing rates and masks of each layer for the second stage. In the second stage, the last layer of SNNs will receive gradients for outputs and perform calculation along the inverse connections as Eq. 7 for Ta time steps, after which we get the "average firing rates' of each layer. Based on the firing rates from two stages, the gradients for parameters can be calculated as in Section 4.2 and then the first-order optimization algorithm is applied to update the parameters.
EXPERIMENTS
In this section, we conduct experiments to demonstrate the effectiveness of our method and the great potential for energy-efficine training. We simulate the computation common computational units. Please refer to Appendix D for implementation details and descriptions.
We first evaluate the effectiveness of our method in small number of time steps. As shown in Table we can train nigh-performanee models with low latency (Tp = 30) in a very small number time steps during training (e.g. Tр 50), which indicates the low latency and high energy effi- ciency. Note that the ANN-SNN methods usually require hundreds to thousands of time steps just for satisfactory inference performance, and direct training methods show that relatively small time steps are enough for inference, while we are the first to demonstrate that even training of SNNs can be carried out with spikes in a very small number of time steps. This is due our analysis and improvement to reduce the approximation error, as illustrated in the ablation study in Appendix F.T
Then we analyze the firing rate statistics to demonstrate the potential of energy efficiency. Since the energy consumption on event-driven neuromorphic hardware is proportional to the number of spikes, we present the average firing rates for forward and backward stages (for backward, both positive and negative spikes are considered as firing) in Figure It shows the firing sparsity of our method, and spikes are sparser the backward stage with around only 3%. Combined with the small number of time steps, this demonstrates the great potential for the energy-efficiee training of SNNs based on our method on neuromorphic hardware.
Finally we evaluate the performance of our method on MNIST (LeCun et al. 1998), CIFAR- and CIFAR-100 (Krizhevsky & Hinton 2009). We compare our method to several ANN-SNN meth- ods (Hunsberger & Eliasmith 2015) Sengupta et al. 2019 Deng & Gu 2021), direct SNN training methods (Wu et al., 2018 Xiao et al. 2021), and SpikeGrad (Thiele et al. 2019a). As shown in train models with small number of time steps and our trained models achieve
"abbee
8

---

models while they Under review as a conference paper at ICLR 2022
competitive results on MNIST and CIFAR-10. 10. Compared with the original IDE method (Xiao al. 2021), since we discard the BN component, our generalization performance is poorer (a detailed discussion is in Appendix F.2) Compared with SpikeGrad (Thile al. 2019a), we can use fewer neurons and parameters due to flexible network structure choices, and a small number of time steps while they do not report this important feature. Besides,
common
require impractical models, as indicated in Section 4.1 The results and discussion on CIFAR-100 and CIFAR IO-DVS
Appendix F33 and F.4 due the space limit, and our model could achieve 64.07% and 60.7% accuracy respectively. The result on CIFAR- 100 is competitive for networks without BN, though it poorer than IDE with BN. And the result on CIFARIO-DVS among results of common SNN models. It shows the effectiveness of our method even with con- straints of purely spike-based training. Future work could seek normalization techniques friendly for neuromorphic computation and our desired algorithm to further improve the performance.
competitive
Figure The average firing rates for forward and back- ward stages during training. 'A' means AlexNet-F, 'C' means CIFARNet-F, and means time steps for the backward stage.
Evaluation of training with different time steps in the backward stage. Training is on CIFAR-I0 with AlexNet-F structure and TF 30. Results are based on 3 runs of experiments.
Table 2:
0.1
haaued
Average Firing Raties
0.08
Baddowrd
Fanwed
Badkward
Fanued Radbard
Ts
Std (Best)
Funved
98.41%:100.48
(89.07%)
T-200
100
89.17% (89.35%)
250 89.61 +...% 11% (89.70%)
Hadoward
500
89.57%10.088 (89.67%)
21
31
51
61
91
Epochs
Table 3: Performance on MNIST and CIFAR-10. Results are based on runs of experiments.
MNIST
Method
Network structure
Tr
TB
Sta (Best)
Neurons
Params
BP [Tee et 2016
DBCS-P2-50CS-P2-000
>-00
(99.31%)
33K
518K
STBP (Wo 2013
15CS-P2-60CS-P2-800
30
(99.42%)
26K
607K
IDE
(164C55
99.53%:500004%
SpikeGrad Thiele et
2019a)
15C5-P2-40C5-P2-88
Unknown
Unknown
09.38%:1-0.04
(99,52%)
26K
607K
SPIDE (ours)
64C55-6455-66CC (164C3u)
30
100
99.34%:10000% (99.37%)
20K
275K
SPIDE (ours, degraded)
15C5 P2 40C5 P2-300
30
100
99.44% 10.02 (99.47%)
26K
607K
CIFAR-10
Method
Netwrr
Mian-iSi (Best)
Neurons
ANN-SNN (Hunsberger & Eliasmith 2015
AlexNet
80
(83,52%)
595K
21M
LNN-SSNN Sengupla
VGG-16
2500
(91.55%)
311K
15M
ANN- SNN (Dnn & Gu 204
CIFARNet
400-600
(90.61%)
726K
45M
STRP
Wi
AlexNet
12
(85.24%)
595K
21M
STBP (wlo Nou Wi et
2019
CIPARNet
12
(89.83%)
726K
45M
STBP
(Xiao
AlexNet-F
30
(87.18%)
IDB
Xiao el a
122
AlexNet-IF
30
91.74% -.0009% (91.92%)
159K
3.7M
IDE
Xioo 0
2021
CIFARNet-F
30
92.08% 14% (92.23%)
232K
11.8M
SpikeGrad
Thiele et
2019g
CIFARNet
Unknown
Unknown
89.49% 28% (89 99%)
726K
45M
SPIDE
AlexNet-F
30
250
89.61%
(89. 70%)
159K
3.7M
SPIDE (ours)
CIFARNeI-F
30
250
89.94%8-10.1179 (90.139%
232K
11.8M
CONCLUSION
In this work, we propose the SPIDE method that generalize the IDE method to enable the whole training of SNNs with either feedback or degraded feedforward structures to be based on spikes with common neuron models. We prove that the implicit differentiation can be solved with spikes by our coupled neurons. We also analyze the approximation enror due to finite time steps, and propose adjust the resting potential of SNNs. Experiments show that could achieve competitive performance with a small number training time steps and sparse spikes, which demonstrates the great potential of method for energy-efficient training of SNNs neuromorphic hardware.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-lcaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et TrueNorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE Transactions Computer-Aider Design of Integrated Circuits and Systems, 34(10):153-11557 2015.
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. Deep equilibrium models. In Advances in Neural Information Processing Systems, 2019.
Shaojie Bai, Vladlen Koltun, and J Zico Kolter. Multiscale deep equilibrium models. In Advances in Neural Information Processing Systems, 2020.
Shaojie Bai, Vladlen Koltun, and Zico Kolter. Stabilizzng equilibrium models by jacobian regular- ization. In International Conference on Machine Learning, 2021.
Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-term memory and learning-to-leam in networks of spiking neurons. In Advances in Neural Information Processing Systems, 2018.
Sander M Bohte, Joost N Kok, and Han La Poutre. Error-backpropaggatin in temporally encoded networks of spiking neurons. Neurocompating, 17-1-4 2002.
Léon Bottou. sarge-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT'2010. 2010.
Jianfei Chen, Yu Gai, Zhewei Yao, Michael W Mahoney, and Joseph E Gonzalez. A statistical framework for low-bitwidth training of deep neural networks. In Advances in Neural Information Processing Systems, 2020.
Mike Davies, Narayan Srinivasa, 'sung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1):82-99, 2018.
Shikuang Deng and Shi Gu. Optimal conversion of conventional artificial neural networks to spiking neural networks. In International Conference Learning Representations, 2021.
Peter U Diehl and Matthew Cook. Unsupervised learning of digit recognition using spike-timing- dependent plasticity. Frontiers in Computational Neuroscience, 9:99, 2015.
Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. In- corporating learnable membrane time constant to enhance learning of spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
Jordan Guerguiev, Timothy P Lillicrap, and Blake A Richards. Towards deep learning with segre- gated dendrites. Elife, 6:e22901, 2017.
Eric Hunsberger and Chris Eliasmith. Spiking deep networks with LIF neurons. arXiv preprint arXiv:1510.08829 2015.
Michael F Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Communications in Statistics-Sime Simulation and Computation, 18(3):1059-1077 1989.
Yingyezhe Jin, Wenrui Zhang, and Peng Li. Hybrid macro/micro level backpropagation for training deep spiking neural networks. In Advances in Neural Information Processing Systems, 2018.
Jinseok Kim, Kyungsu Kim, and Jae-Joon Kim. Unifying activation-and timing-based learning rules for spiking neural networks. In Advances in Neural Information Processing Systems, 2020.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, University of Toronto, 2009.
10

---

Under review as a conference paper at ICLR 2022
Jonas Kubilius, Martin Schrimpf, Kohitij Kar, Rishi Rajalingham, Ha Hong, Najbb Majaj. Elias Issa, Pouya Bashivan, Jonathan Prescott-Roy, Kailyn Schmidt, et al. Brain-like object recognition with perh-performin shallow recurrent anns. In Advances in Neural Information Processing Systems, 2019.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 36(11):2278-23244 1998.
Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer. Training deep spiking neural networks using backpropagation. Frontiers in Neuroscience, 10:508, 2016.
Robert Legenstein, Dejan Pecevski, and Wolfgang Maass. A learning theory for soward-modulate cetp-tiiinnnddeeedeeeee plasticity with application to biofeedback. PLoS Comput Biol, 4(10): e1000180, 2008.
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in Neuroscience, 1::09, 2017.
Emre O Neftei, Charles Augustine, Somnath Paul, and Georgios Detorakis. Event-driven random back-propagation: Enabling neuromorphic deep learning machines. Frontiers in neuroscience, 11:324, 2017.
Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient- based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51 (-66)::1133,,) 2019.
Arild Nøkland. Direct feedback alignment provides learning in deep neural networks. In Advances in Neural Information Processing Systems, 2016.
Peter O'Connor and Max Welling. Deep spiking networks. arXiv preprint arXiv:1602.0822. 2016.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. In Advances in Neural Information Processing Systems, 2019.
Jing Pei, Lei Deng, Sen Song, Mingguo Zhao, Youhui Zhang, Shuang Wu, Guanrui Wang, Zhe Zou, Zhenzhi Wu, Wei He, et al. Towards artificial general intelligence with hybrid Tiajjic chip architecture. Nature, 572(7767)100--111,22 2019.
Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy. Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. In International Conference on Learning Representations, 2019.
Kaushik Roy. Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike- spike-based machine intelligence with neuromorphic computing. Nature, 575(7784)607-617,, 2019.
Bodo Rueckauer,
Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Con- version of continuous-valuee deep networks to efficient event-driven networks for image classifi- cation. Frontiers in Neuroscience, 11:622, 2017.
David E Rumelhart, Geoffrey E. Hinton, and Ronald J Williams. Learning representations by back- propagating errors. Nature, 32366688) 1986.
Arash Samadi, Timothy P Lillicrap, and Douglas B Tweed. Deep learning with dynamic spiking neurons and fixed feedback weights. Neural Computation, 29(3):578-60) 602, 2017.
Abhronil Sengupta, Yuting Ye, Robert Wang. Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in Neuroscience, 13:95, 2019.
Sumit Bam Shrestha and Garrick Orchard. Slayer: spike layer error reassignment in time. In Advances in Neural Information Processing Systems, 2018.
Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagorce, and Ryad Benosman. Hats: Histograms averaged time surfaces for robust event- event-based object classification. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, 2018.
11

---

Under review as a conference paper at ICLR 2022
Johannes C Thiele, Olivier Bichler, and Antoine Dupret. Spikegrad: An ann-equivalent computation model for implementing backpropagation with spikes. In International Conference on Learning Representations, 2019a.
Johannes C Thiele Olivier Bichler, Antoine Dupret, Sergio Solinas. and Giacomo Indiveri. A spik- ing network for inference of relations trained with neuromorphic backpropagation. In 2019 Inter- national Joint Conference on Neural Networks (IJCNN), 2019b.
Jibin Wu, Yansong Chua, Malu Zhang, Guoqi Li, Haizhou Li, and Kay Chen Tan. A tandem learning rule for effective training and rapid inference of deep spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2021.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in Neuroscience, 12:331, 2018.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI Conference on Artificial Intelligence, 2019.
Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Yisen Wang, and Zhouchen Lin. Training feed- back spiking neural networks by implicit differentiation on the equilibrium state. In Advances in Neural Information Processing Systems, 2021.
Zhanglu Yan, Jun Zhou, and Weng-Fai Wong. Near lossless transfer learning for spiking neural networks. In Proceedings of the AAAI Conference Artificial Intelligence, 2021.
Wenrui Zhang and Peng Li. Spike-train level backpropagation for training deep recurrent spiking neural networks. In Advances in Neural Information Processing Systems, 2019.
Wenrui Zhang and Peng Li. Temporal spike sequence learning via backpropagation for deep spiking neural networks. Advances in Neural Information Processing Systems, 2020.
Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with directly-traines larger spiking neural networks. In Proceedings of the AAAI Conference Artificial Intelligence, 2021.
12

---

Under review as a conference paper at ICLR 2022
A MORE BACKGROUND ABOUT THE IDE TRAINING METHOD
Due to the complex spiking neuron model which is discontinuous, directly supervised training of SNNs is hard problem, since the explicit computation is non-differentiable and therefore backpropagation along the forward computational graph could be problematic. The IDE training method (Xiao et al. 2021> considers another approach to calculating gradients that does not rely on the exact reverse of the forward computation, which avoids the problem of non-differentiabiiit as well as large memory costs by BPTT-like methods with surrogate gradients. Specifically, the IDE training method first derives that the (weighted) average firing rate of FSNN computation with common neuron models would gradually evolve an equilibrium state along time, which follows a fixed-point equation. Then by viewing the forward computation of FSNN as a black-box solver for this equation, and applying implicit differentiation on the equation, gradients can be calculated only based on this equation and the (weighted) average firing rate during forward computation rather than the exact forward procedure. Therefore, the forward and backward procedures are decoupled and the non-differentiaabiity is avoided.
As briefly introduced Section 3.2 the IDE method defines the average firing rates of spikes during forward computation, i.e. ax Then IDE could derive equivalent update equation for (xt), based on the integrated update equations of membrane potentials Eq. 2 With the equivalent equation for ox IDE proves that under certain conditions, ox[[] converges to an equilibrium state following fixed-point equation. With the assumption that [<]) after simulation of T time steps roughly follows the fixed-point equation of the equilibrium state, gradients of the loss function for parameters can be calculated by implicit differentiation, as introduced in Section 3.3 So the training pipeline for the IDE method can be summarized as: first simulate FSNN computation for T time steps to obtain the rate of spikes ax [T], then solve the implicit differentiation by root-finding methods for gradient calculation based on [ and the derived fixed fixed-point point equation of equilibrium states, finally apply gradient-based optimizers to update parameters.
Contribution of this work compared with IDE. Compared with IDE, this work extends the thought of equilibrium of spikes to solving implicit differentiation, which enables the whole training procedure to be based on spike computation with common neuron models and provides the potential for energy-efficies states of forward FSNN computation and requires general root-finding methods with complex com- putation to solve implicit differentiation for training. It remains unclear if it could be solved by common spiking neuron models with only positive firing. This work design ternary spiking neuron couples and prove that the equilibrium of spike computation can be leveraged to solve implicit dif- ferentiation based on the design, and we propose to modify the resting membrane potential to make it practical in relatively small number of time steps. This enables the proposed SPIDE method be the first to train -egh-peeforrance SNNs with low latency and firing sparsity by spikes with common neuron models, demonstrating the great potential for SNNs on neuromorphic hardware.
training of SNNs neuromorphic hardware. IDE only derives the equilibrium
training of perforaefor
B PROOF OF THEOREM 1
Proof. We first prove the convergence of (tt)). Let Vu and V denote Via- - Ureat and V/b uest respectively. Consider ||8[t + 1] [[t]]], it satisfies:
| + 1] - M|ll
t 1
u B it + 1
1
(MW) + +g-
1+1 V
1+1
t
"
(MW) Bit 1] + g
Vn
1
3(1] + 8 2)) - - V V
1
<
1
(MW)
(MW) B[|
TB B(t- 1[t-1]+3 +g
44
V
+
(MW) 31] +g
u
n
( -
[t
1
MW) B|4 + +g]
1+1V
13

---

Under review as a conference paper at ICLR 2022
1
u
1 )) - 1
1
+
(MW) 3|1 - 1| + g
(MW)
3|( - 1] +
V
a
1
(MW) ((1] + 8 8))
1
1
(MW) T B - 1] +
14
Vi
(1
1
1
It + 11
u [t
+
V
1+1V
(MW) 3/1
1+1
+ iV (MW) Bic 11
(12)
As (IMW) T 1 |||||| + V.. [|gl] +
vV.. V. < 1, and |uf [!] is bounded, we have || + 1]]l
1)
S > |||||||| + e, where e is a constant. Therefore ||]
41
bounded. Then VC > 0, hT, such that when
Ti, we have:
V ()
1
(MW) 3101
u L + 1]
1 (MW) BlC 11
u 14
<(1 7)
t 1v
+
V.
2 (13)
And since (MM)) VVa V.. we have:
1 (MW) [[]
+ ))+
(MW) 3C 1]+ 8) 8)) < -[00--1[1]]] 1]]l
Φ
11811|
Vb Vu
Vb
Vu
(14)
Therefore, when t >> Tı, it holds that:
||s| + 1] - B|t]| S 7 [[]]|] Bit -1]l + <(1 - 7)
(15)
2
By iterating the above inequality, we have 3| + 3[4] < 7 71 Ti 31 + 1] 3[Ti]] + <(1->) (1+7+ ... + --1- <
- Ti
|| +1] [
5. There exists T2 such that when
1> T (1 1] ( < 5. and therefore || 1] |[4] e. According
to Cauchy's convergence test, the sequence { converges to B Considering the limit, it
satisfies 3 = Φ v.r (1. (MW) B + g
When Vi - Wrest = 1, and there exists À < 1 such that [[MW) ( & A(Ven Myear) and
llgl S 1 - A, the equation turns into B = Φ v. (MW) 3" + g
We have:
||3 1loo - -110
1 (MW) 3r +8)
V.
(MW) 3r
+ I1glloo S11181 + -lglloo
V.
(16)
Therefore, 11,8 I
|g 1-x < 1, and || (MW) 8 +
1 (MW) 3-
+ +lllll S
100
A+ (1-1)- A)= 1. It means 3 & ( v.. (MW) 3" +g) V. (MW) 3 g.
Taking fo(x²) a ViA Mrest (Wa* + Fx* + b)) (i.e. the fixed-point equation at the equilibrium state in Section 3.2 explicitly into Eq. 5 the linear equation turns into E (MW) T - 7) B + g = O, where V., M,g g are previously defined. Therefore, f satisfies this equation. And since [[MM) IllSS yVu. < 1, the equation has the unique solution 3
Remark 1. As for the assumptions in the theorem, firstly, when Vh - ubest 1 will take, the assumption for the convergence is weaker than that for the convergence in the forward stage (in Section 3.2 because [[(MW) ||2 I ||]|2 as M is a diagonal mask matrix. We will restrict the
14

---

Under review as a conference paper at ICLR 2022
spectral norm of W following Xiao et al. (2027) to encourage the convergence of the forward stage (in Appendix Di. then this backward stage would converge as well.
DI.
The assumptions for the consistency of the solution is sufficient condition. In practice, the weight norm will be partially restricted by weight decay and our restriction on Frobenius norm (in Ap- pendix D). as well the diagonal mask matrix M which would be sparse if the forward firing events are sparse, and we will rescale the loss so that the input g is in an appropriate range, as indicated in Section 4.3 Even if these assumptions are not satisfied, we can view & as kind of empirical clipping techniques to stabilize the training, as indicated in Section 4.2 The discussion is similar for the multi-layer condition (Theorem 2 in the next section.
C PROOF OF THEOREM 2
Proof. We first prove the convergence of (1(t). Let Vu and V denote Ven Urest and Vah ufeest respectively. Let g (3.g.
u²) (.B.g..)) = &
1
1+1 V. (M W T +
t
1
_
v4
++
(3, n8) = Φ V. V. (M++F(+1 B IN (B, g) = o 1
4" )) , , 1, ... N 1,
(M' W ) 3 + +g)
(i( = o 1/2 +/. (M F ) )3)) -- 1.
Then B [ g (9f (... ... (3~ 10 [11) ...,ul
.ul [1]
g. B
lt+ [++
We have:
||3 + 1| [B]
|ost (of - (3 [t], u 18 (t))
,,,u (1)
g. vB B
[t + 11)
siv (oi
... - (B 11, u N B
11
B - 11)
.g. u 1/ 41)1
1la (gl
(..-gN-1 11 (B [1] ...,)) ON (0l
(..-----
(3 I1) ...). .g)
+ 99
off ... ...gg- (3 (1) u N-1 -1)
u 1
g, u 11) IN (9l (..- (t ... ...).))
(0l 11 .... IN (BN it 1), u N- it 11) u it 11 g. ~ [t]
-N (0) (... (..gN- 1N (B\ /1 - 11) ... B)l
s 1lav (01
(N- (3 ).)))) ON (0) (... -N- (B - 1]) ,g))
(1
1 (M'WW) gf (.. (...g--1 (B [t], u N x [1]
1 1
+ vb
[t]]
1
IV
Vn
1
(M' (M'W')) W (0i ... (... -h (B [t), u N 1-11117 [t])
1 [t)
I1 (- IN (3 ((.)))).))
+
Vi
A
(M' (M' )T g6 (... (B - 1], u N N-1 [t 11)
u u -1)l
+
t
V
(M' WInT (o)
-1 (B - 11. u N - 11) ... uu 11) -- (... -N- (3 1I) ....
u
NB t + 1
N t
(17)
t+1
15

---

Under review as a conference paper at ICLR 2022
For the term A and B, they are bounded by:
A s
1
1
(M
1
(M (ob (....iv- (BN)10), [t), uN (1)
u² 20
92 11 (3 [t]) ...))
9N-1
Vb
Vu
Vu
u [t]
+
(MI'W)))
(M'
(M
W
1
(M'
1 B W
[0]
(M'W)) (M (MW') ... (M
- B [1]
Vb
V
V-
Vv V
t
(18)
and B has the same form as A by substituting /. with / 1- 1.
Since (M' ) TllallMM
11 1l2 S VV V. we have:
||a (01 (--- (3 [4] ...)).) IN (0) (.-- (B - 11) ...),,
1 1 (M-W!) (g) (--- (3 [t] ...) 91 (..- -N 1 (B [t -1) ...
V2 V.
(19)
VbN VN (
1
(B 10 11)
[1| - 3 1||l-
And since uf" R [t] is bounded, then Ve 0, 3T; such that when t > Tú, we have:
|| It+ + 3 S 7 || [t] - 3 11/l + c(1 -7)
(20)
2
Then 1| f + 1 when / > Ti + T2. || 1 (M < c. According to Cauchy's 's convergence test, 3 [t]
< < + + -
- 3 [T + 5. and there exists T2 such that
converges to 3 which satisfies 38 9N (9l o...o 9N ).g). Considering the limit,
('(t] converges to Bd" which satisfies 3 (((
When v - ueest = 1, and there exists < such that W I A(Vin Urest); I(MF'') 11 S A(Ven Urest), = 2. ... NN and |gllo $1- A
we have:
( |awv = 19N o oN- 13""
<
1
(M
w 91 o.... 9N-1 0N
+ +lgll
Vn
< 11 ..... 1
+ |1g||
....... s 2nn5 |
+ +1|811
(21)
Therefore, ||.8% <
< 1, and
JN- (
S 52 52/13
S
( a .... JN (13')
< A N-1
ON 51 ..... 1 ):g)
5 A + +(1 2N) =
1, where IN (3, V.
(M-W!) ) B g, ((3) 1/ (M p(+1)
TT,II B1 1,.., N 1,
(i.e. ğr is ai without the function Φ) It means 3Nr = IN 91 .... o IN 1 (BN ).g) =
IN (1 o.... 1N-1 ). and 3d = =
Taking a f1 (fN tion at the equilibrium state as in Section 3.2 explicitly into Eq. (5). the linear equation turns into ÿ1 .... o -v-1__ (3) 3 +g O. Therefore, 8d satisfies this equation. And since Wi)T |all EM)T T F2
fa 1x(x)))x')) and '-1 f (i.e. the fixed-point equa-
?Va 1, the equation has the unique
16

---

Under review as a conference paper at ICLR 2022
solution BN Further, because (1(3) (ohi,
(ii B, where hi(a")) fi o..o
f2 (fi(a"" ,x)) ,I=N,- ..... ,I, have 3 (ahilant)) BN , N -1,.... ,,,,,,
D TRAINING DETAILS
D.I DROPOUT
Dropout is commonly used technique to prevent over-fitting, and follow Bai et al. (2019 2020
Xiao et al. (202T) to leverage variational dropout, i.c. the dropout of each layer is the same at dif- ferent time steps. Since applying dropout on the output neurons linear operation with a mask
and scaling factor, it can be integrated into the weight matrix without affecting the conclusions of convergence, The detailed computation with dropout is illustrated the pscudocode in Appendix E
D.2 RESTRICTION ON WEIGHT NORM
As indicated the theorems, a sufficient condition for the convergence to equilibrium states in both forward and backward stages is the restriction on the weight spectral norm. Xiao al. (2021) leverages re-parameterization to restrict the spectral norm, i.e. they re-parameterize W as "[W where ||l]|2 is computed as the implementation of Spectral Normalization and learnable parameter to be clipped in the range of [-c, cl (c is constant). However, the computation of spectral norm and ce-parameterization may be hard to realize neuromorphie hardware. We adjust it for a more friendly calculation as follows.
First, the spectral norm is upper- hoper-bounded by the Frobenius norm: ||ll2 | We al- ternatively restrict the Frobenius norm which is easier to compute. Further, considering that con- nection weights may not be easy for readout compared with neuron outputs, we can approximate / (ExeNto.Ta) [lle W1l)), according to the Hutchinson esti- mator (Hutchinson) 1989). It can be viewed as source neurons outputting noises and target neurons accumulating signals estimate the Frobenius norm. We will estimate the norm based the Monte CCrlo estimation (we will take 64 samples), which is similarly adopted by Bai et al. (202T) estimate the norm of their Jacobian matrix. Then based on the estimation, we will restrict W as W TWIF where a min(c, [[],),, , e is constant for norm range. This estimation and calculation may correspond large amounts of noises in brains, and feedback inhibition connection weights based on neuron outputs.
|| by | VIr(WWT
w
Following Xiao et al. (2021). we only restrict the norm of feedback connection weight W for the multi-layer structure, which works well in practice.
D.3 OTHER DETAILS
For SNN models with feedback structure, we set Ves = 1, Urest = -1 in the forward stage to form an equivalent equilibrium state Xiao al. (202T) D. The constant for restriction in Appendix D.2 is 2. Following Xiao et al. (221)) we train models by SGD with momentum for 100 epochs. The momentum is 0.9, the batch size is 128, and the initial learning rate 0.05. For MNIST, the learning rate is decayed by 0.1 every 30 epochs, while for CIFAR-10 and CIFAR-100, it is decayed by 0.1 at the 50th and 75th epoch. We apply linear warmup for the learning rate in the first 400 iterations for CIFAR-I0 and CIFAR-100. We apply the weight decay with 5 X 10 and variational dropout with rate 0.2 for AlexNet-F and 0.25 for CIFARNet-F, The initialization of weights follows Wu et al. 2018). i.e. sample weights from the standard uniform distribution and normalize them on each output dimension. The scale for the loss function (as in Section 4.3) is 100 for MNIST, 400 for CIFAR-10, and 500 for CIFAR-100.
For SNN models with degraded feedforward structure, our hyperparameters mostly follow Thiele al. (2019)), i.e. Vin 0.5, Uyest -0.5, train models by SGD with momentum 0.9 for
17

---

Under review as a conference paper at ICLR 2022
60 epochs, set batch size as 128, and the initial learning rate as 0.1 which is decayed by 0.1 every 20 epochs, and apply the variational dropout only on the first fully-connected layer with rate 0.5.
The notations for structures mean: '64C5' represents convolution operation with 64 output channels and kernel size 5, 's' after '64C5' means convolution with stride (which downscales 2x) while 'u' after that means transposed convolution to upscale 2x, 'P2' means average pooling with size 2, and 'F" means feedback layers. The network structures for CIFAR-10 are:
AlexNet (Wu et al.
2019) 96C3 00t.::/000...2233333322300111010
AlexNet-F Xiao et al.
2021): 9CC5:3550C--44C33338833533333333
CIFARNet (Wu et al.
2019):
CIFARNet-F Xiaoo et al.
2021
(F128C33)
We simulate the computation commonly used computational units. The code implementation
based on the PyTorch framework (Paszke et al. 2019). and experiments are carried out on one NVIDIA GeForce RTX 3090 GPU.
E PSEUDOCODE OF THE SPIDE ALGORITHM
Our algorithm consists of two-stage SNN computation, as explained Section The detailed computation for both stages are illustrated in Algorithm | and Algorithm 2
respectively.
Algorithm 1 Forward procedure of SPIDE training Stage 1.
Input: Network parameters Fl b² ... FN I,, W1 Wo,Wr Input data x; Time steps Tri
Forward threshold Va:; Dropout rate Output: Output of the readout layer
Initialize [0] 0, 1,2.
2: If use dropout, randomly generate dropout masks Di 1,,, NN,, DJ with rate D', D are saved for backward
3: for 1,. .TF do
4:
if f- then
5:
u u'[t] = u² 1] + D
O (F'z(t) + +b))
6:
else
7:
1| D2 @ (F x(1 + b4) + DJ の (W1 11)
8:
I' = [[[='[]] Va))
9:
-
|| Urest Vih, the same below
10:
for 2.3,....
n -10=="[[-1]4 D (F b')
12:
8 =H(u'(]] Vvn)
u [t] =u'ff ~V
14:
= 0+ Wo [tt++b
o can be accumulated here, or calculated later by o = a +
15: T
S 16: o Dim s'
1,2, ,N // Save for backward, firing rate in Stage
m² Ha' A H( < 1) Save for backward, mask
18: If constant, & Σr =(d] for backward
19: return
F MORE EXPERIMENTAL COMPARISONS
F.1 ABLATION STUDY
In this section, we conduct ablation study on our improvement to reduce the approximation error by setting the resting potential negative threshold. To formulate equivalent equilibrium states,
18

---

Under review as a conference paper at ICLR 2022
Algorithm 2 Backward procedure of SPIDE training Stage 2.
, W/ wo,.b, Forward output o; Label y; Time steps TB; Forward threshold Ven: Backward threshold V 0.5; Other hyperparameters and saved variables;
Input:
Network parameters F* ,",,',,'" b'..... , b",
Output: Trained network parameters F' b.b. ,b',.... FN bb W W0.6".
Calculate g == @L(o,)) I/ for CE loss aL(o,) softmax(o) y. in practice will scale the do loss by factor st, then @L(o.w) 8t (softmax(o) - y)
1:
do
2: Initialize u'(0 [0] = 0, = 1,2 N
3:
for
,Tn do
4: if =- then
5: u [t] = u.x
+ WO
6: else
7:
u = u - 1] + Wo + 2V W (D
T o m² o 1])
m² is the saved
mask in Stage
8:
[ = T'u [t). (.5) 0.5)
// realized by two coupled neurons // realized by two coupled neurons do
N [t] = u -s sN1
9:
10:
for 1 = 1,
u' w'lt u'[t-1]+
+]+
Fr (D' 3 m't1 (t) I/m the saved mask in Stage
12:
s = [(u'(t],0.5
I/ realized by two coupled neurons [t wt st'(t] realized by two coupled neurons
ااميتا =,,,, .N "firing rate" in Stage
8
15:
Calculate gradients:
(I) VPL i (ml ④ )xx 11 m², are the saved mask and average input in Stage
17:
(2) VF: Vah (m' 0a
#-1 T 2.3..... m'', a² are the saved mask and firing C 33') 1,2, ...
rate in Stage
18:
(3) VnL 2V (m'
(4) Vw 2ViA
(m' 8 Na m² a² are the saved mask and firing rate in Stage
20:
(5) V. = N (BL(oM)
aa is the saved firing rate Stage
do
(6) V @L((ow)
21:
do
22: Update Fl, b² .... F bN W W b based on the gradient- based optimizer SGD learning rate 77 momentum & weight decay μ. the base learning rate is scaled by the factor
81 of the loss, i, =
23:
(1) Update the momentum Mg
Me (1 a) VoC. (F....W WW
24:
(2) Update parameters
(1 μu) = + n My (F W. Wo,8%
25:
(3) Restrict the norm of W1
26: return FL pr..b....., b W1 W
Vb. and we consider the following settings: (1) both forward and backward stages apply our improvement, i.e, Urest Vih, ub u
take the same VeA - urest V, and the same Vh ueett
-VA; (2) remove the improvement on the backward stage, i.e. V 4b ub urest 0: (3) remove the improvement on both forward and backward stages, i.e. Vv, Va, trest Ondd V Vb ub The latter two setting are denoted by "w/o B" and "w/o F&B" respectively.
The models are trained on CIFAR-10 with AlexNet-F structure and 30 forward time steps. The training and testing curves under different settings and backward time steps are illustrated Fig- ure 3 and Figure respectively. It demonstrates that without improvement, the training can not perform well within a small number of backward time steps, probably due to the bias and large variance of the estimated gradients. When the backward time steps large, the performance gap reduced since the bias of estimation reduced. It shows the superiority of our improvement to training SNNs within small number of backward time steps.
19

---

Under review as a conference paper at ICLR 2022
100
90
T=30
80
Wo T=50
Training TrainingAAcuraae Accuracy
-Wo F&B T-50
70
T=100
T=100
60
W8O T=100
T-230
50
Wo T=250
--wo F&2R T-230
40
T=500
w/o B, T=300
30
-o F&JB T-500
20
10
1
11
21
31
41
51
61
71
81
91
Epochs
Figure 3: Comparison of training curves under different settings and backward time steps.
100
90
T=50
We B T+00
WOFF&B T+50
70
Assiiggocuree
T=100
w/o T=100
60
-Wo T-100
T-290
50
W/
40
-woo F&dR T-250
T=5500
30
wro B T=500
woo F&B T=500
20
11
21
31
41
51
61
81
91
Epochs
Figure 4: Comparison of testing curves under different settings and backward time steps.
20

---

Under review as a conference paper at ICLR 2022
100
90
IDE, train
18R, test
80
Accuracy
SIDE, test
70
50
40
30
61
71
Epochs
Figure 5: Comparison of training and testing curves between IDE and SPIDE CIFAR-10 with AlexNet-F structure and Tç 30.
F.2 COMPARISON IDE
Table 3 Section shows that SPIDE method performs poorer than the original IDE method (Xiao al. 202Ib. We further investigate the training and testing curves during optimiza- tion to analyze this phenomenon. As shown in Figure [ the SPIDE method could achieve the same training accuracy the IDE method, while the generalization performance is poorer. Since the hy- perparameters are the same for experiments, except that we drop the modified BN component (as explained in Section 4.4). the performance gap may be caused by the implicit regularization effect of BN. Therefore, the optimization ability of the SPIDE method should be similar to that of IDE, while future work would be to investigate how to realize operations similar to BN with possible computation that could be friendly on neuromorphic hardware.
Also, we note that models trained by the IDE method (Xiao forward average firing rates are only around 1 1% (Xiao et al.) 2021). while ours are around 7% as shown in Figure? It is again probably due to BN which would subtract the statistical mean value of neuron inputs, therefore regularizing the weights so that neurons will generate sparser spikes. An interesting future work is to further reduce the number of spikes considering this phenomenon.
2021 have sparser spikes, as their
F.3 RESULTS ON CIFAR-100
Table 4: Performance CIFAR- 100. Results based on 3 runs of experiments.
Method
Network siructure BN
TF
Ts
Mtd (Rest)
Neurons
Parmss
BP (Thicle et
2019g
CIFARNei
Unknown
(64.699)
726K
45M
X
71.56%:-1.311 172.10%)
SpikeGrrud (Thiele al. 2019
CIFARNet
X
Unknown
Unknown
(64.40%)
726K
45M
SPIDE SPIDE (ours)
CIFARNec-
30
100
1...7%
232K
CIFARNa-FF
x
30
250
64.00% (
232K
14.8M
In this section, we present the results on CIFAR-100. As shown in Table our model could achieve 64.07% accuracy. Compared with IDE, the performance is poorer, and the main reason is probably again the absence of BN which could be important for alleviating overfitting on CIFAR- with relatively small number images per class. The training accuracy of SPIDE is similar IDE (around 93% v.S. around 94%) while the generalization performance is poorer. Despite this, the performance of our model competitive for networks without BN and our model is with fewer neurons and parameters and a small number of time steps. Compared with SpikeGrad (Thiele al.) 2019a), we can use fewer neurons and parameters due to flexible network structure choices, and leverage common could investigate more suitable structures and if there are normalization techniques friendly for neuromorphic computation and our desired algorithm further improve the performance.
models while they require complex impractical models. Future work
21

---

Under review as a conference paper at ICLR 2022
F.4 RESULTS ON CIFAR SEFARIO-DDE 1O-DVS
In this section, we supplement some results on the spiking dataset CIFARIO-DVS (Li al. 2017 The CIFARIO-DVS dataset is the neuromorphic version of the CIFAR-10 dataset converted by Dynamic Vision Sensor (DVS), which is composed of 10,000 samples, one-sixth of the original CIFAR-10. It consists of spike trains with two channels corresponding ON- and OFF-event spikes. The pixel dimension is expanded to 128 x 128. Following the common practice, we split the dataset into 9000 training samples and 1000 testing samples. As for the data pre-processing. we reduce the time resolution by accumulating the spike events (Fang et al. (021) into 30 time steps, and we reduce the spatial resolution into 48 x 48 by interpolation. We apply the random crop augmentation as CIFAR-I0 to the input data. We leverage the network structure: 512C9s (F5 (2C)), where the notations follow Appendix D.3] We train the model by SGD with momentum for 70 epochs. The momentum is 0.9, the batch size is 128, the weight-decay is 5 x 10 4 and the initial learning rate is 0.05 which decayed by 0.1 at the 50th epoch. No dropout is applied. The initialization of weights follows the widely used Kaiming initialization. The constant for restriction in Appendix D.2 is c= 10 due to the large channel size, and the scale for the loss function as well as the firing thresholds and resting potentials are the same as the CIFAR-IO experiment.
Table 5: Performance on CIFARIO-DVS.
Method
Model
Tp Tn Accuracy
Gabor-SNN (Sironi al 2018
Gabor-SNN
/
24.5%
HATS
Sirom al 2018
HATS
/
52.4%
STBP
Wi
Spiking CNN (LIF, w/o NeuNorm)
40
58.19
STBP
Wareer au 2019
Spiking CNN (LIF, w/ NeuNerm)
40
60.5%
Tandern Learning
Spiking (IF)
58.65%.
Spike- based BP Fang et al
A1ZT
Spiking CNN (PLIF, w/ BN)
20
/
74.8%
SPIDE (ours)
Spiking CNN (IF)
30
250
60.7%
As shown in Table our model could achieve 60.7% accuracy, which competitive among sults of common SNN models, demonstrating the effectiveness of our method. Fang et al. (2021) leverages many techniques such learnable membrane time constant, batch normalization, and max pooling to achieve better performance. We do not aim at outperforming the state-of-the-an results, but demonstrate that a competitive performance could be achieved even with our constraints of purely spike-basee based training in relatively small number of time steps, verifying the effectiveness.
22


Reviews:
[{"review_id": "cc2tMpSst0t", "reviewer": "Reviewer_dKfz", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposed a meaningful implementation of the spike-based learning method for SNNs with higher plausibility.\nThere have been several works on training SNNs with spike-based learning rules based on the back-propagation or the equilibrium propagation, but most of them failed to provide successful experimental results.\nIt is impressive that the experimental results in this paper showed successful accuracy in several datasets including CIFAR-100.\n\nThe writing can be improved for easier understanding.\nIt is hard to understand basic concepts without reading the reference, so It would be better to explain more details about the background such as the IDE.\nEven though I have not fully understood the mathematical derivations in the paper, it was not easy to follow mathematical derivations in the paper without referring to other papers.\n\n========== Comments after rebuttal ==========\n\nI apologize for the late response and thank the authors for adding a detailed explanation of the proposed idea. It was very helpful to deepen my understanding.\nI agree that this paper has a clear contribution of proposing a fully spike-based derivation of IDE training methods for SNNs.\n\nHowever, there remains another concern.\nI missed in my initial review that this paper solely used the IF neuron model. Can the proposed method also be applied to other spiking neuron models?\n\nEven though the IF neuron model is 'common' in many SNN literature, I strongly believe that the study on SNN is meaningful only when temporal information carried by spikes is utilized in information processing. Averaging out spikes with rate-coding with IF neuron is functionally identical to quantized non-spiking neural networks, in which the activation function has quantized output. Multiplication with linearly-quantized values (e.g. low-bit fixed-point numbers) is already cost-effective and can also be implemented only by accumulation. I don't think there is a justifiable reason to use SNNs instead of this counterpart which can simply use conventional hardware. Major benefits of neuromorphic hardware such as event-drivenness can also be replaced by the zero-skipping feature that is common in NN hardware. Even though the proposed training method is fully spike-based, with the IF neuron model, it seems not to use any temporal information of spikes.\n\nI would like to hear the authors' opinions on this. Thank you.", "summary_of_the_paper": "In this paper, the authors proposed a method to train Spiking Neural Networks (SNN) with spike-based implicit differentiation on the equilibrium state.\nMain idea is to use a spike-triggered event instead of average firing rate to approximate implicit differentiation of Feedback Spiking Neural Networks (FSNN).\nTo enable such idea and to further reduce the approximation errors, the authors proposed several techniques such as adopting ternary spiking neuron couples and shifting resting potential.\nThe experimental results showed that the proposed method can achieve high accuracy in several tasks such as MNIST, CIFAR-10, and CIFAR-100 with fewer time steps for training compared to existing methods.", "summary_of_the_review": "The proposed method effectively handles several obstacles while adopting the existing IDE method with spike-based implementation and provided convincing experimental results.\nHowever, I doubt that the theoretical improvements of this paper is significant enough especially with my poor understanding on the mathematical derivations in the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Ubrdawfds5", "reviewer": "Reviewer_FWiT", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors attempt to support their conclusion with mathematical evidence, which improves the reliability of the quantitative part of the results. Yet every coin has two sides. At the same time, the math details cannot tell the upper bound and the potential of the proposed method, which is my main concern for the series of approaches. \n\nIn this paper, the authors extend the work Xiao et. al., 2021 NeurIPS to elaborate on the ideas of training SNN based on the equilibrium and address some of the shortcomings. Overall, this paper is clearly written and easy to follow. I list my comments and concerns below and am more than happy to discuss with the authors about these issues. \n\n1. I think the idea of approaching SNN training via equilibrium is interesting, but it requires more reasoning about why it has the potential to go beyond existing methods and what is its unique methodological contribution given Xiao et. al., 2021 NeurIPS. \n\n2. If I understand it correctly, the equilibrium basically says that an integrally convergent input would result in an integrally convergent average firing rate. Since the assumption satisfies Lipschitz's condition with L < 1, I feel like the prove could be simplified with some known ODE theorems on perturbation. \n\n3. The derivative on the equilibrium gives the dependence between the convergent $a^*$ and $x^*$. I think it is more related to backpropagation rather than the  Hebbian learning rule. The analog here is not evident. \n\n4. Since the whole framework is built on the equilibrium, it implicitly requires the dynamics to be stable. Thus, it seems that the latency $T_F$ cannot be extremely shortened to an efficient case where the dynamics are not stable across time. As the authors also pointed out, this paradigm is not compatible with setups that may amplify the norm and maps, e.g. the BN-layers in training. This shortcoming from assumption may limit the practical utility of the proposed methods. In practice, the comparison in Table 3 is not up-to-date. For example, Ref [1] and [2-3] significantly shorten the conversion and training latency and work well for big datasets. The experiments in the current work should at least extend to CIFAR-100 and ImageNet. It would be also great if the authors can additionally add the results on the spiking dataset like CIFAR-gesture and CIFAR10-DVS as well. \n\n5. in Figure 3, it seems that the accuracy has a jump around epoch 50. Can the author explain the reason? \n------------------------------------------------------------------------------------------------------------------------------------------------------\n[1] Yuhang Li, Shikuang Deng, Xin Dong, Ruihao Gong, Shi Gu. A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration. *Proceedings of the 38th International Conference on Machine Learning, PMLR 139:6316-6325, 2021.*\n\n[2] Fang, Wei, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. *Conference on Neural Information Processing Systems (2021). *\n\n[3] Wu, Jibin, Yansong Chua, Malu Zhang, Guoqi Li, Haizhou Li, and Kay Chen Tan. A tandem learning rule for effective training and rapid inference of deep spiking neural networks. *IEEE Transactions on Neural Networks and Learning Systems (2021).*\n\n------------------------------------------------------------------------------------------------------------------------------------------------------\nMy comments after the rebuttal: \n\nThe authors made a very attentive rebuttal and emphasized their contribution as a purely spike-based training approach. Yet my two major concerns still remain here. \n1. I cannot fully track every step in the updating formula to make sure that it is purely spiked-based. For example, does the calculation of (10) and (11) and the associated gradient calculation on page 7 still involve multiplication? If the general multiplication is somewhat supported by neuromorphic hardware, the other methods mentioned in the comparison would also work as well? \n2. If we accept the claim it is purely spike-based, we still have the question of whether the potential energy efficiency is worthy. For example, the performance on DVS-CIFAR-10 is ~15\\% less than the mentioned benchmarks on page 22. As the authors mentioned, the energy cost is related to both the latency and spiking rate. Thus it would be necessary to at least provide a comparison of the estimated cost with the relevant approaches and better to provide an implementation of at least the proposed method on proper hardware to support the purely-spike-based assertion. Also, it would be necessary to demonstrate that the proposed method can achieve at least comparable accuracy with Fang et. al., ICCV 2021 to support that the cost of changing to spike-based training is acceptable. \n\nBased on these two points, I tend to keep my rating at 5 but am open to marginally accepting this paper if the other reviewers strongly think it should be accepted. \n------------------------------------------------------------------------------------------------------------------------------------------------------\nLatest update: \n\nI encouragingly increased my score to 6 given the rebuttal effort but my judgment remains on the borderline as the major concerns are not fully addressed with sufficient numeric proof. \n\n", "summary_of_the_paper": "In this paper, the authors extend the work Xiao et. al., 2021 NeurIPS to elaborate on the ideas of training SNN based on the equilibrium and address some of the shortcomings. \n", "summary_of_the_review": "Overall, this paper is clearly written and easy to follow, but the methods may not apply to the more complex scene with less latency. My concern is that this limitation is from its setup. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "OJVvBzgjjR3", "reviewer": "Reviewer_PJRp", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "If one has an application of this method for neuromorphic hardware in mind, it becomes essential that one does not arrive in a rate-based coding regime. It has been demonstrated, e.g. for Intel's Loihi chip in (Davies et al., 2021) that one hardly gets and energy advantage in spike-based hardware if one works in a rate-based coding regime. Hence I am missing a discussion of this issue, and methods for arriving at sparsely firing neurons. If one aims instead at biologically plausible models, one also would have to look for solutions with biologically realistic firing rates of a few Hz.\n\nAltogether I find it difficult to identify the specific innovations of this paper because of its inadequate literature review. In particular, there are already quite a number of spiking neural network solutions that achieve comparable or higher accuracies with comparable latency and number of parameters, see e.g. \n\nZhou S, Li X, Chen Y, et al. Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performance[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2021, 35(12): 11143-11151.\nFang W, Yu Z, Chen Y, et al. Incorporating learnable membrane time constant to enhance learning of spiking neural networks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 2661-2671.\n\nThe abstract makes the really imprecise statement \"--most SNN training methods require complex computation or impractical neuron models\", which are hard to reconcile with publications such as\n\nBellec, G., Scherr, F., Subramoney, A., Hajek, E., Salaj, D., Legenstein, R., & Maass, W. (2020). A solution to the learning dilemma for recurrent networks of spiking neurons. Nature communications, 11(1), 1-15.\n\nThe authors should summarize and discuss more benefits of the proposed method. It is important for enhancing the impact of this work and drawing more audiences in the neuromorphic community.  In section 4.1, please explain why negative information is necessary for implicit differentiation calculation. \n\n\n", "summary_of_the_paper": "The paper aims at porting the IDE method  into a spike-fbased and more bio-plausible version. The previous IDE used firing rates rather than spikes for computation, although reference Xiao et al in NeurIPS 2021 had already addressed implementations in spiking neural networks. The authors analyze the approximation error that results from solving implicit differentiation by spikes and report a solution based on ternary spiking neurons, that can be implemented with pairs of standard spiking neurons. They achieve in this way quite good performance for MNIST and CIFAR10.", "summary_of_the_review": "The paper presents a nice step in an interesting direction. But it does not manage to clarify what exactly its innovations and possible applications are.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a purely spike based method for training spiking neural networks with recurrence, by extending the recently published "implicit differentiation on the equilibration state (IDE)" technique. As a purely spike based method for both the forward pass and the gradient computation, the proposed technique potentially represents an important advance.

Based on the original submission, the reviewers had difficulties understanding the paper's contributions and verify its claims. I commend the authors for engaging with the reviewers by answering their questions and updating the paper to better explain the contributions. However, even after considerable back and forth, the most positive reviewer still expressed major concerns [[1](https://openreview.net/forum?id=VQyHD2R3Aq&noteId=Ubrdawfds5)], and the other reviewers appeared unmoved, based on their scores.

The reviewers' principal concerns were a little hard to distill. It is possible that their initial difficulties with understanding and validating the paper's contributions made it hard to fully appreciate the paper. One reviewer is unable to verify that the algorithm is purely spike based, and that the energy costs are appropriately calculated. They are also unsure if the method will scale to more complex settings [[1](https://openreview.net/forum?id=VQyHD2R3Aq&noteId=Ubrdawfds5)]. A second reviewer was also initially unable to verify the same claims, was unsure if the theoretical improvements were sufficiently significant, and whether the method could apply to non-IF models of spiking neural networks [[2](https://openreview.net/forum?id=VQyHD2R3Aq&noteId=cc2tMpSst0t)]. The authors addressed this in their response by performing additional experiments with LIF neurons, but it wasn't clear if the main concerns were sufficiently addressed, since the reviewer did not change their score.

Based on the largely negative appraisal by all reviewers, I recommend that be paper be rejected. However, I strongly encourage the authors to revise and resubmit their paper to a future conference, focusing on making sure that their central claims can be more easily understood and verified.
Decision: Reject