Online Training Through Time for Spiking Neural Networks
Authors: Mingqing Xiao
Qingyan Meng
Zongpeng Zhang
Di He
Zhouchen Lin
Venue: NeurIPS 2022 Accept (2022)
Keywords: spiking neural networks
online training through time
neuromorphic computing

Abstract: Spiking neural networks (SNNs) are promising brain-inspired energy-efficient models. Recent progress in training methods has enabled successful deep SNNs on large-scale tasks with low latency. Particularly, backpropagation through time (BPTT) with surrogate gradients (SG) is popularly used to enable models to achieve high performance in a very small number of time steps. However, it is at the cost of large memory consumption for training, lack of theoretical clarity for optimization, and inconsistency with the online property of biological learning rules and rules on neuromorphic hardware. Other works connect the spike representations of SNNs with equivalent artificial neural network formulation and train SNNs by gradients from equivalent mappings to ensure descent directions. But they fail to achieve low latency and are also not online. In this work, we propose online training through time (OTTT) for SNNs, which is derived from BPTT to enable forward-in-time learning by tracking presynaptic activities and leveraging instantaneous loss and gradients. Meanwhile, we theoretically analyze and prove that the gradients of OTTT can provide a similar descent direction for optimization as gradients from equivalent mapping between spike representations under both feedforward and recurrent conditions. OTTT only requires constant training memory costs agnostic to time steps, avoiding the significant memory costs of BPTT for GPU training. Furthermore, the update rule of OTTT is in the form of three-factor Hebbian learning, which could pave a path for online on-chip learning. With OTTT, it is the first time that the two mainstream supervised SNN training methods, BPTT with SG and spike representation-based training, are connected, and meanwhile it is in a biologically plausible form. Experiments on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS demonstrate the superior performance of our method on large-scale static and neuromorphic datasets in a small number of time steps. Our code is available at https://github.com/pkuxmq/OTTT-SNN.

Online Training Through Time for Spiking Neural Networks
Mingqing Xiao', Qingyan Me Zongpeng Zhang', Di He', Zhouchen Lin L.n 1,56"
Key Lab. of Machine Perception (MoF), School of Intelligence Science and Technology, Peking University
The Chinese University of Hong Kong, Shenzhen
Shenzhen Research Institute of Big Data
'Center for Data Science, Academy for Advanced Interdisciplinary Studies, Peking University Institute for Artificial Intelligence, Peking University
*Peng Cheng Laboratory, China
(mingqing_xiag xiao, dihe, zlin]&pku. edu, en, gingyanmeng@lind cuhk, edu. en, zongpeng zhang96@gmail.cmm com
Abstract
Spiking neural networks (SNNs) are promising brain-inspiree eeergy-efficient models. Recent progress training methods has enabled successful deep SNNs large-scale tasks with low latency. Particularly, backpropagation through time (BPTT) with surrogate gradients (SG) is popularly used enable models achieve high performance in a very small number of time steps. However, it is at the cost of large memory consumption for training, lack of thcoretical clarity for optimization, and inconsistency with the online property of biological learning rules and rules on neuromorphic hardware. Other works connect the spike representations of SNNs with equivalent artificial neural network formulation and train SNNs by gradients from equivalent mappings to ensure descent directions. But they fail to achieve low latency and are also not online. In this work, propose online training through time (OTTT) for SNNs, which is derived from BPTT to enable forward-in-tinn learning by tracking presynaptic activities and leveraging instantaneous loss and gradients. Meanwhile, we theoretically analyze and prove that the gradients OTTT can provide similar descent direction for optimization gradients from equivalent mapping between spike representations under both feedforward and recurrent conditions. OTTT only requires constant training memory costs agnostic to time steps, avoiding the significant memory costs of BPTT for GPU training. Furthermore, the update rule of OTTT in the form of three-factor Hebbian learning, which could pave path for online on-chip learning. With OTTT, it is the first time that the two mainstream supervised SNN training methods, BPTT with SG and spike eepresentationnbasee training, are connected, and meanwhile it is in a biologically plausible form. Experiments on CIFAR- 10, CIFAR- 100, ImageNet, and CIFAR100 11-DDSS demonstrate the superior performance of our method on large- scale static and neuromorphic datasets small number of time steps. Our code is available at httph:tt///// thub. wwt. wwtp///uxmm/OTTT--S
Introduction
Spiking neural networks (SNNs) are regarded as the third generation of neural network models
and have gained increasing attention in recent years 2 3
6
7 g
g
10
[1] 12 [3]
SNNs
composed of brain-inspired spiking neurons that imitate biological neurons transmit spikes
"onresponniig author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

between each other. This allows event-based computation and enables efficient computation on neuromorphic hardware with low energy consumption [14 15 [6]
However, the supervised training of SNNs is challenging due to the non-differentiable neuron model with discrete spike-generation procedures. Several kinds of methods are proposed to tackle the problem, and recent progress has empirically obtained successful results. Backpropagation through time (BPTT) with surrogate gradients (SG) is one of the mainstream methods which enables the training of deep SNNs with high performance on large-scale datasets (e.g., ImageNet) with extremely
low latency (e.g.. 4-6 time steps) 6 10 13 These approaches unfold the iterative expression
of spiking neurons, backpropagate the errors through time 17 and use surrogate derivatives to
19 20 22 23). As a result, during training, they suffer from significant memory costs proportional to the number of time steps. and the optimization with approximated surrogate gradients is not well guaranteed theoretically. Another branch of works builds the closed-form formulation for the spike representation of neurons, e.g. the (weighted) firing rate or spiking time, which is similar to conventional artificial neural networks (ANNs). Then SNNs can be either optimized by calculating the gradients from the equivalent mappings between spike representations 22
approximate the gradient of the spiking function B 4
[8]
9 27 or converted from a trained equivalent The optimization of these methods is clearer than surrogate gradients. However, they require a larger number of time steps compared to BPTT with SG. Therefore, they suffer from high latency, and more energy consumption is required if the representation is rate- base- Another critical point for both methods is that they are indeed inconsistent with biological online learning, which is also the learning rule neuromorphic hardware [15].
24
25 26
ANN counterpart [28
29
30
31
32
7
33
8
34
In this work, we develop novel approach for training SNNs to achieve high performance with low latency, and maintain the online learning property to pave path for learning on neuromorphic chips. We call our method online training through time (OTTT). We first derive OTTT from the commonly used BPTT with SG method by analyzing the temporal dependency and proposing to track the presynaptic activities in order to decouple this dependency. With the instantaneous loss calculation, OTTT can perform forward-in-tine learning, i.e. calculations are done online in time without computing backward through the time. Then we theoretically analyze the gradients of OTTT and gradients of spike representation-br based methods. We show that they have similar expressions and prove that they can provide the similar descent direction for the optimization problem formulated by spike representation. For the feedforward network condition, gradients are easily calculated and analyzed. For the recurrent network condition, we follow the framework in [12] that weighted firing rates will converge an equilibrium state and gradients can be calculated by implicit differentiation. With this formulation, the gradients correspond to an approximation of gradients calculated by implicit differentiation, which can be proved to be a descent direction for the optimization problem as well (35 33). In this way, a connection between OTTT and spike representation-basee methods is bridged. Finally, we show that OTTT in the form of three-factor Hebbian learning rule (37], which could pave a path for online learning on neuromorphic chips. Our contributions include:
1. We propose online training through time (OTTT) for SNNs, which enables forward-in-ting learning and only requires constant training memory agnostic to time steps, avoiding the large training memory costs of backpropagation through time (BPTT).
2. We theoretically analyze and connect the gradients of OTTT and gradients based on spike representations, and prove the descent guarantee for optimization under both feedforward and recurrent conditions.
3. We show that OTTT in the form of three-factor Hebbian learning rule, which could pave a path for on on-chip online learning. With OTTT, it is the first time that a connection between BPTT with SG, spike representation-bice based methods, and biological three-factor Hebbian learning is bridged.
4. We conduct extensive experiments on CIFAR-10. 10, CIFAR- ImageNet, and CIFAR 10- DVS, which demonstrate the superior results of our methods large-scale static and neuromorphic datasets in a small number of time steps.
2 Related Work
SNN Training Methods. for supervised training of SNNs, there are two main research directions. One direction is to build a connection between spike representations (e.g. firing rates) of SNNs with equivalent ANN-like closed-form mappings. With the connection, SNNs can be converted from

---

ANNs 28 29 30
or SNNs can be optimized by gradients calculated from Variants following this direction also include [[2] which connects feedback SNNs with equilibrium states following fixed-point equations instead of closed- form mappings. These methods have clearer descent direction for the optimization problem, but require a relatively large number of time steps, suffering from high latency and usually more energy consumption with rate based representation. Another direction is to directly calculate gradients based on the SNN computation. They follow the BPTT framework, and deal with the non-differentiable
31
32 7 33 8 34 38
equivalent mappings 2 24
25
26
27].
problem of spiking functions by applying surrogate gradients [3 18 19 20 21 6 23
10 11
or computing gradients with respect to spiking times based the linear assumption [39] 40], or
combining both [22]. BPTT with SG can achieve extremely low latency. However, it requires large training memory to maintain the computational graph unfolded along time, and it remains unknown why surrogate gradients work well. [10] empirically observed that surrogate gradients have a high similarity with numerical gradients, but it remains unclear theoretically. And gradients based on spiking times suffer from the "dead neuron" problem [3] so they should be combined with SG in practice [40] [2] Meanwhile, methods in both directions are inconsistent with biological online learning, i.c. Forward-in-time learning, to pave a path for learning on neuromorphic hardware. Differently, our proposed method avoids the above problems and maintain the online property.
Online Training of Neural Networks. In the research of recurrent neural networks (RNNs), there several alternatives for BPTT to enable online learning. Particularly, real time recurrent learning (RTRL) [41] proposes to propagate partial derivatives of hidden states over parameters through time to enable forward-in-tee calculation of gradients. Several recent works improve the memory costs of RTRL with approximation for more practical usage 42
43 44 Another work proposes to online update parameters based decoupled gradients with regularization at each time step [45). However, these are all for RNNs and not tailored to SNNs. Several online training methods are proposed for SNNs [46 47 48), which are derived in the spirit of RTRL and simplified for SNNs. 49 leverages local losses and ignores temporal dependencies for online local training of SNNs, and 50 directly apply the method in 45 train SNNs. However, these methods also leverage surrogate gradients without providing theoretical explanation for optimization. Meanwhile, [46] 49
use feedback alignment (5I). (47 is limited to single-layer recurrent SNNs, and [8] requires much larger memory costs for eligibility traces, so they cannot scale to large-scale tasks. [50] requires a
specially designed neuron model and more computation for parameter regularization, and also does
not consider large tasks. Differently, our work explain the descent direction under both feedforward and recurrent conditions with convergent inputs, and is efficient and scalable to large-scale tasks including ImageNet classification.
3 Preliminaries
3.1 Spiking Neural Networks
Spiking neurons are brain-inspired models that transmit information by spike trains. Each neuron maintains membrane potential u and integrates input spike trains, which will generate spike once exceeds a threshold. We consider the commonly used leaky integrate and fire (LIF) model, which describes the dynamics of the membrane potential as:
du - urest) + R- I(),
Tm di
u <<< Ven,
(1)
where / is the input current, Vnn is the threshold, and R and Tm are resistance and time constant, respectively. A spike is generated when u reaches Ve, at time tf, and u is reset to the resting potential Ureat, which is usually set be zero. The output spike train is defined using the Dirac delta function: s(t) E., ö(t (t)).
A spiking neural network is composed of connected spiking neurons with connection coefficients. We consider a simple current model (t) = Σ, Wey Miiss(()) (t) + b, where the subscript i represents the i-th neuron, Way is the weight from neuron j to neuron i, and b, is a bias. The discrete computational form is:
[t 1] X(u. [t] Vt)+s(t) + Σ mysstt]+
(2)
(u, [t + 1] - Vin). s + (u,
where (xx) is the Heaviside step function, 8, [1] is the spike train of neuron i at discrete time step f, and X < 1 is leaky term (typically taken as 1 absorbed into the weights and bias. The reset operation is implemented by subtraction.
Fy. 2). The constant R, Tmi and time step size are
3

---

3.2 Previous SNN Training Methods
Spike Representation. The (weighted) firing rate or first spiking time of spiking neurons can be proved to follow ANN-like closed-form transformations 77 26 9 [2 27]. We focus on the weighted firing rate 12 27
which has connection with OTTT in this work. Define weighted firing rates and
weighted average inputs a(I] - A x[r] x[[
> x(r]
in the discrete condition. Given
Eizo
convergent weighted average inputs x[[] + xx*, it can be proved that a(t) - a² = VaA with
bounded random error, where a is a clamp function (o(x) = min(max(( æ)) 1)) in the discrete condition while a ReLU function in the continuous condition. For feedforward networks, the closed- form mapping between successive layers is established based on weighted firing rate after time T:
(W'
+ +b(]+) b(+))) and gradients are calculated with such spike representation:
Vin
at SW.
al N
3T' 0a IIT) For the recurrent condition, a[t] will converge to an [7] OW
Da
equilibrium state following an implicit fixed-point point equation, e.g. a* Vin (Wa" Fx + +b) for single-layer network with input connections F and contractive recurrent connections W. and gradients can be calculated based on implicit differentiation [1]. Let = fo (a) denote the fixed-point equation (f are parameters). We have As aL (T ((T))) JalT
0 (m(T)) where
J0
30
@fo(a (T) is the Jacobian of fe at a[T]. Ba[T]
J |a
BPTT with SG. BPTT unfolds the iterative undate equation in Eq 2 and backpropagates along the The gradients with time steps are calculated by
computational graph as shown in Fig. (a) 1(c)
AL
AL
Os Du
Ou Oul+1 [t]
Ou (u++ 0ul+1p]
Ou Su
as tu
Ou Ou'+"(r)
Σ II TET
Owi
1
Ow
Uw
where W is the weight from layer to I + and T. is the loss. The non-differentiable terms
ds'(4)
Zu'l
will be replaced by surrogate derivatives, e.g. derivatives of rectangular or sigmoid functions
4
1 sign (Iu Veal
(Van WV//a2 where aj and 02 are hyperparameters. Du (14elVas w//uzzz
Bu 01
Online Training Through Time for SNNs
This section contains four sub-sections. In Section 4.1 we introduce our proposed OTTT by decoupling the temporal dependency of BPTT. Then in Section 4.2 we further connect the gradients methods, and prove that OTTT
of OTTT and spike representation-base direction for optimization, which is not guaranteed by BPTT with SG. In Section 4.3] we discuss the relationship between OTTT and the three-factor Hebbian learning rule. Implementation details presented in Section 4.4
provide descent
4.1 Derivation of Online Training Through Time
Decouple temporal dependency. As shown Fig. [(c) BPTT has maintain the computational
graph of previous time to backpropagate through time. We will decouple such temporal dependency to enable online gradient calculation, as illustrated in Fig. [(d)
We first focus on the feedforward network condition. In this setting, all temporal dependencies lie in 0u' 0u'+1 and 0s! 0 8ul Tig
the dynamics of each spiking neuron, i.e.
in Eq. We consider
86 0
the case that we do not apply surrogate derivatives to Bu in such temporal dependency. Since the
Dut+i 0g
0s 0s!-1(
derivative of the Heaviside step function is 0 almost everywhere, we have
Du
Note that follow the numerator layout convention for derivatives, i.e. VoL with the same dimension
(0
the gradient
"Note that this is consistent with some released implementations of BPTT with SG methods which detach the neuron reset operation from the computational graph and do not backpropagate gradients in this path 23 MT.

---

W
W
W
W
Time
Time
Time
Time
(a) BPTT Forward
(b) OTTT Forward
BPTT Backward
(d) OTTT Backward
Figure 1: Illustration of the forward and backward procedures of BPTT and OTTT.
Then the dependency only includes Ou [+ 0
which equals AL. Therefore, we have
OL ow
T
ÖL
Os
[T] Vw L
T
gu [4]
Σ x ( T
Ost
1
owi
where gut+ [8]
(xl+i(c) Bulti (1) is the gradient for (t)]. Based on 04.0 we can track
AL 067+1
presynaptic activities al [L[ = Erst [T]] for each neuron during the forward procedure by
at 1] = Aá' [ s 1), so that when given gu'+: [t], gradients at each time step can be
calculated independently by V W [[(] gut [1] without backpropagation through 0u au+++pa [++1] Ou
As for the recurrent network condition, there are additional temporal dependencies due to the feedback
connections between neurons. If there is feedback connection from layer l2 to l (l2 2 41). there
(i+2(i+1] (s(2(;) au'2(() in the calculation of gradients (note that Eq. (3) omit
would be terms such as [x] [f] @u'a(i) 0u'i [6)
feedback connections for simplicity). We also consider not applying surrogate derivatives to Du in
the temporal dependency that gradients are not calculated in this path. Similar to the feedforward condition, we can derive that the gradients of the general weight W4, from any layer I to any layer
1j can be caleulated by Wt for optimization will be presented in Section
L[t] Eu's
each time step. A theoretical explanation a [t]
Instantaneous Loss and Gradient. Calculating online gradients, e.g. the above g [4] for Lf), requires instantaneous computation of the loss at each time step. Previous typical loss for
SNNs is based on the firing rate, e.g. L,r L (+ EL= s stll.y). where y the label, s [t] is the spike at the last layer, and L can take cross-entropy loss. This loss depends all time steps and does not support online gradients. We leverage the instantaneous loss and calculate the above gntt: [4] as:
1 - (s)))) T[t] =
[
I+1
[1] 0 [4] Os Ou [t]
gu/+i [t]
II
(5)
Os
Ns
The total loss L = E Ltl is the upper bound of Lyr when L is convex function such as cross-entropy. Then gradients can be calculated independently at each time step, as shown in Fig. [(d]
Os [t)
We apply surrogate derivatives for Bul
in this calculation, which will be explained Section 4.2
Since gradients are calculated instantaneously at each time step, OTTT does not require maintaining the unfolded computational graph and only requires constant training memory costs agnostic to time steps. Note that instantancous gradients of OTTT will be different from BPTT with the instantaneous loss for multi-layer or recurrent networks, we do not consider future influence in the instantaneous
BPTT considers terms such as aL('') Bu N [t') IT' Bu'+'( [t] (t' >> t) for u' [t] while
calculation:
au [ET J0
du'l
we do not. The equivalence of OTTT and BPTT only holds for the last layer, and we do not seek
the exact equivalence to BPTT with SG which is theoretically unclear, but will build the connection
with spike representations and prove the descent guarantee. Also, note that the tracked presynaptic
activities are similar to the biologically plausible "eligibility traces" in the literature 46 we will provide a more solid theoretical grounding for optimization in Section 4.2
47 52
and
5

---

4.2 Connection with Spike Representation-Bian Bassd Methods for Descent Directions
In this section, we connect gradients of OTTT and spike representation- -basd methods, and prove that OTTT can provide a descent direction for optimization under feedforward and recurrent conditions with convergent inputs.
Feedforward Networks. As introduced in Section 3.2 with convergent inputs, methods based on spike representations establish closed-form mappings between successive layers with weighted (W'a'() + b'+ and calculate gradients
firing rate a|t]
Eis [<T]((r as at+ [T] Rs
X
Via
by
aL WW
[T]
81
N-=
[+1 1 (a'
(27 @al+!(r]
OW
Note that a(t] is similar the tracked presynaptic
activities a[t] Di= [(T] in OTTT. We can obtain:
Öhyr
[T] Oa
(V w Lisr
II N=N-1
| al a'(r)',
(6)
7 TXT Os
11
da [T]
where Lar is the loss based on spike representation, d d' VeA (W'a'(T) + b(t)), and "O' is element-wise product. The detailed derivation can be found in Appendix A.
It can be easily seen that Eq. ( has similar form as gradients in Eq. (4), (), and we build the connection between them in three steps. For sake of clarify, in the following, we denote gradients of OTTT and spike representation by Vw and (V Tusr )rr)) respectively.
Ox 0x.+'()
In the first step, we can use appropriate surrogate derivatives of But+T( 1, T) approximate
[T+. As introduced in Section 3.2 o is a clamp function (a(x) = min (max(0) (0, x)) 1)) in the discrete condition while a ReLU function in the continuous condition. Then d' [T] almost equals sign (/W'a a [T] + b Varl < ViA)) if we slightly relax the clamp bound caused by the
[t) = sign (lu'+1 Vaal < Vin) at
discretization, and this can be approximated by diag (aul+10)
each time step. Note that surrogate derivatives here approximate the well-defined derivative of the mapping function between a(T", rather than the pseudo derivative of the non-differemtiable Heaviside step function as in BPTT with SG. The approximation neuron generate spikes (i.e. the average input is positive) while sometimes the membrane potential is less than trest (or the reverse). For simplicity in the following theoritical analysis, we assume the equivalence between surrogate derivatives and d[T).
exact except the rare case that averagely
Assumption 1. VI 1, N₁ 1 1,.=1,... T, diag
05 (1
= [T)
Du Pu
In the second step, we take Ler in Eq. (6) as Lorup
Σ Σ - so that
T x 1 in Eq. O aligns with 0g []]
in Eq. (5)
except a constant term. Note that Lgrup is (
an upper bound of the common loss Lg L(a (T),))
L(
x
if L is convex
function. Then let Buits [4] (BC(s ()y) Oz
II==N1 1+1
06-11(1
[t] 08' (6
with Assumption 1 we have:
W T D. Bu [t]a"(t] and (V Lar)
1
1
Σ Su [T]
Please
T TEE
refer to Appendix A for details.
In the third step, we handle the remaining difference that Vw: leverages instantancous presynaptic activities a|t] during calculation while Lar) uses the final ['']' after time T'. Note that the weighted firing rate gradually converges a[t] + a" with bounded random error. Suppose the errors e'[t] = a' [t] a' [T' are small (1 = 0 represents inputs, i.c. a a x(t), then we have that L
can provide a descent direction, as shown in Theorem [
Theorem 1. If Assumption
7 holds, Vvn = 1, and the errors e'(4) = a at [TT] are small such 111-A=-1 1- X² garts [t]
that | 11 [t]e]
EL B1 [t]a/ || ΣT=
T when
(V Lor) # O, then we have wV (V )rr) > O.
For the proof and discussion of the assumption please refer to Appendix A. With this conclusion, we can explain the descent direction of gradient descent by OTTT for the optimization problem

---

formulated by spike representation. Some random error can be viewed as randomness for stochastic optimization.
Recurrent Networks. For networks with feedback connections, we first consider the single-layer condition for simplicity (see Appendix A for general conditions). We consider feedforward con- nections F from inputs to neurons and contractive recurrent connections W between neurons. As introduced in Section 3.2] given convergent inputs x[t] + x", a[t] of neurons will converge to Vih (Wa' + Fxxx+ with bounded random error, an equilibrium state a' fo(a") o ViL
T DLam (1- (o
ff(aa) T
and gradients are calculated as (Voler)ar
30
where 0 E F.. We consider replacing the inverse Jacobian by an identity matrix: (VoLor) pr
ah. af Previous works have proved that this gradient can provide [a([]
00
( ((ooLer) (Volsrr)r) > O.
a descent direction for the optimization problem 35
36 i.e.
It has a similar form as the OTTT gradient: Vw = Σ 8a [(a[t]] and (V
El ( 1 aL. e d(r] a[r]T. Similarly, we can prove the descent guarantee for OTTT as shown in Theorem
D For details refer to Appendix A.
Theorem 2. If Assumption i holds, VnA 1, | Sn <
Lin, where max and min are the arm
maximal and minimal singular value of | Bult)e
Bfg lafry- and the errors e a(t) a[T],
x(t]
30
x[[] are small such that
no
T E
oc(sttly) (1 J la(rr)
Jear
E X01-X--1
1-X" gu [t)aa(t] (where 0, a'( and [t represent alt] and x(t), respectively)
when (Volsr) A O, then we have (VoL, (Vr > 0, where O are parameters in the network.
4.3 Connection with Three-factor Hebbian Learning Rule
By explicitly writing the instantaneous gradients of OTTT for the general weight from layer 4 to Ij. V Wi L[[] = Bus [1]a"[0 [1] and dive into connections between any two neurons i and j. we have:
Vw.., Ltt] (u,])((
(7)
where a, is the tracked presynaptic activity, [t) is the surrogate derivative function which can represent the change rate of the postsynaptic activity analyzed in Section 4.2 and 8y[t] 9s, [t] is the gradient for neuron output 8, [f] which represents a global modulator. This is kind of three-factor Hebbian learning rule (37] and the weight can be updated locally with a global signal. The error signal 6,[0] can be propagated in an error feedback path simultaneously with feedforward propagation, which is shown biologically plausible with high-frequency bursts (53). Note that the analysis in Section 4.2 still holds if we consider the delay of the propagation of the error signal, i.c. the update is based on 0, t + (t) 66))6,
4.4 Implementation Details
As introduced in Section 4.1 will calculate instantancous gradients V W L(t)= at each time step. We can choose to immediately update parameters before the calculation of the next accumulate the gradients by T time steps and then time step, which we denote OTTTo, OTTT~- A. For OTTTo, we assume that the online update is update parameters, which we denote as small and has negligible affects for the following calculation. Pseudo-codes are in Appendix B.
Bu's [ta' [t]
An important issue in practice is that previous BPTT with SG works leverage batch normalization (BN) along the temporal dimension to achieve high performance with extremely low latency on large-scale datasets [6] 10 [1 13, which requires calculating the mean and variance statisties for all time steps during the forward procedure. This technique intrinsically prevents online gradients and has to suffer from large memory costs. To overcome this shorteoming, we do not use BN, but borrow the idea from normalization free ResNets (NF-ResNets) [54 555 to replace batch normalization by
W.,, PW.. scaled weight standardization (sWS) (56). sWS standardizes weights by W., 7 ow, /N and
the scale is determined by analyzing the signal propagation with different activation functions. We apply sWS for VGG [57] and Re-ResNet architectures in our experiments. For details please refer to Appendix C.
7

---

Experiments
In this section, conduct extensive experiments on CITAR-IO [58, CIFARI00 588 ImageNet 59 CIFARIO-DVS [60], and AVS128-Gesture [6]] to demonstrate the superior performance of our proposed method on large-scale static and neuromorphic datasets. We leverage the VGG network architecture
122-51122--123--AAPPO0 for experiments on CIFAR-10, CIFAR- 100, CIFARIO-DVS, and DVS128 128128-Ge and the NF-ResNet- 34 [54] network architecture for experiments on ImageNet. For all our SNN models, we set VAA 1 and X 0.5. Please refer to Appendix C for training details.
5.1 Comparison of Training Memory Costs
40
OTTT
-PTB
GPU Training Memory
0
5
10
15
20
25
30
Time Steps
Figure 2: Comparison of training memory costs between OTTT and BPTT under different time steps.
A major advantage of OTTT over BPTT is that OTTT does not require backpropagation along the temporal dimension and therefore only requires constant training memory costs agnostic time steps, which avoids the large memory costs of BPTT. We verify this by training the VGG network on CIFAR-10 10 with batch size 128 under different time steps and calculating the memory costs on the GPU. As shown in Fig. 2 the training memory of BPTT grows linearly with time steps, while OTTT maintains the constant memory (both OTTT and OTTTo). Even with a small number of time steps, e.g. 6, OTTT can reduce the memory costs by 2 ~ 3x. This advantage may also allow training acceleration of SNNs by larger batch sizes with the same computational resources.
5.2 Comparison of Performance
We conduct experiments on both scale static and neuromorphic datasets. We first verify the 0g'
effectiveness of the surrogate derivative au e sign (lul+(tt - Vaal < Von) in Section 4.2 For the VGG (sWS) network on CIFAR-10, OTTT and OTTTo achieves 91.46% and 92.47% test accuracy respectively. We empirically observe that applying the sigmoid-like surrogate derivative achieves a higher generalization performance, e.g. OTTT and OTTT, achieves 93.58% and 93.73% test accuracy respectively under the same random seed, and a possible reason may be that this introduces some noise for the approximation to regularize the training and improve the generalization. Therefore, in the following performance evaluation, we take sigmoid- like surrogate derivative for OTTT. As shown in Table I both OTTT and OTTTo achieve satisfactory performance on all datasets, and compared with BPTT under the same training settings, OTTT achieves higher performance. The proposed OTTT also achieves promising performance on all datasets compared with other representative conversion and direct training methods. Besides, it shows that the performance gap between our SNN model and ANN is around 0.7% and 2.08% on CIFAR-10 10 and CIFAR-100, respectively, Usually, SNNs with a very small number of time steps do not reach the performance of equivalent ANNs due to the information propagation with discrete spikes rather than floating-pon ppint numbers. The results of our model with 6 time steps are competitive.
We also evaluate our method on the DVS128- different hand gestures recorded by a DVS camera. As shown Table our method can achieve the same high performance as BPTT. While our theoretical analysis mainly focus on convergent inputs (e.g. static images or neuromorphic inputs converted from images like CIFAR 10-DVS,, the results show that method can also work well for time- sare-varying inputs.
dataset, which is more time- varying with
8

---

Table Performance on CIFAR- 10, CIFAR-100, 100, ImageNet, and CIFARIO-DVS, Results are based
on 3 runs of experiments (except ImageNet). Our OTTT is mainly compared with BPTT under the same settings, and also compared with other representative conversion and direct training methods.
Dataset
Method
Network structure
Params Time steps
Mean (Best)
ANN-SNN
7
VGG-16
40M
16
(92.29%%
BPTT 6
ResNet-19 (tdBN)
14.5M
6
(93.16%)
BPTT 25
9-layer CNN (PLIF, BN)
36M
8
(93,50%)
CIFAR- 10
BPTT
VGG (sWS)
9.2M
6
92.78=0.3450 (93.23%
OTTT A (ours)
VGG (sWS)
9.2M
6
03.52=00.669 % (93.58%)
OTTTo (ours)
VGG (sWS)
9.2M
6
93.49=0.17%%
(93.73%)
ANN
VGG (sWS)
9.2M
N.A.
(94.43%)
ANN-SNN
VGG-16
40M
400-600
(70.55%)
Hybrid Training
VGG-11
36M
125
(67.87%)
DIET-SNN 62
VGG-16
40M
5
(69.67%)
CIFAR-100
BPTT
VGG (sWS)
9.3M
6
69.06=00007 (69.15%)
OTTT A (ours)
VGG (sWS)
9.3M
6
71.05=0.044% (71.11% )
OTTTo (ours)
VGG (sWS)
9.3M
6
71.05=0.0664 (71.11 %)
ANN
VGG (sWS)
9.3M
N.A.
(73.19%)
ANN-SNN
ResNet-34
22M
32
(64.54%)
Hybrid Training
31
ResNet-34
22M
250
(61.48%)
ImageNet
BPTT c
ResNet-34 (tdBN)
22M
6
(63.72%)
OTTTA (ours)
NF-ResNeet-3
22M
6
(65.15%)
OTTT (ours)
NF-ResNect33
22M
6
(64.16%)
Tandem Learning
CifarNet
45M
20
(65.59%)
BPTT 6
ResNet-19 (tdBN)
14.5M
10
(0%)
BPTT 23
7-layer CNN (PLIF, BN)
1.1M
20
(74.80%)
DVS-CTFAR10
BPTT
VGG (sWS)
9.2M
10
72.60
1. 1.26% (73.90%)
OTTT, A (ours)
VGG (sWS)
9.2M
10
76.27 - 0.05 % (76.30%)
OTTTo (ours)
VGG (sWS)
9.2M
10
76.63+
(77.10%)
Table 2: Performance on
Method
Network structure
Time steps
Accuracy
SLAYER DECOLLE
8-layer CNN 3-layer CNN
93.64110.499
1800
95.54-1:0. 166%
BPTT 23
8-layer CNN (PLIF, BN)
20
97.57%
BPTT 23
8 8-yayer CNN (LIF, BN)
20
96.88%
BPTT
VGG (sWS)
96.88%
OTTTA (ours)
VGG (sWS)
20
96.88%
5.3 Effectiveness for Recurrence
As introduced in Section 4 the proposed OTTT is also valid for networks with feedback connections. Previous works have shown that adding feedback connections can improve the performance of SNNs without much additional costs, especially on the CIFAR-100 datasets [2,63] 63). Therefore, we conduct experiments on CIFAR-100 with the VGG-F network architecture which simply adds a feedback connection from the last feature layer to the first feature layer following (12). and this weight is itro-intialized. As shown in Table 3 the training of VGG-F is valid and VGG-F achieves higher performance than VGG due to the introduction feedback connections. Results in Appendix D show that the improvement of OTTT from feedback connections is more significant than that of BPTT. The architectures with feedback connections can be further improved with neural architecture search 63
5.4 Effectiveness for Training with Batch Size
To further study the online training, i.e. not only online in time but also one sample per training, which is consistent with biological learning and learning on neuromorphic hardware, we verify the effectiveness for training with batch size 1. The VGG network on CIFAR- 10 is studied, and batch size is compared with the default batch size 128 under the same random seed. Models are only trained for 20 epochs due to the relatively long training time with batch size 1. As shown in Table 4 training with one sample per iteration still valid, indicating the potential to conduct full online
training with the proposed ОTTТ.

---

Table 3: Performance CIFAR-100 for VGG and VGG-F trained by OTTTo. Results are based on runs of experiments.
Table 4: Performance of VGG CIFAR-10 10 with different batch sizes for 20 epochs under the same random seed.
Network structure Params
Std (Best)
Method
Batch Size
Accuracy
VGG VGG-F
9.3M 71.05.10.06% (71.11%)
OTTT / /TTTTo OTTT /OTTTo
128
88.20% / 88.62%
9.6M
72.63.10.23% (72.94%)
88.07% / 88.50%
5.5
Influence of Inference Time Steps
We study the influence of inference time steps on ImageNet as shown Fig. It illustrates that the model trained with time step 6 can achieve higher performance with more inference time steps.
5.6 Firing Rate Statistics
We study the firing rate statistics of the models trained by OTTT and BPTT, as shown in Fig. demonstrates that models trained by OTTT have higher firing rates in first layers while lower firing rates later layers compared with BPTT. Overall the firing rate around 0.19 and with 6 time steps each neuron averagely generate 1.1 spikes, indicating the low energy consumption. Considering that each neuron has more synaptic operations in later layers than first layers (because the channel size increasing with layers), the synaptic operations of models trained by OTTT and BPTT are about the same (1.98 x 10 vs 1.93 10*). More results please refer to Appendix D.
Figure 3: Influence of inference time steps for the model trained with 6 time steps on ImageNet.
Figure 4: The average firing rates for the models trained by OTTT and BPTT on CIFAR-I0.
66
65,15
65.35
65.4
0.35
64.52
(O)
0.3
Average Firing Rate
0.25
62.63
Accuracy
62
58,02
0.05
58
10
12
Layer Layar Zaaer 3aayer 4Layer 5Layer 6Layer 7Layer 8
Inference Time Steps
Layer Index
6 Conclusion
In this work, we propose neural networks. We first derive OTTT from BPTT with SG by decoupling the temporal dependency with the tracked pre-synaptic activities, which only requires constant training memory agnostic to time steps and avoids the large training memory costs of BPTT. Then we theoretically analyze and connect the gradients of OTTT and gradients of methods based spike representations, and prove the descent guarantee OTTT for the optimization problem under both feedforward and recurrent network conditions. Additionally, rule, which is the first to connect BPTT with SG, spike representation- based methods, and biological learning rules. Extensive experiments demonstrate the superior performance of our methods on large-scale static and neuromorphic datasets in a small number of time steps.
training method, online training through time (OTTT), for spiking
show that OTTT is in the form of three-factor Hebbian learning
Acknowledgement
Lin was supported by the major key project of PCI. (No. PCI.2021A12) the NSF China (No.s 62276004 and 6173 1018), and Project 2020BD006 supported by PKU-Baidu Fund.
10

---

References
[1] Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Neural Networks, 10(9):165--1671 1997.
[2] Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer. Training deep spiking neural networks using backpropagation. Frontiers in Neuroscience, 10:508, 2016.
[3] Sumit Bam Shrestha and Garrick Orchard. Slayer: spike layer error reassignment in time. In Advances in Neural Information Processing Systems, 2018.
[4] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Fromtiers in Neuroscience, 12:331, 2018.
[5] Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575(7784):607-617, 2019.
[6] Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with directly-trained larger spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
[7] Shikuang Deng and Shi Gu. Optimal conversion conventional artificial neural networks to spiking neural networks. International Conference on Learning Representations, 2021.
[8] Yuhang Li, Shikuang Deng, Xin Dong, Ruihao Gong, and Shi Gu. A free lunch from ann: Towards efficient, accurate spiking neural networks calibration. International Conference on Machine Learning, 2021.
[9] Jibin Wu, Yansong Chua, Malu Zhang, Guoqi Li, Haizhou Li, and Kay Chen Tan. A tandem learning rule for effective training and rapid inference of deep spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2021.
[10] Yuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng, Yongqing Hai, and Shi Gu. Differentiable spike: Rethinking gradient-descent for training spiking neural networks. In Advances in Neural Information Processing Systems, 2021.
[11] Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. In Advances in Neural Information Processing Systems, 2021.
[12] Mingqing Xiao, Qingyan Meng. Zongpeng Zhang, Yisen Wang, and Zhouchen Lin. Training feedback spiking neural networks by implicit differentiation on the equilibrium state. In Advances in Neural Information Processing Systems. 2021.
[13] Shikuang Deng, Yuhang Li. Shanghang Zhang, and Shi Gu. Temporal efficient training of spiking neural network via gradient re-weighting. In International Conference on Learning Representations, 2022.
[14] Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-lcaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al. TrueNorth: Design and tool flow of a 65 mw million neuron programmable neurosynaptic chip. IEEE Transactions on Computer-Aidel Design of Integrated Circuits and Systems, 34(10):1537 1557, 2015.
[15] Mike Davies, Narayan Srinivasa, Tsung- Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning, IEEE Micro, 38(1):82-99 2018
[16] Jing Pei. Lei Deng, Sen Song, Mingguo Zhao, Youhui Zhang, Shuang Wu, Guanrui Wang. Zhe Zou, Zhenzhi Wu, Wei He, al. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature, 572(7767::00--111 2019.
[17] Paul J Werbos. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78(10): 1550-150,, 1990.
[18] Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass, Long short- term memory and lcamiig-to- Ieal in networks of spiking neurons. In Advonces in Neural Information Processing Systems, 2018
[19] Yingyezhe Jin. Wenrui Zhang, and Peng Li. Hybrid macro/micro level backpropagation for training deep spiking neural networks. In Advances Neural Information Processing Systems, 2018.
[20] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI Conference Artificial Intelligence, 2019.
11

---

[21] Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient- optimization spiking neural networks. IEEE Signal Processing Magazine, 36(6):11-63 2019.
[22] Jinseok Kim, Kyungsu Kim, and Jae-Joon Kim. Unifying activation and amming-based learning rules for spiking neural networks. In Advances in Neural Information Processing Systems, 2020.
[23] Wei Fang, Zhaofei Yu, Yanqi Chen. Timothée Masquelier, Tiejun Huang, and Yonghong Tian. Incorporating learnable membrane time constant to enhance learning of spiking neural networks. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), 2021.
[24] Johannes C Thiee,, Olivier Bichler, and Antoine Dupret. Spikegrad: An aqu-equivllent computation model for implementing backpropagation with spikes. In International Conference on Learning Representations, 2020.
[25] Hao Wu, Yueyi Zhang, Wenming Weng, Yongting Zhang, Zhiwei Xiong, Zheng-Jun Zha, Xiaoyan Sun, and Feng Wu. Training spiking neural networks with accumulated spiking flow. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
[26] Shibo Zhou, Xiaohua Li, Ying Chen, Sanjecv Chandrasckaran, and Arindam Sanyal. Temporal-codee deep spiking neural network with casy training and robust performance. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
[27] Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang. Zhouchen Lin, and Zhi- Zhi-Quan Luo. Training high-performance low-latency spiking neural networks by differentiation on spike representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2022.
[28] Erie Hunsberger and Chris Eliasmith. Spiking deep networks with LIF neurons. arXiv preprint orXiv:1510. 08829, 2015.
[29] Bodo Rueckauer, Allia-Alexandr Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in Neuroscience, 11:682, 2017.
[30] Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in Neuroscience, 13:95, 2019.
[31] Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy. Enabling deep spiking neural networks with hyhrid conversion and spike timing dependent hackpropagation. In International Conference on Learning Representations. 2020.
[32] Bing Han and Kaushik Roy. Deep spiking neural network: Energy efficiency through time based coding. In European Conference on Computer Vision, 2020.
[33] Zhanglu Yan, Jun Zhou, and Weng-Fai Wong. Near lossless transfer learning for spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
[34] Christoph Stöckl and Wolfgang Maass. Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes. Nature Machine Intelligence, 3(3):230-238, 2021.
[35] Samy Wu Fung, Howard Heaton, Qiuwei Li. Daniel McKenzie, Stanley Osher, and Wotao Yin. Jfb: Jacobian-free backpropagation for implicit networks. Proceedings of the AAAI Conference Artificial Intelligence, 2022.
[36] Zhengyang Geng, Xin-Yo Yu Zhang, Shaojie Bai, Yisen Wang. and Zhouchen Lin. On training implicit models. In Advances in Neural Information Processing Systems, 2021.
[37] Nicolas Frémaux and Wulfram Gerstner. Neuromodulated spike-timing timing- plasticity, and theory of three- -hrte-factor lcarning rules. Frontiers in Neural Circuits, 9:85, 2016
[38] Qingyan Meng, Shen Yan, Mingqing Xiao, Yisen Wang, Zhouchen Lin, and Zhi-Quan Luo. Training much deeper spiking neural networks with a small number of time-steps. Newral Networks, 153:254-268, 2022.
[39] Sander M Bohte, Joost N Kok, and Han La Poutre. Errorbbackropagationn in temporally encoded networks of spiking neurons. Neurocomputing. 48(1 11)-11)77-37,, 2002.
[40] Wenrui Zhang and Peng Li. Temporal spike sequence learning via backpropagation for deep spiking neural networks. In Advances in Neural Information Processing Systems, 2020.
12

---

[41] Ronald Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1(2):700-280, 1989.
[42] Corentin Tallec and Yann Ollivier. Unbiased online recurrent optimization. In International Conference on Learning Representations, 2018
[43] Asier Mujika, Florian Meier, and Angelika Steger. Approximating real-time recurrent learning with random kronecker factors. In Advances in Neural Information Processing Systems, 2018.
[44] Jacob Menick, Erich Flsen, Utku Evei, Simon Osindero, Karen Simonyan, and Alex Graves. Practical real time recurrent learning with sparse approximation. In International Conference on Learning Representations, 2021.
[45] Anil Kag and Venkatesh Saligrama. Training recurrent neural networks via forward propagation through time. In International Conference on Machine Learning, 2021.
[46] Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural Computation, 15145151 2018.
[47] Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Legenstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 11(1):1-15, 2020.
[48] Thomas Bohnstingl, Stanislaw Wožniak, Angeliki Pantazi, and Evangelos Eleftheriou. Online spatio- temporal learning in deep neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[49] Jacques Kaiser, Hesham Mostafa, and Emre Neftei. Synaptic plasticity dynamics for deep continuous local learning (decolle). Frontiers in Neuroxcience, 14:424, 2020.
[50] Bojian Yin, Federico Corradi, and Sander M Bobte. Accurate online training of dynamical spiking neural networks through forward propagation through time. arXiv preprint arXiv:2/ 2021.
[51] Arild Nøkland. Direct feedback alignment provides learning in deep neural networks. In Advances in Neural Information Processing Systems, 2016.
[52] James M Murray. Local online learning in recurrent networks with random feedback. Elife, 8:c43299, 2019.
[53] Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake A Richards, and Richard Naud. Burst- dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature Neuroscience, 24(7):1010-1019, 2021.
[54] Andrew Brock, Soham De, and Samuel L Smith. Characterizing signal propagation to close the performance gap in unnormalized resnets. International Conference on Learning Representations, 2021.
[55] Andy Brock, Soham De. Samuel L Smith, and Karen Simonyan. Highh-performance large-scale image recognition without normalization. International Conference on Machine Learning, 2021.
[56] Siyuan Qiao, Huiyu Wang, Chenxi Liu, Wei Shen, and Alan Yuille. Weight standardization. arXiv preprint arXiv: 10520. 2019.
[57] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. International Conference on Learning Representations, 2015.
[58] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
[59] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2009.
[60] Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifarl0-dvs: an event-stream dataset for object classification. Frontiers in Neuroxcience, 1 11:309, 2017.
[61] Arnon Amir, Brian Taba, David Berg. Timothy Melano, Jeffrey McKinstry, Carmelo Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. A low power, fully event- based gesture recognition system. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, 2017.
13

---

[62] Nitin Rathi and Kaushik Roy. Diet-smn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization. IEEE Transactions Neural Networks and Learning Systems, 2021.
[63] Youngeun Kim, Yuhang Li, Hyoungseob Park, Yeshwanth Venkatesha, and Priyadarshini Panda. Neural architecture search for spiking neural networks. European Conference on Computer Vision, 2022.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Sce Appendix E.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Appendix E.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
If you are including thcoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Scction 4
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix A.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or URL)? [Yes] See Supplementary Materials.
(b) Did specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix C.
(c) Did you report error bars (e.g.. with respect to the random seed after running experiments multiple times)? [Yes] See Scction
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix
4. If you using existing assets (e.g.. code, data, models) curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] See Section
(b) Did you mention the license of the assets? [Yes] See Appendix C.
(c) Did you include any assets either in the supplemental material URL? [N/A] We do use assets.
(d) Did you discuss whether and how consent obtained from people whose data you's 're us- ing/curating? [No] We use the existing common datasets.
(e) Did you discuss whether the data you using/curating contains personally identifiable informa- tion or offensive content? [No] We use the existing common datasets.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "evzCJ1zUn5W", "reviewer": "Reviewer_AZJQ", "summary": "The paper presents online training through time (OTTT) for SNNs, which is derived from BPTT to enable learning without backward passes through time. The goal compared to other approaches is to improve memory consumption, biological plausibility, or latency. Theoretical analysis shows that the gradients of OTTT are in a direction that is suitable for optimization. Experiments show that, compared to BPTT, OTTT has an advantage in terms of memory consumption, especially as training sequence lengths increase.", "questions": "Could the authors explain what are their expectations from the comparisons and analyses that are missing, if they were performed?\nCould these actually be peformed?", "limitations": "There is no discussion of limitations by the authors.", "rating": 5, "confidence": 3, "soundness": 1, "presentation": 1, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper addresses a topic that has attracted a lot of interest by the community. It also includes significant theoretical analysis and has given effort in explaining the derivations.\n\nOn the other hand, the paper:\n- does not even mention some recent methods (FPTT, OSTL) [1, 2] that have similar goals, let alone explain the differences. Therefore the extent of the advance in this paper cannot be evaluated. \n- does not provide experimental comparisons to such methods as [1-3], but only to BPTT\n- if any advantage - theoretical or other - exists compared to these prior methods, the paper does not summarise clearly what element in the new theoretical derivation enables that advantage.\n- the work does not aim to achieve something brand new, as successful SNN training has already been possible. Therefore its impact is expected to be limited.\n- does not show results on sequential tasks, but only on image classification, even though the learning rule concerns recurrent networks\n- does not compare experimentally with non-spiking networks, in terms of accuracy, memory, or computational efficiency.\n- could improve its writing, by removing redundant repetitions, providing a sketch/summary of the derivation before the derivation, and by clarifying terms such as forward-in-time\n\n[1] Yin, Bojian, Federico Corradi, and Sander M. Bohte. \"Accurate online training of dynamical spiking neural networks through Forward Propagation Through Time.\" arXiv preprint arXiv:2112.11231 (2021).\n[2] Bohnstingl, Thomas, et al. \"Online spatio-temporal learning in deep neural networks.\" IEEE Transactions on Neural Networks and Learning Systems (2022).\n[3] Bellec, Guillaume, et al. \"A solution to the learning dilemma for recurrent networks of spiking neurons.\" Nature communications 11.1 (2020): 1-15.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "WgB_bPmVz-a", "reviewer": "Reviewer_bMnW", "summary": "In this paper, the authors propose an online learning algorithm (OTTT) that's applicable to spiking neural networks (SNNs). OTTT uses a combination of eligibility traces (trace of past activity of the neuron) and use of instantaneous loss values to achieve an online algorithm. The authors show its connection to spike representation based methods as well as three-factor learning rule and demonstrate empirically that this method works better than existing methods for training feedforward spiking neural networks.\n", "questions": "## Questions\n\n* Does $s[\\tau]$ in line 141 refer to the output spikes of each layer?\n* Can OTTT also be applied to non-spiking neural network architectures?\n\n## Suggestions\n\n* In the abstract:\n    * l.5: it's not clear why BPTT with SG leads to extremely low latency. Rephrase?\n    * l.6: theoretical unclarity -> lack of theoretical clarity.\n    * l.11: what does it mean for online learning to be a learning rule? Rephrase?\n* l.32: The supervised training of SNNs is challenging not because of its \"complex neuron model\" but because of non-differentiability in the neuron model\n* Sec 2.  I would suggest giving brief descriptions of individual references rather than grouping large numbers of references together with vague sentences, which doesn't convey much useful information about related work.\n* Related work that uses eligibility traces should be cited: e.g. (Murray 2019; Bellec et al. 2020)\n* l.142: A few details of how this can be proved would be useful.\n* Table 1: It would be helpful to see comparison with equivalent non-spiking architecture trained with BPTT as an (upper) baseline to put the performance of OTTT in perspective.\n\nBellec, G., Scherr, F., Subramoney, A., Hajek, E., Salaj, D., Legenstein, R., Maass, W., 2020. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications 11, 3625.\n\nMurray, J.M., 2019. Local online learning in recurrent networks with random feedback. eLife 8, e43299.", "limitations": "The authors do not discuss the limitations of the work.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\nThe paper derives an novel online learning rule, primarily applied to feed-forward SNNs. The connection to spike representation derived in the paper is very interesting, and connects these seemingly disparate methods. \n\nThe empirical results are impressive, and strongly support the utility of OTTT. The fact that OTTT works with batch size of 1 is particularly impressive, and makes it of strong practical interest for neuromorphic hardware. \n\nThe paper is also well written and the exposition is clear and easy to understand.\n\n## Weaknesses\n\nThe results of OTTT with feedback connections and with smaller batch sizes needs more baselines, esp. compared to BPTT to understand how it stands in comparison to conventional methods. i.e. to understand if the advantage there because of OTTT or other reasons?\n\nIt would also have been interesting to see results with fully recurrent architectures.\n\nMinor: The connection with spike representation methods is certainly interesting, but might be a bit over-emphasized and could be confined to a single section so that it doesn't interrupt the flow of OTTT. Specifically, the Spike Representation heading in Sec. 3.2 threw me off a bit, since I was looking for a connection with Sec. 4.1.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "JejHTho56Vh", "reviewer": "Reviewer_Mcuq", "summary": "The authors introduce a new approach to training spiking neural networks, online training through time (OTTT). The approach only requires constant memory since it does not need to backpropagate through time. Instead, tracks presynaptic activities and leverages instantaneous loss and gradients. The proposed approach provides good results on typical benchmarks such as CIFAR-10 and 100.\n\n", "questions": "1. How does the authors' work relate to other approaches that implement forward propagation through time?\n\n2.  What are the effects of the assumptions on the proof? For instance, in line 167 the reset moment is ignored. Line 147-150: Does the equilibrium condition depend on constant inputs? And why can only equilbria happen and not limit cycles? \n\n3. The \"instantaneous\" loss is defined as a sum over time (lie 184). How is that instantaneous and the definition higher up (line 182) not? They both sum over time, although 1/T is inside the loss function higher up.\n\n4. Intuitively, what is remembered in the trace that allows to bypass remembering all activations over time? \n\n5. What are the gradients of the \"spike representation\"? Are these the gradients from BPTT?\n\n6. What is meant with \"time steps\" in Fig 2? The time steps BPTT goes back in time? How does this apply to OTTT?\n", "limitations": "N/A", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* How to best train SNNs is still an open problem, especially if one aims for online learning on neuromorphic hardware. This article can form a substantial step in that direction.\n* If I understand the mathematical proof well, the authors show that their method's gradient points in the same direction as the BPTT one, without needing to track many variables in memory.\n* The results of the approach seem good.\n\nWeaknesses:\n* The work currently does not discuss other SNN  approaches that also do not backpropagate gradients through time, such as: Accurate online training of dynamical spiking neural networks through Forward Propagation Through Time, B Yin, F Corradi, SM Bohte - arXiv preprint arXiv:2112.11231, 2021https://arxiv.org/pdf/2112.11231.pdf\n* Some of the main intuitions behind not needing to backpropagate through time remain unclear (see questions).\n*  The tasks to which the approach is applied are quite \"static\", and do not immediately seem to need much memory. Moreover, the approach seems to rely on neurons moving to an equilibrium state, which would be less valid for real-world, time-varying data.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept