Spike Accumulation Forwarding for Effective Training of Spiking Neural Networks
Authors: Ryuji Saiin
Tomoya Shirakawa
Sota Yoshihara
Yoshihide Sawada
Hiroyuki Kusumoto
Venue: Accepted by TMLR ()

Abstract: In this article, we propose a new paradigm for training spiking neural networks (SNNs), spike accumulation forwarding (SAF). It is known that SNNs are energy-efficient but difficult to train. Consequently, many researchers have proposed various methods to solve this problem, among which online training through time (OTTT) is a method that allows inferring at each time step while suppressing the memory cost. However, to compute efficiently on GPUs, OTTT requires operations with spike trains and weighted summation of spike trains during forwarding. In addition, OTTT has shown a relationship with the Spike Representation, an alternative training method, though theoretical agreement with Spike Representation has yet to be proven. Our proposed method can solve these problems; namely, SAF can halve the number of operations during the forward process, and it can be theoretically proven that SAF is consistent with the Spike Representation and OTTT, respectively. Furthermore, we confirmed the above contents through experiments and showed that it is possible to reduce memory and training time while maintaining accuracy.

Published in Transactions on Machinc Learning Rescarch (06/2024)
Spike Accumulation Forwarding for Effective Training of Spiking Neural Networks
Ryuji Saiin*
ryuj..sciinnaaisin-ssffuuuu
Tokyo Research Center, AISIN, Tokyo, Jepan
AISIN SOFTWARE, Aichi, Japan
Tomoya Shirakawa"
Sinezdayo@ggmai...m
Graduate School of Mathematics, Nagoya University, Aichi, Japan
Sota Yoshihara
sota. yoshihaaa www.yysshhta.eemmaaate. www.:e/eeemm...ooooo....oom uu.c.j.........p......
Graduale School of Mathematics, Nagoya Universily, Aichi, Japan
Yoshihide Sawada
Tokyo Research Center, AISIN, Tokyo, Japan
Hiroyuki Kusumoto
kusumoto-1088Bouloook.oooo
Craduate School of Mathemalicss, Nagoya Universily, Aichi, Japan
Reviewed OpenReview: https: // //tpr/vvee.......... net/ forum? id= RGQsUQDAd9
Abstract
this article, we propose new paradigm for training spiking neural networks (SNNs), spike accumulation forwarding (SAF). It known that SNNs are energy-efficient but difficult train. Consoquently, many rescarchers have proposed various methods to solve this problem, among which online training through time (OTTT) is method that allows inferring at each time step while suppressing the memory cost. However, to compute efficiently on GPUs, OTTT requires operations with spike trains and weighted summation of spike trains during forwarding. In addition, OTTT has shown a relationship with the Spike Representation, an alternative training method, though theoretical agreement with Spike Representation has yet to be proven. Our proposed method can solve these problems; namely, SAF can halve the number of operations during the forward process, and it can be theoretically proven that SAF is consistent with the Spike Representation and OTTT, respectively. Furthermore, we confirmed the above contents through experiments and showed that it is possible to reduce memory and training time while maintaining accuracy.
Introduction
Due the carbon emission reduction problem, onergy-officient spiking neural networks (SNNs) are attract- ing atlention (Luo el. al., 2023). SNNs are known be more bio-plausible models than artificial neural networks (ANNs) and can replace the multiply-accumulae (MAC) operations with additive operations. This characteristic comes from propagating the spike train (belonging to {0. where T is the number of time steps) and is energy-efficiet on neuromorphic chips (Akopyan et al. 2015 Davies el al. 2018).
Despite the usefulness SNNs for CO2 reduction, their neurons are non-differentiable,, which makes them difficult train. Solving this problem in the mainstream of SNN rescarch, and propagatonn through
"Equal contribution. t CCoresponding author.

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
a
1 a
[ 1]
a
SUM
SUM
HQ}
HQ
HO.)
HG.)
[t
[c]
U [
Switch
Switch
W
W
W
W
s'(t 1l[t-1]
s'(t]
a'[c]
a'[ - 1]
OTTT
SAF
Figure 1: Overview of OTTT and SAF training. OTTT requires operations with spike s [t] and spike accu- mulation a'[t] during the forward process, while SAF requires operations with @' Also, u u'(t] represents the membrane potential, and [++[[] represents the potential accumulation (see Sec. 4 The SUM layer computes the spike accumulation, and the Switch layer propagates W's and W' in forwarding and back- warding, respectively. Note that, unlike membrane potentials, the potential accumulation does not require the retention of past information.
time (BPTT) with surrogate gradient (SG) (Zhnng et al.) 2021 Xiao et al. 2022 is one of the main methods to achieve high performance. particular, the recently proposed Online Training Through Time (OTTT) can train SNNs at each time step, just like our brains, and achieve high performance with few time steps Xiao eet al. 2022).
To enable online training, OTTT uses different information for the forward and backward processes. For forwarding, the spike train is used; for backwarding, the weighted summation of spike trains (which we refer to as spike accumulation) is used. Therefore, efficient computation on GPUs using the Autograd of libraries such as PyTorch (Paszke et al.] 2019) requires operations with spike train and spike accumulation during the forward process (see Fig. [p. Mcanwhile, OTTT has the theoretical guarantee that the gradient descent direction is similar to that of Spike Representations by the weighted firing rate coding by summing up the gradients of each time step (Xiao et al. 2021 Meng et al. 2022 However, these gradients are not shown to be perfectly consistent. To accurately bridge them, it is essential to develop a method that guarantees the gradient can be consistent with each of above two gradients.
In this article, we propose Spike Accumulation Forwarding (SAF) as a new paradigm for training SNNs. Unlike OTTT, SAF propagates not only backward but also forward processes by spike accumulation (see Fig. D) By using this process, we can halve the number of operations during the forward process. In addition, because SAF docs not require retaining the information of membrane potentials in Zhou et al. (2021) can reduce memory usage during training compared to ОTTT. Furthermore, this propagation strategy only executed during training, and during inference, we can replace the propagation with the spike train without approximation error. We demonstrate this by proving that the neurons for spike accumulation are identical to the Fire (IF) [Tapique] 1907) neuron, which are commonly used as SNN neurons. This result indicates that the SNN composed of LIF neurons can achieve the same accuracy using the trained parameters of SAF (i.e., SAF is capable of inference by the SNN composed of LIF neurons). Furthermore, we prove that the gradient of the SAF is consistent with that of the OTTT, which trains at each time step, and that by summing up the gradients at each time step, SAF is also consistent with the gradient of the Spike Representation. This shows that SAF can accurately bridge the gap between Spike Representation and OTTT.
(LIF) (teein 1965) neuron, which is a generalization of the Integrato-and-
2

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
Main Contributions
(A) We propose SAF, which trains SNNs by only spike accumulation, halving the number of operations in the forward process, reducing the memory cost, and enabling inference on SNNs composed of LIF neurons.
(B) We prove theoretically that the neurons for spike accumulation absolutely identical to the LIF neuron.
(C) Our study also shows that the gradient of SAF is theorctically consistent with the gradient of the Spike Representation and also with that of OTTT, which trains each time step.
(D) We consider the situation that SNNs have feedforward or feedback connection like brain and discuss the equivalence of SAF with OTTT and with Spike Representation.
(F) Brief experiments confirmed that for training at each time step, the training results were in close agreement with OTTT while reducing the training cost.
2
Related Work
Regarding SNN training, there are two research directions: conversion from ANN to SNN and direct training. The conversion approach reuses the parameters of the ANN while converting the activation function for the spiking function (Dichl ct al. 2015 Deng and Gu 2020: Han et al. 2020). This approach can be employed by wide range of many trained deep-learning models, and there are use cases for Lasks other than recognition (Kim let al., 2020 Qu et al. 2023 However, because the accuracy tends to be proportional to the number of time steps and although several improvement methods have been proposed (Chowdhury et al., 2021 Wu al.) 2021), high-precision inference is still difficult for few time steps. Meanwhile, direct training does not use the parameters of the trained ANNs. Thus, the non-differentiable SNNs are trained by some approximation techniques. One of the most significant techniques is to utilize the surrogate gradient (SG). SGs enables backpropagation by approximating the gradient of non-differentiable activation functions, and various types of SGs (e.g., rectangular or derivatives of sigmoid functions) have been propased (Shrestha and Orchard
2018: Wu et al. 2018 Lian et al., 2023; Suctake et al., 2023). Other recently proposed methods include those based on the clamp function (Meng et al.) 2022) or implicit differentiation on the equílibriun state (Xiao ct 2021b. These employ Spike representation, which propagates information such as firing rates, not spike trains, and have the advantage of being able to train SNNs like ANNs (Thiele et al.. 2019 Zhou et al. 2021 However, these methods assume the time step T oo, then T must be sufficiently large to achieve high accuracy. In addition, these are only differences in how to approximale; the basic approach is similar to SGs. Although there are bio-inspired training methods, such as Hebbian learning rule (Hebb 2005 Frémaux and Gerstner 2016) and spike timing dependent. plasticity (STBP) (Bi and Poo 1998 Bengio et al. 2015): In particular, learning rules based on eligibility traces, such as SuperSpike (Zenke and Ganguli 2018 a method for improving SpikeProp (Bohte et al., 2002), are associated with three-factor plasticity rules (Neftii al. 2019). OTTT, the main focus of our study, has also been mentioned being associated with the three-factor plasticity rule. However, training multi-layered SNNs with SuperSpike is challenging, and it is necessary to introduce local errors in each layer (Kaiser et al. 2020). Nonetheless, in Kaiser et. al. (2020), fewer than ten layers were trained, and training deeper SNNs has still been challenging. On the other hand, OTTT can train deeper networks such as VGG, which is useful for many applications.
In the following, we discuss the OTTT (Xiao et al. 2022) most relevant to our study. Because OTTT variant of BPTT with SG, allows for low-latency training. In addition, is sufficient for OTTT to maintain the computational graph only for the current time step during training, different from the standard BPTT with SG. Thus, training can be performed with constant memory usage However, OTTT requires additional information for propagating the spike accumulation as well as the spike train for the forward process, which can increase training time. In addition, because OTTT is based on
time steps increase.
3

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
the LIF neuron, it must retain the membrane potential, which can increase memory usage. Furthermore, it is important to note that while OTTT and Spike Representation have similar gradient directions, their gradients do not necessarily match exactly (i.e., the inner product between their gradients is positive).
Preliminaries
3.1 Typical Neuron Model
In this subsection, explain the LIF neuron, which is widely used in SNNs. The LIF neuron neuron model that considers the leakage of the membrane potential, and its discrete computational form is follows:
u ( = 1( - 1] Vin 8 s - 1]) + + +b+++ s HHllt(l11 Vin),
(1)
where s'[t], u'(t), w' and b' are the spike train, membrane potential, weight and bias of l-th layer, respectively, A I is the leaky term, and À is set to 1 if we use the IF neuron, which is special case of the LIF neuron. Also, H is the element-wise Heaviside step function, that is, = when the membrane potential u|l] exceeds the threshold Vin- From this relation, the membrane potentials u'[t] are computed sequentially and must retain the previous membrane potential u'(t [f[t-1]. 1]. This the same in the case of OTTT, which uses the LIF neuron described below.
3.2 Training methods for SNNs
This subsection introduces two training methods that are closely related to our method: Spike Representation and OTTT.
Spike Representation
Spike Representation is method of training
SNNs by propagating information differently the spike In this article, we consider the weighted firing rate all in Xiao al. (2021 Meng et al. (002); Xiao et. al. (2022). Likewise, where x is the value of the input data. Then, given convergent sequence m m (t- 00), known that a(t) + a (m²/Va) (t + oo) Using this convergence and under the assumption that the time step T is sufficiently large, the weighted firing rate in the (1 + 1)-th layer is approximated as (TT a(W'a'[T] + bb+1 /Va)). We consider the loss L as [[] = [(a(T), v) (where L. a convex function like cross-entropy and is the label). Then the gradient of with respoct to W is computed as follows:
trains (Xiao al. 2021 Meng al. 2022
Ei x//// -T=
define the weighted average input mlt
holds (Xiao et al. 2021), where is the element-wise clamp function (xx) min miinmaa(00,x))
ÖL
OL
I+1
da i+1 [T]
da Da'+ITT '[T) Ow
II N
(2)
dw!
da NaN
da [T]
SR
where N represents the number layers.
Spike Representation can include a feodforward or feedback connections, which have been frequently used in recent years (Xiao al., 2021 2022; 2023), where notice that, feedforward and feedback connections do not refer the weights between adjacent layers but to additional weight matrices connecting any layers | and V. When there is feedforward connection, the gradient ( holds. However, in the case of feedback connection, it does not hold because we need to calculate implicit differentiation. See Appendix B.6 for detailed explanation.
4

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
Online Training Through Time
OTTT (Xiao et al. 2022 is a training method based on BPTT with the surrogate gradient (SG). BPTT with SG enables low-latency training; however, during training, it requires the computational graph to be maintained at each time step, resulting in substantial memory usage when a large number of time steps are involved. OTTT solves this problem and allows for training with minimal memory consumption.
In OTTT, for the forward process, (weighted) spike accumulation a|4] Ei= x is propagated in addition to spike trains s(t] computed by the LIF neuron. Defining the loss at each time step L C(slt), OTTT computes the gradient at time t as follows:
OLL! owi
01 Os
Os Os [t]
0g 4 [1] Orl +
a
II M=N-1
(3)
Note that since this equation uses the accumulated information of spikes up a' [t] at the current time OTTT implicitly trains using information up to i, not just the current time. This corresponds to the fact that membrane potentials accumulate information from the past. Note also that the term ds 0s/Ou 00/0u is non-dilferentiable at VaN; thus, approximate it with the SG; for example, since [1]+2[ can decomposed into (0s [t]//u/ (1) [t)) we also replace 0f"+"[]]/0e
with the SG. Like the case Spike Representation, OTTT can include a feodforward or feedback connection, and the gradients are almost the same as (3)
Xiao et al. (2022) proposed two types of training approaches: OTTTo, where parameters are updated at each time step using OL[t)/Ow', and OTTTA, where parameters are updated collectively by summing @L(t)/öW' up to T.
In particular, they proved that the gradient descent directions in OTTTA and Spike Representation are similar, ie., the inner product between their gradients is positive.
Spike Accumulation Forwarding
In this section, we introduce proposed method, SAF, which only propagates (weighted) spike accumulation a[t). We first explain the forward and backward processes. Then, we prove that SAF can be consistent with OTTT and Spike Representation. We also show that the feedback connection can be added to SAF, as in Xiao et al. (2022), and furthermore, foodforward connections can also be incorporated into the SAF. For summaries of the main formulas, see Appendix
4.1 Details of SAF
Forward process
As mentioned earlier, for the forward processes of conventional SNNs, the spike trains s[t] are propagated. In other words, the firing state of the spike for each neuron al each time step is relained. In SAF, as OTTT and SR, instead of the spike trains, propagates (weighted) spike accumulation a|t] Σi=o TT meaning that it retains the (weighted) count of the fired spikes up to the current time for cach neuron. Additionally, although in conventional SNNs, the spike firing is determined with the membrane potential ulf], in SAF, it is determined with (weighted) potential accumulation Ü[t] defined by Ü++(t] x + 1 Aal(t [t- 1J) + b4+1, which corresponds to the membrane potential in the relation With these considerations, SAF is updated as follows:
04+111] w'a' + E 1
(4)
a'+111]= = xa
Ven (Aā' -a 1)),
5

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
where U [0] is the initial value for potential accumulation, and here, assume it to be the initial membrane potential u 2 [0]. Here, the membrane potential u (1] and spike trains [ in the LIF model can be
expressed by (t) and [t 1] as follows:
uu + [ = 04+10 Vin Aā Aa 11,
(5)
s s [t) = (0 Vih (Aaaaaa11 -( 1] +))-
For derivations of (4) and (5), refer to Appendix B.1 Note that, shown in 4 SAF does not need to retain the past potential accumulation ( - 1]. Meanwhile, the various SNNs, including OTTT, require the LIF neurons used for training retain the previous membrane potentials (t 1I, as described above. Therefore, SAF can reduce the memory usage for the forward process compared to OТTT.
As a result, it becomes possible to compute (1) and ( during the process of obtaining Ú++( and a/+ Because it is possible to compute [++ and [++ from w/+ and [ (T= 1,..., (), the forward processes of SAF and SNN composed of LIF neurons mutually convertible. Additionally, because the IF neuron is special case of an LIF neuron (i.c., À =), the forward processes of SAF, when À = 1, and SNN composed of IF neurons are mutually convertible. Furthermore, in OTTT, both s(4] and @[t] need to be propagated during the forward process for efficient GPU computation, whereas in SAF, only a(l) needs to be propagated (see Fig. D. Therefore, SAF can reduce the computation time during training.
Backward process
As with OTTT, SAF can be trained in two different ways. The first method updates the parameters by computing the gradient at each time step. We call this SAF-E. Let LE([] = L(s N g)TT be the loss function. Assuming that Le(t] depends only on (' and (tt,, i..., not on anything up to t - 1, we calculate the derivative based on the definition of forward propagation as
[Lg(t] Da N
da [1]
0a' [t] O0
a'(t]
II
(6)
aw
(t
Note that 00/00 is non-differentiable; we approximate it with SG. Detailed calculations are given in Ap- pendix B.2 Here, we set
dL ! Hā N
[+1 oa 0a--1( (4]
Ja 00-+110 [1] 00
[t]
0āi(t]
Then, it holds that
ÖLg
a 9
(7)
OW
The second method calculates the gradient only at the final time step and updates the parameters. We call this SAF-F. Let LF = fIΣ XT xT E X--1, y) be a loss function. As with SAF-E, suppose that Le depends only on a' a'[r] and Ü'[T]. Simply replacing with T and LE with LF in the above calculation, we obtain
0LF a [' [T). owi
(8)
4.2 Equivalence with OTTT, and Spike Representation
In this subsection, show that SAF-E is equivalent to OTTTo and SAF-F is equivalent to Spike Represen- tation, i..., the forward and backward processes are consistent, respectively. This means that we can train SNNs by SAF and infer by LIF neurons.
6

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
Equivalence with OTTTo
We will transform the gradient SAF-E to be consistent with that of OTTTo when the loss function is Lg(t]. Because Lx[t] does not include any argument before t, we obtain
aLett as
(9)
da N
[8]
The following two equations hold from the forward processes of SAF and OTTT:
a [t]
Ds'+(("
0a!+
as
(10)
0a
[1]
[
aü
Ou
By substituting 99,
10
for
++1 U
in we have
0
0
Os 0s!+100
|+1 14
πI
Os
(
Ou Oul+1
N=N-1
Hence, the following theorem holds from
0L ow aw oT
[T
Theorem The backward processes of SAF-E and OTTTo identical, that is,
A detailed proof is given in Appendix B.3 consistent, SAF-E and OTTTo are equivalent.
Because have already confirmed that the forward process is
Equivalence with Spike Representation
Now, we show that SAF-F is equivalent. to Spike Representation. Setting the loss function of Spike Repre- sentation Lp, from the expression we have
2
OLF
a'
ÔL
II I=N-1
aa T da [T]
d d
OWi
Ven
SR
where d'+ d' ((W'a assume that
+ Vvn), A Et X and is the element-wise product. Now
diag(d (TT))
au ++
for any / = ,....... N 1, where diag( d + (T]) is a diagonal matrix constructed from d'+] The reason why this assumption is valid discussed in Xiao al. (022). Then, we obtain
OLF ow
1 a a'r11 [2].
Vi
SR
Hence, the following theorem holds.
Theorem 2. Suppose that ml() converges when + 0o. Then, for sufficiently large T, the backward processes of SAF-F and Spike Represenlation are identical up to scale fuclor, that is, Owl OLF
ÖLF
Vth
TW
See Appendix B.4 for the complete proof. Because we have already confirmed that the forward process is consistent, SAF-F and Spike Representation are equivalent.
7

---

Published in Transactions on Machine Learning Rescarch (06/2024)
4.3 Feedforward and Feedback Connection
In the brain, there are not only layer stacking, in normal SNNs, but also feedforward and feedback con- nections (Semedo et al. 2022). Therefore, it is important to consider SAF-E/F including these connections. In this subsection, we discuss the relationship between OTTTo, SR, and SAF-E/F in the context of both feedforward and feedback connections.
Feedforward Connection
To begin, we consider the SNN composed of LIF neurons. The forward process of the 11th layer of the SNN with a feedforward connection from the p-th layer to the (a+ 1)-th layer (where q 2 p) with weight W as follows:
u [t] A(u [t 1] - Vea sa+ (t- - 1)) + W bb+t + Wys*(t),
(11)
(t+ H( [t] Ven).
Note that the layers other than the (g + 1)-th layer are the same as of the (q 1)-th layer of the SAF is follows:
Meanwhile, the forward process
W, @"(t), @ [t] A@ - x 1 H(U+1 - Vn Vn 1 1)).
Ü [t] [ b (
x (T)++
[0] +
X
(12)
The layers other than the (q 1)-th layer the with feedforward connection are mutually convertible.
in
The forward processes of SAF and LIF
Regarding the backward processes, the gradients for parameters other than Wy are the same as when there no feodforward connection. The derivative with respect to W, is calculated as
ÖL [t]
g+1
oā
aā++1
= a"(t)
II
aw,
da'tt
a0 [t)
--1
Therefore, 0Lg|t//W, = a [1]
9 2 [d] for SAF-E and 0LF/OW, @F [T] 7+l [2] for SAF-F.
Feedback Connection
The forward process of the (q 1)-th layer of the SNN with feedback connection from the p-th layer to the (q + 1)-th layer (where p) with weight W, is follows:
1--1 [ * Itt 1] Vtn 9 1J) [t b0+1 + Wa s 1],
(13)
8 [t] = H (w Vin).
Note that the layers other than the (g + 1)-th layer are the same as in Mcanwhile, the forward process of the (q + 1)-th layer of SAF is as follows:
1 = w w + (E X7) + [0 + W, [P[1-1],
(14)
a 1 = 1a++1 [- HÜ0++1 Ven (xa 1)).
The layers other than the (q + 1)-th layer are the same as in 14 The forward processes of SAF and LIF with foedback connection are mutually convertible.
Regarding the backward processes, the gradients for parameters other than W, are the same as when there no feedback connection. The derivative with respect to W, calculated as
aLgM a - da [t]
TLg [1]
3a 30
02++1 aū++1[1]
II
OW,
0a'lt]
N-
à 1 [T] for Therefore, we obtain @Lg @tlt 1] 1 1 [t] for SAF-E and OL OLF/OW, /0W, SAF-F.
8

---

Published in Transactions on Machine Learning Rescarch (06/2024)
Equivalence with OTTTo and Spike Representation
We can show the equivalence of SAF-E and OTTTo with a feedforward connection, or with a feedback connection, as well as in Theorem | Morcover, as mentioned in Sec. 3.2 Spike Representation computes the gradients (2), then the equivalence of SAF-F and Spike Representation also holds, even with a feedforward connection.
Corollary 3. For SNN with a feedforward connection or feedback connection 13 the following hold.
(i) The backward processes of SAF-F and OTTTo with feedforward connection are identical.
(ii) Suppose that ml[] converyes when I + oc. Then, for sufficiently large T, the backward processes of SAF-F and Spike Representation with feedforward connection identical up to scale factor.
(iii) The backward processes of SAF-F and OTTTo with a feedback connection are identical.
The proof stated in Appendix B.5
Proximity to Spike Representation
Assume that the SNN have feedback connection same as in 13 Then, the same assertion Theorem does not hold, that is, SAF-F is not equivalent to Spike Representation in general. However, we can show that the gradient descent directions in SAF-F and Spike Reprosentation are close.
Theorem 4. Suppose that m|[] converges when + co. Then, for sufficiently large T, the backward processes of SAF-F and Spike Representation with a feedback connection are similar, that is, for all parameters O.
/0LF
(0L
00
00
SR
See Appendix B.6 for the proof.
Experiments
In Sec. 4.2 we theoretically proved that SAF-E and OTTTo as well as SAF-F and spike representation are oquivalent. In this section, we experimentally compare these methods. As complex and large datasets make difficult to analyze the results, we trained SAF on the CIFAR-10 and the CIFAR-100 datasets (Krizhevsky and Hinton| 2009) and inferred with SNN composed of LIF neurons. This experiment times with different initial parameters, and all approximation was executed by the sigmoid-like SG for fair comparison. We used the same experimental setup as (Xiao et al. 2022), including the choice of SG. The code was written in PyTorch (Paszke et al. 2019), and the experiments were execuled using one GPU, an NVIDIA Tesla V100, 32GB. We show the pseudo-code code of SAF-E and SAF-F in Algorithm | to better understand our mothods. The implementation dctails are in Appendix C The main objective here analyze whether there any inconsistencies between theory and experiment rather than to achieve state-of-the-ar performance.
performed five

---

end for Published in Transactions on Machinc Learning Rescarch (06/2024)
Algorithm 1 One iteration of SAF training.
Require: Network parameters {W"), (b'+1; Time steps T; Number of layers N; Other hyperparameters;
Input dataset
Ensure: Trained network parameters (W"), (b/+1}
1: for == 1,2,..., do
2: % Forward
3: for 1 11,,2..... do
4: Update the (weighted) potential accumulation ( and spike accumulation @'(t] using (4)
5: end for
6: % Backward
for N - 1,.... do
8:
if training option is SAF-E then
Update parameters with (L.(1)/WW = [1 [4 based on the gradient-based optimizer. clse if training option is SAF-F and = then
9:
10:
Update parameters with @L/0OWi -a' ++1 based the gradient-based optimizer. 12: end
13:
end for
14:
5.1 Analysis of SAF-E
We experimentally analyze the performance of SAF-E. First, we compare accuracy. Table [ shows the accuracy when we set T (the difference of gradients are shown in Appendix D). As shown in this table, SAF-E and OTTTo have almost the same accuracy. The values in parenthescs Table show the change in accuracy due to inference by SNN composed of LIF neurons. From this table, we can that the accuracy change due to inference by SNN consisting of LIF neurons is almost negligible in the case of CIFAR-10, On the other hand, CIFAR-100 shows a minor difference in accuracy (compared to the difference in gradient). This could be attributed to the increased task complexity, resulting in more intricate loss function and greater susceptibility to even minor numerical errors affecting final accuracy.
Figure 2 shows the accuracy and loss curves during the training of CIFAR-10. This indicates that the progress during training are comparable. Therefore, we confirmed experimentally that SAF-E and OTTTo are mu- merically close. Similar results were obtained when there Appendix
feedforward or feedback connection (see
Next, we compare the training costs. From Table ! it can be seen that SAF-E takes less time to train and uses less memory during training than OTTT However, OTTT can be executed with constant memory usage even as time steps increase. Therefore, we compared the training time and memory usage at different lime steps. Figures 3 (A) and (B) show the training time and memory different time steps. Note that the training time was measured in one batch. It can be seen that the memory usage of SAF-E does not increase even if the number of time steps increases, similar to OTTTo- Also, from Fig. 3 (B), uses less memory than OTTTo This result stems from the fact that SAF does not need to maintain the previous membrane potential.
that SAF-E
Finally, we compare the firing rate. As shown in Table I the total firing rates of SAF-E and OTTT, are close. Also, the amount of change due to inference with SNNs consisting of LIF neurons also almost negligible, similar to the case for accuracy. Furthermore, from Fig. 3 (C), the firing rates of each layer are almost close too.
These results indicate that using SAF-E reduce the training time and memory compared to OTTTo while achieving close firing rate and accuracy. It was also shown that using the parameters trained with SAF-E to infer with the SNN consisting of LIF neurons is feasible.
10

---

Published in Transactions on Machine Learning Rescarch (06/2024)
Table 1: Performance comparison of SAF-E and OTTTo CIFAR-10 and CIFAR-100. The values in parentheses were the changes in accuracy and total firing rate due to inference by the SNN composed of LIF neurons. Note that training times were measured in one minibatch, and training time and memory were not perturbed between trials.
CIFAR-10
Method T' Memory [GB]
Training Time |sec]
Firing rate [%)]
Accuracy [%]
OTTTo 6
1.656
0.666
15.1440.17 14.76:0..1 (1.048x10-9) 93.54+0.17 (0.016)
93.44410.15
SAF-E
6
1.184
0.468
C0F
Method
T
Memory [GB]
Training Time [sec]
Firing rate [%)
Accuracy [%]
OTTTo
6
1.656
0.666
17.27 16.77++0.08 (1.1131105)
70.70=:019 71.56+00.35 (0.042)
SAF-E
6
1.186
0.464
OTTT_O O
14
Accurac)
0.7
0.4
OTTT_O
1.9 Epoch
Epoch
(A) Accuracy
(B)
Loss
Figure 2: Accuracy and loss curves of SAF-E and OTTTo on CIFAR-10 =T=6)
2.00
OTTT_O SAF-E
OTTT_O SAF-E
1.75
1.5
Avcrage Rate
24
Memoor (G3)
Time [occ]
0.5
038
SAF-E
10
15
20
25
0.6
30
Time Steps
Time Stops
Layer Index
(A) Training Time
(B) Memory Consumption
(C) Firing
Figure 3: Training time, memory consumption, and firing rate of each layer of SAF-E and OTTT,
CIFAR-10.
11

---

Published in Transactions on Machine Learning Rescarch (06/2024)
5.2 Analysis of SAF-F
In this subsection, experimentally analyze the performance of SAF-F. First, compare accuracy. The results are shown in Table 2 and Fig. 1 (the difference of gradients are shown in Appendix D). Since the spike representation method is effective when T is large, there is no theoretical guarantee that SAF-F can infer well in a short time step. However, Table 2 shows that the accuracy of SAF-F is almost the same for T= and T = 32. Note that Spike Representation methods, except for SAF-F, do not use SGs, which makes precise comparisons difficult. Therefore, we only compared SAF-F with OTTTA
As with the previous results, the accuracy change due to inference with SNNs consisting of LIF neurons is almost negligible. Meanwhile, the accuracies of SAF-F and OTTTA A are close, though from the perspective of standard deviation, there seemss to be a difference. From Sec. 1.2 the gradient directions of Spike Represen- tation and SAF-F are identical, but those of Spike Representation and OTTTA are only similar. Therefore, the gradient directions of SAF-F and OTTTA are also only similar. This is thought to be the cause of the differences accuracy and loss.
Next, we compare the training costs. From Table 2 and uses less memory than OTTTA, This trend is also similar when the time step is varied (see Figs. (A) and (B)).
it can be seen that SAF-F requires less time for training
Finally, we compare firing rate. As shown in Table 2 as for accuracy, the change of the total firing rate by inferring with SNNs consisting of LIF neurons is almost negligible. Meanwhile, the total firing rate of SAF-F is smaller than of OTTTA- In addition, from Fig. (C), can be seen that the firing rate of each layer (especially the first layer) is smaller in SAF-F than OTTTA- These differences also indicate that SAF-F and OTTTA A are generally not identical.
From the above analysis, we can say that SAF-F is a better choice than OTTTA in terms of training time, memory usage, and firing rate. Also, as with SAF-E, inference can be performed by the standard SNN using the parameters trained by SAF-F.
Limitation and Discussion
In this paper, we have shown that SAF-E coincides with OTTTO and that SAF-F coincides with OTTTA On the other hand, Xiao et al. 2022) shows that OTTT methods are more accurate than BPTT. Given the concordance between SAF and OTTT, SAF is more accurate than BPTT. We consider that the better accuracy than BPTT due to the difficulty of achieving optimal rollout all times in BPTT, as known by vanishing gradients, and the fact that most of the datasets used in the field of deep SNNs are time-independent labels. Therefore, we assume that dā~ 11/0W' is zero (see Appendix B.2 This assumption is intended to even out the gradient's 's effect on parameter updates at each time, as described in Appendix B..2 and is also valid for widely used time-independent datasets. On the other hand, since SAF trains using information up to t by the spike accumulation à, not just the current time, SAF can implicitly train at cach time while using information up to the previous time. The above assumption can be easily removed; however, its validation would require labeled time-dependent data set. This would require the preparation of an appropriate data set, which is outside the scope of this paper, considering theoretical consistency with OTTT and is therefore considered one for future rescarch.
12

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
Table 2: Performance comparison of SAF-F and OTTTA on CIFAR-10 and CIFAR-100. The values in parentheses are the changes in accuracy and total firing rate due to inference by the SNN composed of LIF neurons, Note that training times were measured in one minibatch, and training time and memory were not perturbed between trials.
C-FAR-10
Method
T
Memory [GB]
Training Time [sec]
Firing rate [%)
Accuracy [%
A
6
1.656
0.661
15.51+0.10
93.39-10.16
OTTTA
32
1.656
3.474
13.96:0.20
93.62=0.04
SAF-F
6
1.157
0.247
10.50+1.119 (3.306x10 10
93.09110.15 (0.076)
SAF-F
32
1.157
1.077
10.65+:.12 (0.963 10 5)
93.25=0.07 (0.002)
CIFAR-100
Method
T
Memory [GB]
Training Time (sec]
Firing rate [%]
Accuracy [%]
OTTTA
6
1.656
0.666
18. 08.26+0.21
70.18=0.20
OTTTA
32
1.656
3.479
16.6240.20
70.77:0.16
SAF-F
6
1.157
0.249
12.19::0.22 (1.106 10
70.58::.34 (0.002)
SAF-F
32
1.157
1.077
12.2110.11 (1.016x 5)
71.73110.04 (0.096)
69
14
SAF-F
12
0.
Accuracy
3.5
0.41
0.5
OTTT_A A
641
250
Epoch
(A) Accuracy
(B) Loss
Figure 4: Accuracy and loss curves of SAF-F and OTTTA on CIFAR-10 (T=6) 6).
OTTT_A
OTTT_A
1.15
1.50
Time (sec)
239
Memory (GB)
1:25
Average Firing
1.00
22
OTTT_ A
0.8
Steps
Time Steps
Layer
(A) Training Time
(B) Memory Consumption
(C) Firing rate
Figure 5: Training time, memory consumption, and firing rate of each layer of SAF-F and OTTTA on CIFAR-10. It can be seen that the firing rates do not match, which agree with the theoretical result that SAF-F is not always consistent with OTTTA
13

---

Published in Transactions on Machine Learning Rescarch (06/2024)
7 Conclusion and Future Work
This article proposed SAF. SAF is a training method of SNNs that propagates the spike accumulation during training; however, SAF propagates the spike trains during inference, as do other SNNs. This article showed that SAF trained at each time step (SAF-E) is equivalent to OTTTo, and SAF trained at the final time step (SAF-F) is also equivalent to Spike Representation. We conducted experiments on the CIFAR-10 datasel and confirmed that the experimental results are consistent with these assertions and that training time and memory of SAF are reduced compared OTTT.
Most SNNs, including OTTT, Spike Representation, and SAF, train under time- time-independent labeled training data. Then, the issue of improving SAF to train even if the labels change at cach time step remains for future work. Other remaining tasks are to experimentally confirm the similar results of this work for other surrogate gradients and to investigate the theoretical relationship with other learning rules such as SuperSpike.
SAF, as presented this article, assumes training on a GPU. Therefore, it may not be suitable for training on neuromorphic chips. However, we believe executing training on GPUs and inference on neuromorphic chips is reasonable. In the future, also plan to verify inference the neuromorphic chip, including the extent to which numerical errors are affected by differences in computing environments.
References
Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-lcaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al. Truenorth: Design and tool flow of a 65 mw million neuron programmable neurosynaptic chip. TEEE transactions on computer-aided design of integrated circuits and systems, 34(10):1537 1557, 2015.
Yoshua Bengio, Thomas Mesnard, Asja Fischer, Saizheng Zhang, and Yuhuai Wu. Stdp as presynaptic activity times rate of change of postsynaptic activity. arXiv preprint orXie:1509 05936, 2015.
Guo-qiang Bi and Mu-ming Poo. Synaptic modifications in cultured hippocampal neurons: dependence spike timing, synaptic strength, and postsynaptic cell type. Journal of neuroscience, 18(24):10464 10472, 1998.
Sander M Bohte, Joost N Kok, and Han La Poutre. Brror-backpropagation in temporally encoded networks of spiking neurons. Nearocomputing, 48(1-4):17-31 37, 2002.
Sayeed Shafayet Chowdhury, Nitin Rathi, and Kaushik Roy. One timestep is all you need: Training spiking neural networks with ultra low latency. ai preprint 100.5990,, 2021.
Mike Davies, Narayan Srinivasa, Tsung-Hay Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. leee Micro, 38(1):82 99, 2018.
Shikuang Dong and Shi Gu. Optimal conversion of conventional artificial neural networks to spiking noural networks. In International Conference on Learning Representations, 2020.
Peter U Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, and Michael Pfeiffer. Fast- classifying, high-accuracy spiking deep networks through weight and threshold balancing. In 2015 Inter- national joint conference on neural networks (IJCNN), pages 1-8, IEEE, 2015.
Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun. Repvgg: Making
vgg-style convnets great again. Tn Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pages 13728 13737, 2021.
Nicolas Frémaux and Wulfram Gerstner. Neuromodulated three-factor learning rules. Frontiers in neural circuits, 9:85, 2016.
plasticity, and theory of
14

---

Published in Transactions on Machine Learning Rescarch (06/2024)
Samy Wu Fung, Howard Heaton, Qiuwei Li, Daniel McKenzie, Stanley Osher, and Wotao Yin. Jfbrjacobian- free backpropagation for implicit networks. In Proceedings of the AAAI Conference on Artificial Intelli- gence, 2022.
Zhengyang Geng, Xin- Yu Zhang, Shaojie Bai, Yisen Wang, and Zhouchen Lin. On training implicit models. In Advances in Neural Information. Procesing Systems, 2021.
Bing Han, Gopalakrishnan Srinivasan, and Kaushik Roy. Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network. Tn Proceedings of the IEEE/CVE conference computer vision and pattern recogration, pages 13558 13567, 2020.
DO Hebb. The Organization of Behavior: A Neuropsychological Theory. Psychology Press, 2005.
Jacques Kaiser, Hesham Mostafa, and Emre Neftci. Synaptic plasticity dynamics for deep continuous local learning (decolle). Frontiers in Neuroscience, 14:515306, 2020.
Seijoon Kim, Seongsik Park, Byunggook Na, and Sungroh Yoon. Spiking spiking neural network for energy-efficient object detection. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 11270-11277, 2020.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, 2009. URL htttp:///www www..tooottoooooo httpaa/b.ettu//kra///aaaannnnnn. featuress-- -aaiereee--00---1 1009-TR.ppd
Louis Lapique. Rescarches quantatives sur l'excitation electrique des nerfs traitce comme une polarization. Journal of Physiooogy, Palhology and Genelics, 9:620 635, 1907.
Shuang Lian, Jiangrong Shen, Qianhui Liu, Ziming Wang, Rui Yan, and Huajin Tang. Learnable surrogate gradient for direct training spiking neural networks. In Proceedings of the Thirly-Second International Joint Conference Artificial Intelligence, IJCAI-23, pages 3002 3010, 8 2023.
Tao Luo, Weng- Fai Wong, Rick Siow Mong Goh, Anh Tuan Do, Zhixian Chen, Haizhou Li, Wenyu Jiang, and Weiyun Yau. Achieving green ai with energy-efficient deep learning using neuromorphic computing. Communications of the ACM, 66(7):52-517 57, 2023.
Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, and Zhi-Quan Luo. Training high- performance low-latency spiking neural networks by differentiation on spike representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12444-122 1144332 2022.
Emre O Neftei, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient- based optimization spiking neural networks. IEEE Signal Processing Magazine, 36(6):51-63, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, tigh-performance deep learning library. Advances in neural informalion processing systems, 32, 2019.
Siyuan Qiao, Huiyu Wang, Chenxi Liu, Wei Shen, and Alan Yuille. Micro-batch training with batch-channel normalization and weight standardization. arXiv preprint arXiv:1903. 10520, 2019.
Jinye Qu, Zeyu Gao, Tielin Zhang, Yanfeng Lu, Huajin Tang, and Hong Qiao. Spiking neural network for ultra-low-latency and aigh-accurate object detection. arXie preprint arXiv:2306. 12010, 2023.
João D Semedo, Anna I Jasper, Amin Zandvakili, Aravind Krishna, Amir Aschner, Christian K Machens, Adam Kohn, and Byron M Yu. Feedforward and feedback interactions between visual cortical areas use differont population activity patterns. Nature communications, 13(1):1099, 2022.
Sumit B Shrestha and Garrick Orchard. Slayer: Spike layer error reassignment in time. Advances in neural information processing systems, 31, 2018
15

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
Richard B Stein. A theoretical analysis of neuronal variability. Biophysicol Journal, 5(2):173 194, 1965.
Kazuma Suetake, Shin ichi Ikegawa, Ryuji Sain, and Yoshihide Sawada. N Time step reduction spiking surrogate gradients for training energy efficient singlo-step spiking neural networks. Neural Networks, 159: 208 219, 2023.
Johannes C Thiele, Olivier Bichler, and Antoine Dupret. Spikegrad: An ann-equivalent computation model for implementing backpropagation with spikes. International Conference on Learning Representations, 2019.
Jibin Wu, Yansong Chua, Malu Zhang, Guoqi Li, Haizhou Li, and Kay Chen Tan. A tandem learning rule for effective training and rapid inference of deep spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2021.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience, 12:331, 2018.
Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Yisen Wang, and Zhouchen Lin. Training Feedback Spiking Neural Networks by Implicit Differentiation the Equilibrium State. Advances in Neural Information Processing Systems, 18(Neu11 14528, 2021.
Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, and Zhouchen Lin. Online training through time for spiking neural networks. Advances in Neural Information Processing Systems, 35:20777-07300, 2022.
Mingqing Xiao, Qingvan Meng, Zongpeng Zhang, Yisen Wang, and Zhouchen Lin. SPIDE: A purely spike- based method for training feedback spiking neural networks. Neural Networks, 161:9 24, 2023.
Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural compulation, 30(6):1514 154111 2018.
Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with directly-traincd larger spiking neural networks. In Proceedings of the AAAI conference artificial intelligence, volume 35, pages 11062 11070, 2021.
Shibo Zhou, Xiaohua Li, Ying Chen, Sanjeev T Chandrasekaran, and Arindam Sanyal. Temporal-coded deep spiking neural network with casy training and robust performance. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11143 11151, 2021.
16

---

Published in Transactions on Machine Learning Rescarch (06/2024)
Appendix
A List of main Formulas
The neurons and gradients are as follows. Note that 8'1]. u w11, W and b' are the spike train, membrane potential, weight, and bias of 1-th (also denoted by p-th and q-th) layer, respectively. Also, À I 1 the leaky the clement-wise Hcaviside step function,
term, Vn is the threshold, I, Lg, and Lr are the loss functions, N is the number of layers, alt] Σ s//r/ > A the weighted firing rate, @|t] X Aā
the (weighted) spike accumulation, and (1+1(0) = -U - [t- 1]) b'+11 the (weighted)
potential accumulation.
A.1 Neurons
LIF neuron
w a [t] = A(ult1(t 1]
Vin s - 1]) + w ss [t] + +
[/+([t] H(u' V.n))
SAF neuron
(++(()) w'a + 1 ~ a'+1( xa Xt (Ū++ Vel (Aā' + 1)).
A.2 Neurons with feedforward connection
LIF neuron
( A(uotI(t 1]- Veh 80 11) + W bet1 + w
10+1( H(u Ven).
SAF neuron
12
Ū [1] + b b+
(0] Wy aP(1),
ar+1 [t] = x@ t 1 + Vi 1] + 1)).
A.3 Neurons with feedback connection
LIF neuron 13
(0- X(u0+1(t -1] 1] Vo 80 11) W9 bo+1 + W, sP(t 1},
s = H(u - Vin).
SAF neuron 14
1 = + + + W,, @0 1P(1-1 [-1],, 1],
a--1[1] = Xa++1/ # (Aa 1)).
17

---

Published in Transactions on Machine Learning Rescarch (06/2024)
A.4 Gradients
Spike Representation
OL
dL
[+1
da 0a d [T)
da 0a!+i(r) ow
II
Ow
TT
SR
oTTTo
3
t
ts {
0g [8]
a'
OW
as [1]
Ore Dul+110
-=
OTTTA:
0L OW
a1D
1+1
08 (1ttie0
Σ
Üs
Or +
-=N-1
SAF-E
6
aL
1+1
0a [t
06 [t]
(1))
OW
0a N
0a
00++1
V=N-1
SAF-F 8
0LF OW
ÖLF
oā [T]
[T]
00 [T]
a' [
Na.N [T)
Oa'
i=N-1
B Derivation and Proofs
B.1 Derivation of 44 and
5
In this subsection, we derive and 5 through proving that 44 and 5 hold for any t E {1...., T} with mathematical induction.
First, we prove that 4 and
5 hold for f 1. We compute 0'1(11] based on definition as follows:
(++((1]
Xo - [0] (a' Xa xā'(0) + b
- 10
Taking account into a0+ [ 0, we obtain
u [1] Mut+100 Vo + (0) W s [1] + b
Ū [0] Vu a (0)++)))=-000)))+) W' (a' [1] - Aāl(o)) bl+1
= 0 Vin (0)
With this equation, we have
+ = H( Vin)
=H(Ū++[1] Via(aa(110)) + 1)).
18

---

Published in Transactions on Machine Learning Rescarch (06/2024)
Then, [1] can be computed as follows:
a ['} Σ x x [7]
=aa [0) + 8 [1]
1a H [1 Via [0] 1)).
Therefore, 44 and (5) hold for t= 1.
Next, we prove that 4 and
5 hold for when they hold for any {I,.... , 1}. Assuming that 4 and
hold for any l.... 1}, we have
0 1 lI + (ā'lt) Aā' 11) + +
=x(w'att- 1] 6/-1
1-2
X + - x1-10++10)) w'(''(t xā'it 1]) + b
= =W'a [1] 1 Σ
x + x0
==0
Also, utt [t] is computed follows:
u ( [1]
A(u + [t
Vih s s 1]] W s's b'+1
1(01+11-11 -1 Vinn (1a [1-
- - (1 w'(''1 - xā'l 1]) + 6
-Vadall 1|.
With this equation, we have
s/ + [t] = (u - Vin)
= Ven(Aā't1 1]+ 1)).
Then, [t+ can be computed as follows:
a++ = Σ - (
T-
=ā =aa
+ [1]
= =aa+ 1] + H 2Ū [t] - Vea (Xa
1] + 1)).
Hence, [4 and 15 hold for when they hold for any E {l....,t 17.
Therefore, 4 and hold any t E (1....,77 T).
B.2 Derivation of and 8
First, since LE((] C(s v)/T, we have
ÖLg OW
aL It ds
Os
[ OW!
Assuming that L [t] depends only on a'(t] and ū+ i.c., not on anything up to - 1, we regard à @N(t) N [t] + Const. Then, we calculate that
as
ÖL it] OW
Os
N
aa N It]
@L
0ā N [t]
(15)
Ns
Jo
OW!
No
OW?
19

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
Next, it follows from that
0a 1
0a 2 [t]
0à aam1 - da
La"1 [8] (t
(16)
OW
1 1] OW a ow
0a a [t] 0a"-116
a [t] 0a O
0ā M [t] 0ā 0aw-1p
(17)
0aN-1[4 OW
where we regard 0ā [t 11/0wl as 0, and U [t] function of @% [N) It should be noted that the assumption 0t 1//0w = 0 implies equalizing the effect of gradient on parameter updates at each time. Indeed, /// is already used for updating parameters at time 1-1. Therefore, using also (/[t-1)/OW in the calculation of @@W will result in excessive influence of the gradient at t on the parameter update at L.
By repeating the process from 16
17
in the same way, we derive
aā NaNDy [1)
oa 0a!+1't OW
OW
0a
Therefore, the gradient SAF-E calculated as follows:
0 0
8n
[4]
Oa 0a't1
II
OW
JāNp [
OW
\=N-1
0gg
I+1
aa'+ [t]
da 0a'+'(t
[t]
(18)
Ja
0a'()
a 1
N-
where note that the Heaviside step function H is elemnent-wise. This concludes the derivation of by setting
aL
tl
I+1
da aa-
[+]
0a'+
06
- ++ f
[1
Ja
da [t]
aU++
Derivation of 8 is almost the same as that and Ü'rr.. Since LF = [() Et X y). 15 holds when is replaced with T, and Læ((] is replaced
As with SAF-E, suppose that Le depends only on a'(
with Lp. The rest derivation the same as case
Then we have
OLF ow = a|
OLF
da (
oa 0a'+1[7] = o [T)) a01-1[T]
(19)
(aN(T)
0T
Y=
100 in 18 and 19 non-differentiable; we approximate it with the SG (refer to Note that 0a (t/00444
30
B.3 Proof of Theorem
dLE(!] OW!
@LEl! OW
Theorem
The backward processes of SAF-E and OTTTo are identical, that is,
OT
Proof. The gradient of SAF-E and OTTTO are as follows:
dL
[0Lelt]
OL No [t]
[+1 II
as [t]
0s [t]
a o [t).
a'(t)
ow
owi
ds
tu
or
N=
20

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
We show that the gradient of SAF-E is equal to the gradient of OTTT by transforming o [t]. First, we calculate 0Lelt/00ä [t]. Because Læ[t] does not include any argument up to t it holds that
[ p as (
0hel 2& [d]
da N [1] dg [t] 0a
[1]
Then, have
9Lg [t]
aa [
Oa 0at+1 [t]
0
4 [t]
[t)
aa
00 [1]
Second, because of the forward process of SAF, i.e.,
and we obtain that
Oa I [t]
8(
[ [t] Vin(Aa
+ [t 1]+11 1)
au
8(u' [t] Van).
where 8 represents the delta function. forward process of OTTTo, i.e., 1
On the other hand, the following equation can be derived from the
0s (1 [t] 5(t4+11) [4] V.n.).
Ou +1 [
Hence,
06 [1]
as
(20)
a0
Ou 0u'+ 11
Then, we have that
[1 0
(a
0s +
0 0s [t]
da [1]
Ou [[]]
W=N-1
From 4 ā [1] depends on at+ [ 1] and Ü'((), while a - 1] does not depend on a/+1[(] so 0a
1//8a' 1 = 0. Also a 14/ 10/000000 W since the forward process of SAF 44. Thus, we calculate that
oa
[t]
06'+ + [t] aū
aa [t]
aa
aā
W
(21)
0a
a
da
aū+
au
Approximating 0a +1 [t)/0
1+ [t]
with the SG (refer to 30 ) 0a [t)+t[t]//0 [ta/ becomes implementable. More-
over, (++ depends on u [1] because the forward process of OTTTo is given by Hence,
as [t]
Os
Ou [t]
0s* [
Oui+ w
(22)
Os [t)
Ou
[t]
as
Combining 20
21 and
22
we have
0a + [t]
0 [tt] Os [t)
aa
In the end, we transform 1 as follows:
aLg(t it]
1+1
as [t] [s((t]
0s t
-
++
(23)
2s N
N=
This concludes the proof.
21

---

Published in Transactions on Machine Learning Rescarch (06/2024)
B.4 Proof of Theorem 2
Theorem 2. Suppose that ml[) converyes when 0o. Then, for sufficiently large T, the backwarrd processes
dLF of SAF-F and Spike Representation are identical up to scale factor, that is, ow! = Víh
OLF
dwi
SR
Proof. From assumptions, + [2] = a (W + where o(x) min(maxe (0, x), 1). Therefore,
the followings hold for I + 1, N 1:
OLF
OLF da [T
(24)
oW da [T] owi
0a++1p [T) da (T) (
(25)
Ow
da'[T] OW!
By repeatedly substituting 25 for
24
we can calculate 0L, / @w as follows:
OLF
OLF
1+1
da
+ [ [T]
[T
II \=N-1
(26)
OW
da
TT
da
OW
This is the gradient Spike Representation and denote it by (0Lp/OW") SsR. (0Lp/OW')sn is in proportion to @Lp/OW = = of [T], which the gradient of SAF-F.
SSR.
We will show that
First, let A = Erz x Then a a'( = ( for any layer I. From linearity of differentiation and change of variables, we have followings:
dLp
OLF
Da i [ da
Oa + [
da ++ [T]
Ada +1 T
(27)
Da [T]
AOa
[T]
[T]
0a'
OW
OW
Substituting
27 into
26
then we obtain
ÖLp OW!
OLF
00 [T]
0a'+1(T)
aa
Oa [T)
OW!
N=N-1
Second, it follows from a' [T] = a' [T]]AA and a = oo a bl+1) V/an)) that
1 (1 W [ + ( +
a a Ao
Ven
Here, laking care that is element-wise, calculate as follows:
OLF
a' a'
OLF Jā~(T) [T]
da aat+IT
1
1 ( (
② Ao
OW
VnA
Ta
Vin
A
I=
a' [T]
ÖLF
I+1
da [
Vah
da [T]
where we set d'+1[T] = o' (WW' + b(+1) (++1//Va) Vu.), and is the clement-wise product. Now we assume that
0s 0 1+1 T
diag d d''''(T) [T]),
In +1 [T]
22

---

Published in Transactions on Machine Learning Rescarch (06/2024)
for any 1 NV 1,, - where diag [ is a diagonal matrix constructed from [T). The reason why
this assumption is valid discussed in Xiao et al. 2022). Then, we obtain from (20 that
OLF OW!
a' T'T
OLF
0ā 0āa
0a'+1r] aU(+1[]]
1
a 1o [T].
Veh
da~N
[T)
da [T]
Veh
SR
N=N-1
B.5 Proof of Corollary
Corollary 3. For SNN with a feedforward connection or a feedback connection (13 the following hold.
(i) The buckward processes of SAF-E and OTTTo with feedforward conneclion are identical.
(ii) Suppose that m|[] converges when l + oc. Then, for sufficiently large T, the backward processes of SAF-F and Spike Representation with a feedforward connection are identical up a scale factor.
(iii) The backward processes of SAF-E and OTTTo with a feedback connection are identical.
Proof. First, we show (i). The gradients for parameters other than W are equal to the gradients when there no feedforward connection. From Theorem D these gradients are same as OTTTo Therefore, we only need to check @Le(t//WW, = (@Le(t)/OWy)om In fact, the gradient of OTTTo calculated (see Xiao et al.
(2022) as
@LEIt O OT
g+1
0g On
@014
O&
(s((1
[1)
\==-1
and 23 holds, then we have
[Tgl!
0T [1] 9 [t] =
aw, OT
aw,
Hence, SAF-E is equivalent OTTTo even if there is leedforward connection in SNN. The same method can be used to prove (iii).
Next, we show (ii). The gradients for paramcters other than W remain the same as when there is no leedback connection. According to Theorem 2 these gradients are identical up to a scale factor to Spike Representation. Therefore, we only need to check åLr (t]/Ww, Van (oLrlttt/WW))ss
When there is
feedforward connection, 2 holds. Therefore, can be proved in the same way as Sec. B.4 noting that
a [T] ((Wa) 1+1 (T)/V))
B.6 Proof of Theorem
Theorem 4. Suppose that m[() converges when + co. Then, for sufficiently large T, the backward processes of SAF-F and Spike Representation with feedback connection are similar, that is, for all porameters 0.
Ö..
00
00
SR
Proof. From the assumption, the firing rates of each layer converge to these equilibrium points: (al+2)* f(41((a))*) (1 q). (a = ff fo+2((a*t))) (a?)*) fo+1((a") (a?)*), where fiti((a))))
(W"(aa) + V/VV), and f (a9)") (aW) + bo+1 + W, (a(a") /Van). Since T 1,
0Lp
000 00 can be calculated using the spike representation as follows (see Xiao et al. 2021 Xiao et al. 2022
23

---

Published in Transactions on Machine Learning Rescarch (06/2024)
OLF 30
aLF 0a!+i[T] (1-
fi+ [Tl(T)
-1 i
(28)
80
SR
where 0ft41/0a [T] denotes Jacobian matrix and I denotes identity matrix. Here, we regard (I (T)) of (88 as an identity matrix, and denote by
OLF 00
OLp
0f4 (T))
(29)
0a!+1
00
SR
Then, it is proved in Fung et al. 20022 and Geng et al.
2021 that
OLF
ÖLF
0,
00
00
SR
SR
where ) denotes inner product. If 0 is not Wb, right- side of (29 is equal to right- hand side (26 then from Thcorem
0LF
1 aLF
00
Ven 00
SR
Next, consider as W.. [T} can be approximated by (W b0+1 + W, (TT)))/ and a²[7] aT - 1] because large. Therefore, a ~ ((W b + W, a² T 1))/Vi))
Setting | o' ((W'a' (T + bit1 + W. [T 1l)/Van) and calculating similar to Sec. B.4 we derive that
[T]
ÖLp ow,
@+ - 1]
ØLp
[+1 II
aa T
o ddt'(rr) [TM]
Vih
da T
(T|TT
i N-1
1
@T(T"
1] g a+1
[T]
Ven
U
OOF
Ven
(WW()
In the end, take the inner product between their gradient we obtain that
0LF
@L.
(Otr)
ÖLr
00
00
=
Vun
00
00
> 0.
SR/
SR
SR
C Implementation detail
In the experiments of our study, we used the VGG network as follows: 64C3-12868
where xCy represents the convolutional layer with r-output channels and y-stride, APr represents the average pooling with the kernel size 2, GAP represents the global average pooling, and FC represents the fully connected layer.
As for the time step, we set it to 6 if there is no mention. We used the stochastic gradient descent (SGD) as the optimizer with the batch size, epoch, initial learning rate for the cosine annealing, and momentum at
24

---

Published in Transactions on Machinc Learning Rescarch (06/2024)
Table 3: Gradient comparison SAF-E and OTTTo on CIFAR-10 and CIFAR-100.
Dataset
T Correlation Coefficient MAE (x10-))
CIFAR-10
0.9841000001
2.26+0.10
CIFAR-100
0.969=00001
12.4+0.40
Table 4: Gradient comparison SAF-F and OTTTA on CIFAR-10 and CIPAR-100.
Dataset
Correlation Coefficient
MAE (x10-9)
CIFAR-10
0.566::0.097
23.2+4440
CIFAR-10
32
0.557=00004
22.5+:4000
CIFAR-100
0.590++0.021
12.8
CIFAR-100
32
0.557:10.030
73.6+17.2
128, 300, 0.1, and 0.9. As for the loss function, we applied the combination of the cross-entropy (CE) and the mean-squared error (MSE) losses, i.e., L (1 a)CE + aMSE (where 0.05). In addition, set the leaky term as À 0.5 and the threshold as Vth = 1 and used the scaled weight standardization (sWS) 'Qiao et al. 2019). We applied derivative of sigmoid function as surrogate gradient:
oa' Ld
1 exp((Van (1)13))
(30)
a
B (1 + exp((Vin (1)
where 8 is hyperparameter and we set 3 = 4. Note that all settings, including the above, were the same as Xiao ct al. (022))
D Comparison of gradient
Tables 3 and show the correlation coefficients and mean absolute errors (MAE) of gradients of the SAF and OTTT. These values were computed in the input layer after training, where the gradient error between SAF and OTTT were the largest. As can be seen from the table, it is considered that SAF-E and OTTTo have the same gradients, while SAF-F and OTTTA have different (but close direction) gradients. These results clearly support our theory.
E Experimental Analysis of SAF-E with Feedback and FeedForward Connection
In this section, we confirm that SAF-E and OTTTo are equivalent even if there feedback feedforward conncction. First, compare the performance of SAF-E with OTTTo when both networks have foedback connection from the first layer to the N-th layer. The setup was the same as in the experiments of Xiao et al. (2022).
Table shows the accuracy and total firing rate when we set T = 6. As shown in this table, SAF-E and OTTTo with feedback connection are almost close. Also, the change due to inference with SNNs consisting of LIF neurons also almost negligible. Figure shows the accuracy, loss curve, and the firing rates of each layer. From these results, we confirmed experimentally that SAF-E and OTTTo with feedback connection are close, which consistent with the assertion in Corollary. 3
Next, we compare the performance of SAF-E with OTTTo when both networks have feedforward connec- tion. We used the RepVGG network (Ding et al.) 2021 The results are shown in Table 6 and Fig. 7 As
in the case of feedback connections, we can also see that SAF-E and OTTTo with a feedforward connection are close.
25

---

Published in Transactions on Machine Learning Rescarch (06/2024)
Table 5: Comparison of SAF-E and OTTTo with feedback connection CIFAR-10 (T 6). The values in parentheses are the changes in firing rate and accuracy due to inference by the SNN composed of LIF.
Method
Firing rate [%]
Accuracy [%]
OTTTo with feedback
0..74=0.34
93.23+0.28
SAF-E with feedback (ours)
14.22=0012 (1.78 x 10 10 93.20:10.18 (6.67 x 10
Table 6: Comparison of SAF-E and OTTTo with feedforward connection CIFAR-10 (T= The values in parentheses are the changes in firing rate and accuracy due to inference by the SNN composed of LIF.
Method
Firing rate [%]]
Accuracy [%]
OTTTo with feedforward
15. 05.03
93.53=0.15 93.53++0117 (0.004)
SAF-E with feedforward (ours) 14.72+00.04 (3.342x10-)
OTTT_O
SAF-E
SAF-E
14
Average Firing Rate
Accuracy
0.74
68-
OTTT_0
0.
SAF-E
1.0
352
250
300
300
1.50
250
300
0.
Epoch
Epoch
Layer Index
Figure 6: Accuracy and loss curves, and firing rates of each layer of SAF-E and OTTTo with feedback connection on CIFAR-10 (T = 6).
OTTT O SAF-E
OTTT O SAF-E
34-
Average Fining Rate
0.2
Accuracy
03
66-
OTTT O
SAF-E
0.5
353 Epoch
1.50 Epoch
200
Layer Index
Figure 7: Accuracy and loss curves, and firing rates of each layer of SAF-E and OTTTo with feedforward connection CIFAR-10 (T 6).
26


Reviews:
[{"review_id": "bxuAULlng2", "reviewer": "Reviewer_Jq86", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall I like this paper because it clearly provides the link with existing training algorithms and also shows decreased complexity. I don't have major criticisms, although I feel focusing on training non-recurrent spiking networks is still over-simplified.\n\nMinor:\n1. Please write the full name of SG at the first paragraph under \"Online Training Through Time\" at page 4.\n2. At page 4, I see there are both $a(t)$ and $\\hat{a}(t)$ differing in being normalized or not. Is it possible to use the same quantity?\n3. Eq. 8: is it missing a factor $\\lambda$?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a training algorithm for spiking neural networks and finds it can be equivalent to previous algorithms but reduce the memory requirement during training.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "My major suggestion for the paper is its writing. There are several places of the neural dynamics, e.g., Eqs. (1), (4), (10), (11), and others, and the differences among them are not easy to identify. I need to turn the pages back and forth to carefully compare each term between equations. I strongly suggest below each equation the author explains its similarities and differences with the standard equation which will greatly help readers to distinguish them. The same suggestion for the equations of the gradient.", "broader_impact_concerns": "This paper proposes a revised spiking network training algorithm with reduced memory requirement, which may help the practical applications of spiking networks.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "D9DezGUDdI", "reviewer": "Reviewer_1jyv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The problem of training spiking networks is important. The method proposed here seems to provide a quantifiable (though somewhat incremental) advantage.\n\n\nWeaknesses:\n- The paper is extremely unclear and difficult to follow. I am still not sure that my understanding of the method, or its supposed advantage, is correct.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Apparently the authors propose a new method for training spiking networks, which builds (and improves) upon previous methods.\n\nThe first few sections mostly describe existing approaches (\"Surrogate gradients\", \"spiking representation\", \"Online training through time\").\n\nIIUC the proposed method, SAF,  only carries the (decaying running average) firing a[t] across time steps. From this, we can reconstruct the accumulated membrane potential U[t], spiking s[t] and instantaneous membrane potential u[t]. The requirement for only one quantity to be propagated through time apparently reduces memory usage and compute time.\n\nThe authors apparently prove that this process is equivalent to previously mentioned methods, and that weights trained with SAF can be used unchanged in networks standard spiking LIF neurons. \n\nSimple experiments on CIFAR10 and CIFAR100, with SAF training and inference on standard LIF SNNs, suggest that the method is somewhat (linearly) faster and consumes less memory, for identical accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- The main problem is that methods are not adequeately described. For example, we see multiple mentions of \"surrogate gradient\" throughout the paper, but unless I missed it, it is never explained what \"surrogate gradients\" actually are. What is \"Surrogate Gradient\"? \n\n- p. 3, we read: \"approximating the gradient of non-differentiable activation functions\" - how is it approximated?\n\n- \"Spike Representation, which propagates information differently from the spike train\" : what does that mean? What does it do?\n\n- Section 3, equation 1. What is l?\n\n- Eq 3 is supposed to describe training through time, but all values have the same time index t? What is being backpropagated through time exactly?\n\n- Equation 3: $s$ is a binary variable. $ds/du$ is zero everywhere except u=Vthres where (as the authors point out) it is undefined. I have no idea what $ds^{i+1}/ds^i$ even means: how can you differentiate with regard to a binary variable? Please explain briefly how the gradients $ds^{i+1}/ds^i$ and $ds^{l+1}/du^{l+1} are approximated.\n\n\n- p. 6: again, please explain how exactly the \"SG\" approximation works.\n\n- Equation 6: \"it hold\" -> \"it holds\".", "broader_impact_concerns": "I do not see any broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "cTu1gf8AGH", "reviewer": "Reviewer_nWJ4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n- Streamlines previous OTTT method which was using both spike trains and accumulated currents\n\n## Weaknesses\n\n- Seemingly conflicting claims about the equivalence relations between SAF-F, OTTT_A, and Spike representation\n- The gradient computation apparently neglects dependencies on the previous time steps, without clarifying this approximation.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present a method for training spiking networks which has reduced computational complexity and memory cost on GPUs compared to a previous method, online training through time (OTTT). The main idea is to propagate only filtered synaptic currents instead of spike trains and membrane potentials. The authors confirm in experiments on CIFAR-10 and CIFAR-100 that their new method retains similar performance to OTTT while reducing training time and memory cost.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "## Main requested changes\n\n1. **Discrete jumps in filtered spike trains**  \nLinear filtering of the spike trains results in a quantity that still has discrete jumps, which is reflected in eq.4 in the Heaviside non-linearity in the update of $\\hat{a}$. As this dynamics also has no well-defined gradient, the authors use surrogate gradients. Could the authors explain in how far their method then differs and improves upon existing surrogate gradient methods (beyond OTTT)? \n  \n2. **Dependence of gradients on previous time-steps**  \nEven if the loss at time $t$ only depends on $\\hat{a}[t]$, these still depend on all previous time-steps $\\hat{a}[t'<t]$ which are a function of the weights. The gradients w.r.t. the weights thus require partial derivatives w.r.t. all $\\hat{a}[t'<t]$ as in BPTT. The OTTT and SAF approaches do not seem to take this dependency fully into account, even given the assumption which seems to be made implicitly in the manuscript that the label is constant over time. \n  \n3. **Equivalence of gradients in experiments**  \nWhile the experiments show that SAF-E/F and OTTT_O/A perform very similar, they still seem at odds with the theoretical claim that the gradients are exactly the same. If they are equivalent, it would not make sense to compare the two implementations in their performance, only in memory consumption, training time and overlap of their gradients. However the results do show minor differences in firing rates and accuracy, e.g. for CIFAR-100 in Table 1. Could the authors explain the origin of these differences and confirm numerically that their gradients are the same?\n  \n4. **Conflicting claims on relation between SAF-F, OTTT_A, Spike representation**  \nOn p.7, at the end of sect.4.2, it is stated that SAF-F and OTTT_A are equivalent. This makes sense if SAF-E and OTTT_O are equivalent, since in both methods the change seems to be to sum the gradients over time-steps. However, the authors also claim equivalence up to a factor $V_{th}$ between SAF-F and Spike representation. So does it simply mean that OTTT_A and Spike representation also have the same gradient directions? On the other hand, in the numerical results sect.5.2, the authors write that SAF-F and OTTT_A are not equivalent, contradicting their previous statement. \n\n## Additional minor changes\n\n5. **Unclear presentation of OTTT and Spike Representation rules**  \n  - maybe it should be $a[t]$ instead of $s[t]$ in the definition of the loss above eq.2 ? \n  - Where is the derivative w.r.t. the parameters $W$ on the right hand side of eq.3 ? \n  - Why does $\\frac{\\partial s^{i+1}}{\\partial s^{i}}$ not have the same differentiability issue as $\\frac{\\partial s^{i+1}}{\\partial u^{i}}$ ?\n  \n\n6. **Typos / small changes**  \n  - What is meant by adding \"the feedback connection\" and \"forward connections\" could be explained already before sect.4.3, e.g. in in sect.3\n  - Why use the ill-defined term \"essentially identical\" in Theorem 2 and Corollary 3 ?", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Two out of the three reviewers were satisfied with the proposed changes, and recommended acceptance. However, one reviewer felt that there were still some small, but critical items to address (and the AE agrees). These were:

- The authors should include the explanations from the author-reviewer discussion in the paper. In particular, they should include in Appendix B2 the explanation of the calculation of $da_{i+1}/da_i$, as shown in this comment: https://openreview.net/forum?id=RGQsUQDAd9&noteId=Q8x47YvxXR . The reviewer noted that while the convention that $dU_{i+1}/da_i = W$ makes sense, it is not obvious for many readers.

- In Appendix B2 the authors write "we approximate it with the SG." For the sake of reproducibility, they should explain what this means and show the equations that are used.

- When the authors write "these methods assume T -> +inf" - What is T?

Note, there is also two small typos:

- In section 2: "for small time steps" -> "for few time steps". 

- In the abstract: "...theoretical agreement with Spike Representation has not to be proven." should be "...theoretical agreement with Spike Representation has *yet* to be proven."

If the authors make these changes, along with the other proposed changes, then the paper would be ready for acceptance.
Decision: Accept (poster)