Original: Training Spiking Neural Networks with Event-driven Backpropagation
Revised: Training Spiking Neural Networks with Event-driven Backpropagation

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Spiking Neural networks (SNNs) represent and transmit information by spatiotemporal spike patterns, which bring two major advantages: biological plausibilityand suitability for ultralow-power neuromorphic implementation.
Revised: Spiking Neural networks (SNNs) represent and transmit information by spatiotemporal spike patterns, which bring two major advantages: biological plausibilityand suitability for ultralow-power neuromorphic implementation.

Original: Despite this, thebinary firing characteristic makes training SNNs more challenging.
Revised: Despite this, thebinary firing characteristic makes training SNNs more challenging.

Original: To learn theparameters of deep SNNs in an event-driven fashion as in inference of SNNs, backpropagation with respect to spike timing is proposed.
Revised: To learn theparameters of deep SNNs in an event-driven fashion as in inference of SNNs, backpropagation with respect to spike timing is proposed.

Original: Although this event-drivenlearning has the advantages of lower computational cost and memory occupation,the accuracy is far below the recurrent neural network-like learning approaches.
Revised: Although this event-drivenlearning has the advantages of lower computational cost and memory occupation,the accuracy is far below the recurrent neural network-like learning approaches.

Original: Inthis paper, we first analyze the commonly used temporal backpropagation training approach and prove that the sum of gradients remains unchanged betweenfully-connected and convolutional layers.
Revised: Inthis paper, we first analyze the commonly used temporal backpropagation training approach and prove that the sum of gradients remains unchanged betweenfully-connected and convolutional layers.

Original: Secondly, we show that the max poolinglayer meets the above invariance rule, while the average pooling layer does not,which will suffer the gradient vanishing problem but can be revised to meet therequirement.
Revised: Secondly, we show that the max poolinglayer meets the above invariance rule, while the average pooling layer does not,which will suffer the gradient vanishing problem but can be revised to meet therequirement.

Original: Thirdly, we point out the reverse gradient problem for time-basedgradients and propose a backward kernel that can solve this problem and keepthe property of the invariable sum of gradients.
Revised: Thirdly, we point out the reverse gradient problem for time-basedgradients and propose a backward kernel that can solve this problem and keepthe property of the invariable sum of gradients.

Original: The experimental results showthat the proposed approach achieves the state-of-the-art performance on CIFAR10among time-based training methods.
Revised: The experimental results showthat the proposed approach achieves the state-of-the-art performance on CIFAR10among time-based training methods.

Original: Also, this is the first time that the time-basedbackpropagation approach successfully trains SNN on the CIFAR100 dataset.
Revised: Also, this is the first time that the time-basedbackpropagation approach successfully trains SNN on the CIFAR100 dataset.

Original:  Introduction
Revised:  Introduction

Original: Motivated by the principles of brain computing, Spiking Neural Networks (SNNs) are consideredas the third generation of neural networks [1, 2].
Revised: Motivated by the principles of brain computing, Spiking Neural Networks (SNNs) are consideredas the third generation of neural networks [1, 2].

Original: SNNs are developed to work in power-criticalscenarios, such as edge computing.
Revised: SNNs are developed to work in power-criticalscenarios, such as edge computing.

Original: When run on dedicated neuromorphic chips, they can accomplishthe tasks [3, 4, 5, 6, 7, 8, 9, 10] with ultra-low power consumption [11, 12, 13, 14, 15, 16].
Revised: When run on dedicated neuromorphic chips, they can accomplishthe tasks [3, 4, 5, 6, 7, 8, 9, 10] with ultra-low power consumption [11, 12, 13, 14, 15, 16].

Original: In contrast,the last generation of neural networks – Artificial Neural Networks (ANNs) [17], generally require alarge amount of computation resource (e.g., GPUs).
Revised: In contrast,the last generation of neural networks – Artificial Neural Networks (ANNs) [17], generally require alarge amount of computation resource (e.g., GPUs).

Original: This advantage of SNNs on power consumptionlargely relies on efficient event-based computations [18, 19].
Revised: This advantage of SNNs on power consumptionlargely relies on efficient event-based computations [18, 19].

Original: Another advantage of SNNs originatesfrom their biological reality (compared to ANNs).
Revised: Another advantage of SNNs originatesfrom their biological reality (compared to ANNs).

Original: The similarity between SNNs and biological brainsprovides an excellent opportunity to study how the brain computes at the neuronal circuit level [20].
Revised: The similarity between SNNs and biological brainsprovides an excellent opportunity to study how the brain computes at the neuronal circuit level [20].

Original: Compared with artificial neural networks, developing supervised learning algorithms for spikingneural networks requires more effort.
Revised: Compared with artificial neural networks, developing supervised learning algorithms for spikingneural networks requires more effort.

Original: The main challenge for training SNNs comes from the binarynature of spikes and the non-differentiability of the membrane potential at spike time.
Revised: The main challenge for training SNNs comes from the binarynature of spikes and the non-differentiability of the membrane potential at spike time.

Original: This difficulty in training impedes the performance of SNNs in pattern classification tasks compared to their ANNcounterparts.
Revised: This difficulty in training impedes the performance of SNNs in pattern classification tasks compared to their ANNcounterparts.

Original: Existing supervised learning methods of SNNs can be grouped into two categories:
Revised: Existing supervised learning methods of SNNs can be grouped into two categories:

Original: The first category consists of recurrent neural network (RNN)-like learning algorithms.
Revised: The first category consists of recurrent neural network (RNN)-like learning algorithms.

Original: Thesealgorithms treat spiking neural networks as binary-output recurrent neural networks and handle thediscontinuities of membrane potential at spike times with continuous surrogate derivatives [21].
Revised: Thesealgorithms treat spiking neural networks as binary-output recurrent neural networks and handle thediscontinuities of membrane potential at spike times with continuous surrogate derivatives [21].

Original: Theytypically train deep SNNs with surrogate gradients based on the idea of backpropagation through time(BPTT)
Revised: Theytypically train deep SNNs with surrogate gradients based on the idea of backpropagation through time(BPTT)

Original: algorithm
Revised: algorithm

Original: While competitive accuracies arereported on the MNIST, CIFAR-10, and even ImageNet datasets [34, 35, 36], the gradient informationis propagated each time step, whether or not a spike is emitted (as shown in Fig. 1).
Revised: While competitive accuracies arereported on the MNIST, CIFAR-10, and even ImageNet datasets [34, 35, 36], the gradient informationis propagated each time step, whether or not a spike is emitted (as shown in Fig. 1).

Original: Therefore,these approaches do not follow the event-driven nature of spiking neural networks, which lose theasynchronous characteristic of SNNs and consume much power when trained on neuromorphichardware.
Revised: Therefore,these approaches do not follow the event-driven nature of spiking neural networks, which lose theasynchronous characteristic of SNNs and consume much power when trained on neuromorphichardware.

Original: The second category is event-driven algorithms, which propagate gradient information through spikes.
Revised: The second category is event-driven algorithms, which propagate gradient information through spikes.

Original: Precise spiking timing acts an important role in this situation, and they are extensively used in suchalgorithms [37, 38, 39, 40, 41, 42, 43, 44].
Revised: Precise spiking timing acts an important role in this situation, and they are extensively used in suchalgorithms [37, 38, 39, 40, 41, 42, 43, 44].

Original: Classical examples include SpikeProp [37] and its variants [38, 45, 46].
Revised: Classical examples include SpikeProp [37] and its variants [38, 45, 46].

Original: These algorithms approximate the derivative of spike timing to membrane potentialas the negative inverse of the time derivative of membrane potential function.
Revised: These algorithms approximate the derivative of spike timing to membrane potentialas the negative inverse of the time derivative of membrane potential function.

Original: This approximationis actually mathematically correct without preconditions [47].
Revised: This approximationis actually mathematically correct without preconditions [47].

Original: Some other works apply non-leakyintegrate-and-fire neurons to stabilize the training process [40, 48, 43].
Revised: Some other works apply non-leakyintegrate-and-fire neurons to stabilize the training process [40, 48, 43].

Original: Most of these works restricteach neuron to fire at most once, which inspires [49] to take the spike time as the state of a neuron, andmodel the relation of neurons by this spike time.
Revised: Most of these works restricteach neuron to fire at most once, which inspires [49] to take the spike time as the state of a neuron, andmodel the relation of neurons by this spike time.

Original: As a result, the SNN is trained similarly to an ANN.
Revised: As a result, the SNN is trained similarly to an ANN.

Original: Among the methods allowing multiple spikes trained in an event-driven fashion, the state-of-the-artmodel is TSSL-BP [44].
Revised: Among the methods allowing multiple spikes trained in an event-driven fashion, the state-of-the-artmodel is TSSL-BP [44].

Original: However, they use RNN-like surrogate gradients (a sigmoid function) toassist training.
Revised: However, they use RNN-like surrogate gradients (a sigmoid function) toassist training.

Original: It is still challenging to train SNNs in a pure event-driven fashion.
Revised: It is still challenging to train SNNs in a pure event-driven fashion.

Original: In this work, we develop a novel event-driven learning algorithm that can train high-performancedeep SNNs.
Revised: In this work, we develop a novel event-driven learning algorithm that can train high-performancedeep SNNs.

Original: The main contributions of our work are as follows:
Revised: The main contributions of our work are as follows:

Original: We analyze the commonly used SNN temporal backpropagation training approach and prove
Revised: We prove that the typical SNN temporal backpropagation training approach assigns the

Original: that it assigns gradients from the next layer to the last layer in layers with weights, which
Revised: gradient of an output spike of a neuron to the input spikes generating it.

Original: keeps the sum of gradients  unchanged between layers.2.
Revised: After summing thisassignment rule altogether, we find that the sum of gradients is kept unchanged betweenlayers.2.

Original: For layers without weights, we analyze the case of  pooling layers and find that averagepooling does not satisfy the above rule, whose backpropagation method can be revised tomeet the requirement.
Revised: We analyze the case of the pooling layer (which does not have weights) and find that averagepooling does not keep the gradient sum unchanged, but we can modify its backward formulas to meet the requirement.

Original: Meanwhile, the max-pooling layer satisfies the rule.
Revised: Meanwhile, the max-pooling layer satisfies the rule initially.

Original: We point out the reverse gradient problem in event-driven learning that the direction of
Revised: We point out the reverse gradient problem in event-driven learning that the direction of

Original:  temporal gradient will be reversed during propagation when thespike kernel  is decreasing,
Revised: the temporal gradient is reversed during backpropagation when the kernel function of an input spike is decreasing.

Original: and propose a backward kernel  that can solve thisproblem while keeping the property of
Revised: Then we propose a backward kernel function that addresses this

Original:  theinvariable sum of gradients.
Revised: problem while keeping the sum of gradients unchanged between layers. The adjusted average pooling layer and the non-decreasing backward kernel enhances the performance of our model as well as the convergence speed.

Original: To our best knowledge, our proposed approach achievesthe state-of-the-art performance on 
Revised: To our best knowledge, ourproposed approach achieves state-of-the-art performance on CIFAR10 among event-driven

Original: CIFAR10 among time-based training methods  for SNNs.
Revised: training methods (with temporal gradients) for SNNs.

Original: In the meantime, our method is thefirst time-based backpropagation approach that successfully trains SNN on the CIFAR dataset.
Revised: Meanwhile, our method is the firstevent-driven backpropagation approach that successfully trains SNN on the larger-scaleCIFAR100 dataset.

Original:  Backgrounds and Related Work
Revised:  Backgrounds and Related Work

Original: The gradient-based learning of spiking neural networks contains two stages: the forward (inference)and the backward (learning) stages.
Revised: The gradient-based learning of spiking neural networks contains two stages: the forward (inference)and the backward (learning) stages.

Original: In the forward stage, Leaky Integrate-and-Fire (LIF) neuronsare most commonly used [26, 22, 32, 44], while other types of neurons are also applicable [37, 40].
Revised: In the forward stage, Leaky Integrate-and-Fire (LIF) neuronsare most commonly used [26, 22, 32, 44], while other types of neurons are also applicable [37, 40].

Original: Typically, these neuron models can be changed to the form of the Spike Response Model (SRM) [50,51, 42], which is easily represented in an event-driven fashion.
Revised: Typically, these neuron models can be changed to the form of the Spike Response Model (SRM) [50,51, 42], which is easily represented in an event-driven fashion.

Original: In the backward stage, the situation is more complex.

Original: Here, we classify existing approaches fromtwo dimensions: whether non-spike information is needed in discrete time steps (RNN-like) or not

Original: Gradient direction 𝑡 𝑢
Revised: Gradient direction 𝑡 𝑢

Original: Output
Revised: Output

Original: Input 𝑡 𝑢 𝑢 𝑗
Revised: Input 𝑡 𝑢 𝑢 𝑗

Original: 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗
Revised: 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗

Original: Spike
Revised: Spike

Original: No spike
Revised: No spike

Original: Gradient direction a b c d 𝑡 𝑡 𝑡𝑡 0 +1 𝑡 0 +2 𝑡 0 + 𝑡𝑡 0 +1 𝑡 0 +2 𝑡 0 +
Revised: Gradient direction a b c d 𝑡 𝑡 𝑡𝑡 0 +1 𝑡 0 +2 𝑡 0 + 𝑡𝑡 0 +1 𝑡 0 +2 𝑡 0 +

Original: Output
Revised: Output

Original: Input
Revised: Input

Original: Spike train
Revised: Spike train

Original: (event-driven) and whether the gradient represents spike scale (activation-based) or spike timing(time-based).
Revised: In the backward stage, the situation is more complex. Here, we classify existing approaches fromtwo dimensions: whether non-spike information is needed in discrete time steps (RNN-like) or not(event-driven) and whether the gradient represents spike scale (activation-based) or spike timing(time-based).

Original: Event-driven learning v.s. RNN-like learning: In both forward and backward computation ofevent-driven learning, information is only carried by spikes in SNNs.
Revised: Event-driven learning v.s. RNN-like learning: In both forward and backward computation ofevent-driven learning, information is only carried by spikes in SNNs.

Original: Specifically, in backwardcomputation, gradient information is propagated through spikes [37, 38, 46, 40] (shown in Fig.
Revised: Specifically, in backwardcomputation, gradient information is propagated through spikes [37, 38, 46, 40] (shown in Fig.

Original: 1a-b).
Revised: 1a-b).

Original: On the other side, in RNN-like learning, information is not only carried by spikes in backwardcomputation.
Revised: On the other side, in RNN-like learning, information is not only carried by spikes in backwardcomputation.

Original: Especially, gradient information can be propagated through a neuron that does not emita spike in backward computation (shown in Fig.
Revised: Especially, gradient information can be propagated through a neuron that does not emita spike in backward computation (shown in Fig.

Original: 1c-d).
Revised: 1c-d).

Original: This gradient propagation is achieved by asurrogate function [16, 21, 22, 28, 25], which is a function of the membrane potential at the currenttime step u t , and the firing threshold θ .
Revised: This gradient propagation is achieved by asurrogate function [16, 21, 22, 28, 25], which is a function of the membrane potential at the currenttime step u t , and the firing threshold θ .

Original: Time-based gradient v.s. activation-based gradient
Revised: Time-based gradient v.s. activation-based gradient

Original: : Time-based gradients represent the (reverse)direction that the timing of a spike should move, that is, to move leftward or rightward on the timeaxis [37].
Revised: : Time-based gradients represent the (reverse)direction that the timing of a spike should move, that is, to move leftward or rightward on the timeaxis [37].

Original: In backward propagation, the derivative of the firing time of a spike to the correspondingmembrane potential ∂t∂u is often approximated as − 1 ∂u∂t [37, 38], denoting how the change of membranepotential will change the spike firing time (Fig. 1b).
Revised: In backward propagation, the derivative of the firing time of a spike to the correspondingmembrane potential ∂t∂u is often approximated as − 1 ∂u∂t [37, 38], denoting how the change of membranepotential will change the spike firing time (Fig. 1b).

Original: On the other side, activation-based approachesreplace the Heaviside neuron activation function Θ( · ) ( s t =
Revised: On the other side, activation-based approachesreplace the Heaviside neuron activation function Θ( · ) ( s t =

Original: Θ( u t − θ ) ) in forward propagationwith derivable functions σ ( · ) in backward propagation, whether there are spikes in the current timestep [22, 32, 36, 26].
Revised: Θ( u t − θ ) ) in forward propagationwith derivable functions σ ( · ) in backward propagation, whether there are spikes in the current timestep [22, 32, 36, 26].

Original: Therefore, activation-based approaches essentially regard SNNs as binary
Revised: Therefore, activation-based approaches essentially regard SNNs as binary

Original: RNNs and train them with approximated gradients, where the gradients indicate whether the valuesin the network (including the binary spikes) should be larger or smaller (Fig. 1d).
Revised: RNNs and train them with approximated gradients, where the gradients indicate whether the valuesin the network (including the binary spikes) should be larger or smaller (Fig. 1d).

Original: As a result, time-based gradients are event-driven by nature, since the temporal gradient could only becarried by spikes.
Revised: As a result, time-based gradients are event-driven by nature, since the temporal gradient could only becarried by spikes.

Original: Meanwhile, activation-based gradients are more suitable for the RNN-like trainingscheme since the diversity of surrogate gradients largely relies on the fact that u t ̸
Revised: Meanwhile, activation-based gradients are more suitable for the RNN-like trainingscheme since the diversity of surrogate gradients largely relies on the fact that u t ̸

Original: = θ in discrete timesteps [21], which no longer holds in continuous time simulation.
Revised: = θ in discrete timesteps [21], which no longer holds in continuous time simulation.

Original: If we want to apply activation-basedgradients to event-driven learning, there should only be one value ∂s∂u when the membrane potentialreaches the threshold.
Revised: If we want to apply activation-basedgradients to event-driven learning, there should only be one value ∂s∂u when the membrane potentialreaches the threshold.

Original: Tab. 1 lists whether a gradient type can be used in a learning fashion.
Revised: Tab. 1 lists whether a gradient type can be used in a learning fashion.

Original: It should be noticed thatalthough activation-based gradient is more suitable for RNN-like learning, it is still able to be usedfor event-driven learning.
Revised: It should be noticed thatalthough activation-based gradient is more suitable for RNN-like learning, it is still able to be usedfor event-driven learning.

Original: [Table]
Revised: [Table]

Original: [Table]

Original:  Methods
Revised:  Methods

Original: 3.1 Forward Formulas
Revised: 3.1 Forward Formulas

Original: We use the spike response model [1] for neurons in the network.
Revised: We use the spike response model [1] for neurons in the network.

Original: The forward propagation in thenetwork can be described as follows:
Revised: The forward propagation in thenetwork can be described as follows:

Original: [Equation]
Revised: [Equation]

Original: Here u ( l ) i ( t ) denotes the membrane potential of neuron i in layer l at time t , w ( l ) ij denotes the weightbetween neuron j in layer l − 1 and neuron i in layer l .
Revised: Here u ( l ) i ( t ) denotes the membrane potential of neuron i in layer l at time t , w ( l ) ij denotes the weightbetween neuron j in layer l − 1 and neuron i in layer l .

Original: t ( l ) i,last is the time of last spike of neuron i inlayer l , and s ( l ) i ( t ) represents the spike emitted from neuron i at time t .
Revised: t ( l ) i,last is the time of last spike of neuron i inlayer l , and s ( l ) i ( t ) represents the spike emitted from neuron i at time t .

Original: The function δ ( · ) is the Dirac
Revised: The function δ ( · ) is the Dirac

Original: Delta function and θ is the firing threshold.
Revised: Delta function and θ is the firing threshold.

Original: The spike response kernel ϵ ( t ) can be described by
Revised: The spike response kernel ϵ ( t ) can be described by

Original: [Equation]
Revised: [Equation]

Original: where τ m and τ s are the membrane time constant and the synapse time constant respectively.
Revised: where τ m and τ s are the membrane time constant and the synapse time constant respectively.

Original: Noticethat we do not use reset kernels as in previous works [46, 44].
Revised: Noticethat we do not use reset kernels as in previous works [46, 44].

Original: Instead, we eliminate the influence ofinput spikes prior to the last output spike on membrane potentials.
Revised: Instead, we eliminate the influence ofinput spikes prior to the last output spike on membrane potentials.

Original: 3.2 Rethinking the Classical Time-based Backward Propagation Formula
Revised: 3.2 Rethinking the Classical Time-based Backward Propagation Formula

Original: In this subsection, we analyze the classical time-based backpropagation formula in SNNs.
Revised: In this subsection, we analyze the classical time-based backpropagation formula in SNNs.

Original: Wefirst theoretically prove that it is essentially a weighted gradient assignment mechanism amonglayers.
Revised: Wefirst theoretically prove that it is essentially a weighted gradient assignment mechanism amonglayers.

Original: Then we check the pooling layer and show that the average pooling should be adjusted inbackpropagation to satisfy the gradient assignment mechanism, while the max pooling naturallysatisfies this mechanism.
Revised: Then we check the pooling layer and show that the average pooling should be adjusted inbackpropagation to satisfy the gradient assignment mechanism, while the max pooling naturallysatisfies this mechanism.

Original: Invariant sum of gradients among layers with weights.
Revised: Invariant sum of gradients among layers with weights.

Original: The most commonly used time-basedgradient backpropagation method origins from [37].
Revised: The most commonly used time-basedgradient backpropagation method origins from [37].

Original: The two key approximations are as follows:
Revised: The two key approximations are as follows:

Original: [Equation]
Revised: [Equation]

Original: where t k ( s ( l ) i ) denotes the firing time t k of neuron i in layer l , t k,last ( s ( l ) i ) is the firing time of the lastspike emitted by neuron i before time t k .
Revised: where t k ( s ( l ) i ) denotes the firing time t k of neuron i in layer l , t k,last ( s ( l ) i ) is the firing time of the lastspike emitted by neuron i before time t k .

Original: ∂t k ( s ( l ) i ) ∂ · means the influence of changing other variableson the timing of a spike, and
Revised: ∂t k ( s ( l ) i ) ∂ · means the influence of changing other variableson the timing of a spike, and

Original: [Equation]
Revised: [Equation]

Original: is the influence of changing spike timing on that variable.
Revised: is the influence of changing spike timing on that variable.

Original: Combining Eqs.
Revised: Combining Eqs.

Original: 4-5 and the forward formulas, we can get an invariant equality:
Revised: 4-5 and the forward formulas, we can get an invariant equality:

Original: [Equation]
Revised: [Equation]

Original: The proof is provided in Appendix.
Revised: The proof is provided in Appendix.

Original: Eq.
Revised: Eq.

Original: 6 implies the fact that the reference time ( t = 0 ) ismeaningless, and only relative spike times matter.
Revised: 6 implies the fact that the reference time ( t = 0 ) ismeaningless, and only relative spike times matter.

Original: If we increase all the spike times in layer l − 1 by1 unit along the time axis, then all the spike times in layer l are also increased by 1 unit along thetime axis.
Revised: If we increase all the spike times in layer l − 1 by1 unit along the time axis, then all the spike times in layer l are also increased by 1 unit along thetime axis.

Original: Denote the loss function as L , then the gradient of L with respect to t m ( s ( l − 1) j ) is:
Revised: Denote the loss function as L , then the gradient of L with respect to t m ( s ( l − 1) j ) is:

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: Output
Revised: Output

Original: Input = + + = + +
Revised: Input = + + = + +

Original: Loss
Revised: Loss

Original: [Figure]
Revised: [Figure]

Original: 𝒍−
Revised: 𝒍−

Original: [Figure]
Revised: [Figure]

Original: 𝒍 where t m,next ( s ( l − 1) j ) denotes the firing time of the next spike emitted by neuron j after timet m .
Revised: 𝒍 where t m,next ( s ( l − 1) j ) denotes the firing time of the next spike emitted by neuron j after timet m .

Original: Therefore, we actually decompose the gradient ∂L/∂t k ( s ( l ) i ) from layer l into (part of) aset of gradients ∂L/∂t m ( s ( l − 1) j ) of the last layer l − 1 , and keep their sum unchanged.
Revised: Therefore, we actually decompose the gradient ∂L/∂t k ( s ( l ) i ) from layer l into (part of) aset of gradients ∂L/∂t m ( s ( l − 1) j ) of the last layer l − 1 , and keep their sum unchanged.

Original: In otherwords, we assign the weighted sum ∂L/∂t k ( s ( l ) i ) by weights ∂t k ( s ( l ) i ) /∂t m ( s ( l − 1) j ) to the gradients∂L/∂t m ( s ( l − 1) j ) in the last layer, as shown in Fig. 2.
Revised: In otherwords, we assign the weighted sum ∂L/∂t k ( s ( l ) i ) by weights ∂t k ( s ( l ) i ) /∂t m ( s ( l − 1) j ) to the gradients∂L/∂t m ( s ( l − 1) j ) in the last layer, as shown in Fig. 2.

Original: If we sum all the gradients together, we can get another invariant in this backpropagation rule:(cid:88)
Revised: If we sum all the gradients together, we can get another invariant in this backpropagation rule:(cid:88)

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which means the sum of gradients (cid:80) i (cid:80) t k ∂L ∂t k ( s ( l ) i ) never changes between layers under this rule.
Revised: which means the sum of gradients (cid:80) i (cid:80) t k ∂L ∂t k ( s ( l ) i ) never changes between layers under this rule.

Original: Gradient sum invariance for pooling layers.
Revised: Gradient sum invariance for pooling layers.

Original: The above function determines the gradient propagation in fully-connected and convolution layers.
Revised: The above function determines the gradient propagation in fully-connected and convolution layers.

Original: The case for pooling layers is illustrated in Fig.
Revised: The case for pooling layers is illustrated in Fig.

Original: [Figure]
Revised: [Figure]

Original: In average pooling with kernel size k , the gradient of one spike (at time t ) in layer l is averagelypropagated to k × k neurons in layer l − 1 connected to it.
Revised: In average pooling with kernel size k , the gradient of one spike (at time t ) in layer l is averagelypropagated to k × k neurons in layer l − 1 connected to it.

Original: Some of these k × k neurons may not emitspikes at time t .
Revised: Some of these k × k neurons may not emitspikes at time t .

Original: However, the gradients are also propagated to those neurons, which cannot furtherpropagate the gradients to the previous layers.
Revised: However, the gradients are also propagated to those neurons, which cannot furtherpropagate the gradients to the previous layers.

Original: For instance, the two white squares (neurons) in layerl − 1 in Fig.
Revised: For instance, the two white squares (neurons) in layerl − 1 in Fig.

Original: 3 receive gradients, but they will not further propagate the gradients to layer l − 2 .
Revised: 3 receive gradients, but they will not further propagate the gradients to layer l − 2 .

Original: Thus,a part of the gradients is lost in the backpropagation of the average pooling layer, which might causethe gradient vanishing problem.
Revised: Thus,a part of the gradients is lost in the backpropagation of the average pooling layer, which might causethe gradient vanishing problem.

Original: Meanwhile, the sum of gradients carried by the spikes is not keptamong layers in this case.
Revised: Meanwhile, the sum of gradients carried by the spikes is not keptamong layers in this case.

Original: We can adjust the backpropagation stage in average pooling to satisfythe gradient sum invariance requirement by increasing the multiplier from 1 /k 2 to 1 /n spike , wheren spike is the number of spikes emitted by the k × k neurons in layer l − 1 at the current time step.
Revised: We can adjust the backpropagation stage in average pooling to satisfythe gradient sum invariance requirement by increasing the multiplier from 1 /k 2 to 1 /n spike , wheren spike is the number of spikes emitted by the k × k neurons in layer l − 1 at the current time step.

Original: On the contrary, in max pooling, the gradient of a spike in layer l is entirely propagated to one of thespikes emitted by its connected neurons in layer l − 1 .
Revised: On the contrary, in max pooling, the gradient of a spike in layer l is entirely propagated to one of thespikes emitted by its connected neurons in layer l − 1 .

Original: This maintains the property of the invariablesum of gradients.
Revised: This maintains the property of the invariablesum of gradients.

Original: It should be noticed that, although max pooling is different from adjusted average pooling in discretesimulation, they are almost surely the same in the continuous simulation since (almost surely) no twospikes emit at exactly the same time in this case.
Revised: It should be noticed that, although max pooling is different from adjusted average pooling in discretesimulation, they are almost surely the same in the continuous simulation since (almost surely) no twospikes emit at exactly the same time in this case.

Original: 3.3 Deficiencies of the Typical Time-based Propagation Formula and a New Approach
Revised: 3.3 Deficiencies of the Typical Time-based Propagation Formula and a New Approach

Original: 𝑡𝑡 𝑘𝑘 𝑡𝑡 𝑘𝑘′ 𝑡𝑡 𝑚𝑚 + 𝛥𝛥𝑡𝑡 𝑡𝑡 𝑚𝑚 𝜖𝜖 ∗ 𝑠𝑠 𝑡𝑡
Revised: 𝑡𝑡 𝑘𝑘 𝑡𝑡 𝑘𝑘′ 𝑡𝑡 𝑚𝑚 + 𝛥𝛥𝑡𝑡 𝑡𝑡 𝑚𝑚 𝜖𝜖 ∗ 𝑠𝑠 𝑡𝑡

Original: Input
Revised: Input

Original: Output
Revised: Output

Original: In this section, we first point out the reverse gradient problem in event-driven learning.
Revised: In this section, we first point out the reverse gradient problem in event-driven learning.

Original: Specifically, the gradient direction for spike timinggets wrong when the spike response kernel isdecreasing.
Revised: Specifically, the gradient direction for spike timinggets wrong when the spike response kernel isdecreasing.

Original: Then we propose a backward kernelthat can not only solve the reverse gradient problem but also keep the property of the invariablesum of gradients.
Revised: Then we propose a backward kernelthat can not only solve the reverse gradient problem but also keep the property of the invariablesum of gradients.

Original: The reverse gradient problem.
Revised: The reverse gradient problem.

Original: Fig.
Revised: Fig.

Original: 4 illustrates the membrane potential response of a neuron (with index i ) to one of its input spikes (witha positive weight) from presynaptic neuron j ,here we use the standard double-exponentialspike response kernel (Eq. 3).
Revised: 4 illustrates the membrane potential response of a neuron (with index i ) to one of its input spikes (witha positive weight) from presynaptic neuron j ,here we use the standard double-exponentialspike response kernel (Eq. 3).

Original: Notice that this isnot the whole membrane potential of the neuroni as it also receives the inputs from other presynaptic neurons.
Revised: Notice that this isnot the whole membrane potential of the neuroni as it also receives the inputs from other presynaptic neurons.

Original: We consider two spike times ( t kand t ′ k ) of neuron i and one spike time t m of presynaptic neuron j .
Revised: We consider two spike times ( t kand t ′ k ) of neuron i and one spike time t m of presynaptic neuron j .

Original: If neuron i spikes at time t k ,moving the presynaptic spike from t m to t m + ∆ t will decrease membrane potential at time t k ,which means postponing the spike at time t k ( t m ↑⇒ u
Revised: If neuron i spikes at time t k ,moving the presynaptic spike from t m to t m + ∆ t will decrease membrane potential at time t k ,which means postponing the spike at time t k ( t m ↑⇒ u

Original: [ t k ] ↓⇒ t k ↑ ).
Revised: [ t k ] ↓⇒ t k ↑ ).

Original: This result shows that if thepresynaptic neuron fires earlier, the postsynaptic neuron will fire earlier.
Revised: This result shows that if thepresynaptic neuron fires earlier, the postsynaptic neuron will fire earlier.

Original: Oppositely, if the neuron spikes at t ′ k , moving t m to t m + ∆ t will cause a increase of u [ t ′ k ] , whichfurther moves t ′ k leftward ( t m ↑⇒ u
Revised: Oppositely, if the neuron spikes at t ′ k , moving t m to t m + ∆ t will cause a increase of u [ t ′ k ] , whichfurther moves t ′ k leftward ( t m ↑⇒ u

Original: [ t ′ k ] ↑⇒ t ′ k ↓ ).
Revised: [ t ′ k ] ↑⇒ t ′ k ↓ ).

Original: As a result, if we want to move the output spikeat t ′ k leftward, we should move the input spike at t m rightward, which reverses the direction .
Revised: As a result, if we want to move the output spikeat t ′ k leftward, we should move the input spike at t m rightward, which reverses the direction .

Original: Thismight cause a problem: When we want to move t ′
Revised: Thismight cause a problem: When we want to move t ′

Original: k leftward, we want the neuron to emit morespikes .
Revised: k leftward, we want the neuron to emit morespikes .

Original: However, in gradient backpropagation, it moves t m rightward (assume weight w > 0 ), whichmay cause the neuron in the last layer to spike fewer, further causing neurons in the current layerto spike fewer .
Revised: However, in gradient backpropagation, it moves t m rightward (assume weight w > 0 ), whichmay cause the neuron in the last layer to spike fewer, further causing neurons in the current layerto spike fewer .

Original: More formally, we assume the neuron i receives a input spike at time t m from presynaptic neuron jwith synaptic weight w ij , then the membrane potential of neuron i at time t k is:
Revised: More formally, we assume the neuron i receives a input spike at time t m from presynaptic neuron jwith synaptic weight w ij , then the membrane potential of neuron i at time t k is:

Original: [Equation]
Revised: [Equation]

Original: where ϵ ( t ) denotes the spike response kernel (Eq. 3).
Revised: where ϵ ( t ) denotes the spike response kernel (Eq. 3).

Original: C denotes the influence of other spikes, whichis not in our concern here.
Revised: C denotes the influence of other spikes, whichis not in our concern here.

Original: In backward pass, according to Eqs. (4)-(5), we have
Revised: In backward pass, according to Eqs. (4)-(5), we have

Original: [Equation]
Revised: [Equation]

Original: Note that when a spike is emitted by neuron i at time t k , the slope of u i ( t ) > 0 at time t k , whichmeans − [Equation] i [Equation]
Revised: Note that when a spike is emitted by neuron i at time t k , the slope of u i ( t ) > 0 at time t k , whichmeans − 1 du i ( t k ) /dt has a negative sign.

Original: t k [Equation] /dt has a negative sign. Considering ∂ϵ ( t k − t m ) ∂t m = − dϵ ( τ ) dτ , where τ = t k − t m , we get:sign 
Revised: Considering ∂ϵ ( t k − t m ) ∂t m = − dϵ ( τ ) dτ , where τ = t k − t m , we get:sign (cid:32)

Original: [Equation]
Revised: [Equation]

Original: When sign (cid:0) dϵ ( τ ) /dτ (cid:1) = − 1 , which is the part of the spike response kernel that decreases (see thecase at t ′ k in Fig. 4), the gradient direction of t m can be classified into two cases: When w ij > 0 ,sign (cid:16) ∂L
Revised: When sign (cid:0) dϵ ( τ ) /dτ (cid:1) = − 1 , which is the part of the spike response kernel that decreases (see thecase at t ′ k in Fig. 4), the gradient direction of t m can be classified into two cases: When w ij > 0 ,sign (cid:16) ∂L

Original: ∂t m ( s j ) (cid:17)
Revised: ∂t m ( s j ) (cid:17)

Original: = − sign (cid:16) ∂L ∂t k ( s i ) (cid:17) , which means the gradient direction is reversed.
Revised: = − sign (cid:16) ∂L ∂t k ( s i ) (cid:17) , which means the gradient direction is reversed.

Original: When w ij < 0 ,sign (cid:16) ∂L ∂t m ( s j ) (cid:17) = sign (cid:16) ∂L ∂t k ( s i ) (cid:17) , which means the gradient direction is kept.
Revised: When w ij < 0 ,sign (cid:16) ∂L ∂t m ( s j ) (cid:17) = sign (cid:16) ∂L ∂t k ( s i ) (cid:17) , which means the gradient direction is kept.

Original: In both cases, the sign of the gradient gets wrong in propagation between layers.
Revised: In both cases, the sign of the gradient gets wrong in propagation between layers.

Original: Thus, the commonlyused double-exponential spike response kernel is incompatible with the time-based gradient inevent-driven learning.
Revised: Thus, the commonlyused double-exponential spike response kernel is incompatible with the time-based gradient inevent-driven learning.

Original: A smoother gradient assigning approach.
Revised: A smoother gradient assigning approach.

Original: Inspired by the above gradient inconsistency as wellas the invariance of gradient sum, we propose a new gradient backpropagation approach here.
Revised: Inspired by the above gradient inconsistency as wellas the invariance of gradient sum, we propose a new gradient backpropagation approach here.

Original: Specifically, we replace the function ∂ϵ ( t k − t m ) ∂t m in Eqs.
Revised: Specifically, we replace the function ∂ϵ ( t k − t m ) ∂t m in Eqs.

Original: (4) and (5) with a new function h
Revised: (4) and (5) with a new function h

Original: ( t k − t m ) .
Revised: ( t k − t m ) .

Original: Therefore, the backpropagation formula between layers turns into:
Revised: Therefore, the backpropagation formula between layers turns into:

Original: [Equation]
Revised: [Equation]

Original: otherwise .
Revised: otherwise .

Original: To guarantee that the gradients are not reversed between layers, we should expect h ( t ) > 0 alwayshold
Revised: To guarantee that the gradients are not reversed between layers, we should expect h ( t ) > 0 alwayshold

Original: when t > 0 .
Revised: when t > 0 .

Original: Besides, it can be see from Eq.
Revised: Besides, it can be see from Eq.

Original: 12 that ∂t k
Revised: 12 that ∂t k

Original: ( s i ) ∂t m ( s j ) will not change if we multiply h ( t )by an arbitrary constant, so we do not need to care about the scale of h ( t ) .
Revised: ( s i ) ∂t m ( s j ) will not change if we multiply h ( t )by an arbitrary constant, so we do not need to care about the scale of h ( t ) .

Original: Meanwhile, the propertyof invariable sum of gradients is kept after this replacement.
Revised: Meanwhile, the propertyof invariable sum of gradients is kept after this replacement.

Original: Therefore, we choose h ( t ) =
Revised: Therefore, we choose h ( t ) =

Original: e − t τgrad to simplify the calculation, where τ grad is a tunable parameter.
Revised: e − t τgrad to simplify the calculation, where τ grad is a tunable parameter.

Original: Notice that the function h ( t ) is only used in backward propagation, which means the spike responsekernel in the forward propagation is not necessarily the integral of h ( t ) .
Revised: Notice that the function h ( t ) is only used in backward propagation, which means the spike responsekernel in the forward propagation is not necessarily the integral of h ( t ) .

Original: 3.4 Overall Learning Rule
Revised: 3.4 Overall Learning Rule

Original: The loss function we use in this work is the counting loss function, which has the form L =1 N (cid:80) N out i = (cid:18)
Revised: The loss function we use in this work is the counting loss function, which has the form L =1 N (cid:80) N out i = (cid:18)

Original: [Equation]
Revised: [Equation]

Original: (cid:16) N targeti − (cid:82)T
Revised: (cid:16) N targeti − (cid:82)

Original: [Equation]
Revised: [Equation]

Original: s i ( t ) dt (cid:17)(cid:19) 2 , where N out is the number of output neurons and equalsto the number of classes, s i ( t ) represents the spike train emitted by neuron i .
Revised: s i ( t ) dt (cid:17)(cid:19) 2 , where N out is the number of output neurons and equalsto the number of classes, s i ( t ) represents the spike train emitted by neuron i .

Original: Besides, N targeti is thetarget of the spike number outputted by neuron i and typically we set N targeti larger when i is thecorrect answer.
Revised: Besides, N targeti is thetarget of the spike number outputted by neuron i and typically we set N targeti larger when i is thecorrect answer.

Original: During the learning process, the gradient is first propagated from the loss function to the firing time ofeach spike from the last layer to the first layer.
Revised: During the learning process, the gradient is first propagated from the loss function to the firing time ofeach spike from the last layer to the first layer.

Original: The formula for this stage is (please refer to Appendixfor the detailed deduction):∂L
Revised: The formula for this stage is (please refer to Appendixfor the detailed deduction):∂L

Original: [Equation]
Revised: [Equation]

Original: where t k,next ( s ( l ) i ) is the firing time of the first spike emitted by neuron i after time t m .
Revised: where t k,next ( s ( l ) i ) is the firing time of the first spike emitted by neuron i after time t m .

Original: After this, the gradient to weights in each layer is calculated by summing up the multiplication of thegradients of spike firing times in the same layer and the derivative of weights with respect to spikefiring times.
Revised: After this, the gradient to weights in each layer is calculated by summing up the multiplication of thegradients of spike firing times in the same layer and the derivative of weights with respect to spikefiring times.

Original: The learning rule for this stage is
Revised: The learning rule for this stage is

Original: [Equation]
Revised: [Equation]

Original:  Experiments
Revised:  Experiments

Original: In this section, we validate the effectiveness of our method on MNIST [52], Fashion-MNIST [53],
Revised: In this section, we validate the effectiveness of our method on MNIST [52], Fashion-MNIST [53],

Original: N-MNIST [54], CIFAR10 [55], and CIFAR100 [55] datasets.
Revised: N-MNIST [54], CIFAR10 [55], and CIFAR100 [55] datasets.

Original: We first introduce the training details, then evaluate the performance of our algorithm and compare it with the state-of-the-art event-drivenlearning approaches.
Revised: We first introduce the training details, then evaluate the performance of our algorithm and compare it with the state-of-the-art event-drivenlearning approaches.

Original: At last, we conduct ablation studies to illustrate the effectiveness of our proposedmodules.
Revised: At last, we conduct ablation studies to illustrate the effectiveness of our proposedmodules.

Original: More details of the configurations can be found in the Appendix.
Revised: More details of the configurations can be found in the Appendix.

Original: 4.1 Training Details
Revised: 4.1 Training Details

Original: Initialization: When training in an event-driven fashion, gradient information is only carried byspikes.
Revised: Initialization: When training in an event-driven fashion, gradient information is only carried byspikes.

Original: Therefore, the gradient information will be completely blocked by a layer when there areno spikes in that layer.
Revised: Therefore, the gradient information will be completely blocked by a layer when there areno spikes in that layer.

Original: To solve this problem, we start with layers of arbitrarily initialized weightsand scale them by certain multiples, which can make the average firing rate to be a certain numberfor each layer.
Revised: To solve this problem, we start with layers of arbitrarily initialized weightsand scale them by certain multiples, which can make the average firing rate to be a certain numberfor each layer.

Original: We obtain these multiple parameters by binary search and this strategy works well inpractice.
Revised: We obtain these multiple parameters by binary search and this strategy works well inpractice.

Original: Supervisory signals: Another problem we face is that neurons corresponding to certain classes donot fire anymore after certain training, which makes recognizing those classes correctly impossible inthe following epochs.
Revised: Supervisory signals: Another problem we face is that neurons corresponding to certain classes donot fire anymore after certain training, which makes recognizing those classes correctly impossible inthe following epochs.

Original: To address this problem, we utilize supervisory signals.
Revised: To address this problem, we utilize supervisory signals.

Original: For each neuron in theoutput layer corresponding to the ground-truth label, we force it to fire at the end of the simulation.
Revised: For each neuron in theoutput layer corresponding to the ground-truth label, we force it to fire at the end of the simulation.

Original: Experiment settings: In our experiments, we use the real-valued spike current representing the pixelintensities of the image as inputs.
Revised: Experiment settings: In our experiments, we use the real-valued spike current representing the pixelintensities of the image as inputs.

Original: We list the network architectures each work uses in table 2.
Revised: We list the network architectures each work uses in table 2.

Original: Noticethat the output layer is, by default, a fully-connected layer containing the same number of neurons asthe number of classes in the dataset, and omitted from the architecture representation.
Revised: Noticethat the output layer is, by default, a fully-connected layer containing the same number of neurons asthe number of classes in the dataset, and omitted from the architecture representation.

Original: We run allexperiments on a single Nvidia A100 GPU.
Revised: We run allexperiments on a single Nvidia A100 GPU.

Original: 4. Comparison with the State-of-the-Art
Revised: 4. Comparison with the State-of-the-Art

Original: [Table]
Revised: [Table]

Revised: [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: CIFARNet: 128C3-256C3-P2-512C3-P2-1024C3-512C3-1024-
Revised: CIFARNet: 128C3-256C3-P2-512C3-P2-1024C3-512C3-1024-

Original: [Table]
Revised: [Table]

Original: VGG11: 128C3-128C3-P2-256C3-256C3-256C3-P2-512C3-512C3-512C3-P2-
Revised: VGG11: 128C3-128C3-P2-256C3-256C3-256C3-P2-512C3-512C3-512C3-P2-

Original: [Table]
Revised: [Table]

Original: Tab. 2 reports the accuracies of the proposed method and other comparing methods.
Revised: Tab. 2 reports the accuracies of the proposed method and other comparing methods.

Original: The performanceof our algorithm is lower than TSSL-BP by 0.06% on the MNIST dataset and 0.01% on the N-MNISTdataset.
Revised: The performanceof our algorithm is lower than TSSL-BP by 0.06% on the MNIST dataset and 0.01% on the N-MNISTdataset.

Original: However, the output of their network is real-valued postsynaptic currents while the outputof our network is binary spikes.
Revised: However, the output of their network is real-valued postsynaptic currents while the outputof our network is binary spikes.

Original: In addition, they use RNN-like gradients to assist learning.
Revised: In addition, they use RNN-like gradients to assist learning.

Original: On theremaining datasets, we have achieved state-of-the-art performance among these works with temporalgradients.
Revised: On theremaining datasets, we have achieved state-of-the-art performance among these works with temporalgradients.

Original: For the Fashion-MNIST dataset, our algorithm performs 0.45% higher than the previous
Revised: For the Fashion-MNIST dataset, our algorithm performs 0.45% higher than the previous

Original: SOTA.
Revised: SOTA.

Original: On the CIFAR10 dataset, we outperform the existing SOTA by 0.47%.
Revised: On the CIFAR10 dataset, we outperform the existing SOTA by 0.47%.

Original: For the CIFAR100 dataset, we are the first work to successfully train SNNs with time-based gradients in an event-drivenfashion.
Revised: For the CIFAR100 dataset, we are the first work to successfully train SNNs with time-based gradients in an event-drivenfashion.

Original: We have achieved a performance of 63.97% on this dataset.
Revised: We have achieved a performance of 63.97% on this dataset.

Original: 4.3 Ablation Studies
Revised: 4.3 Ablation Studies

Original: To show the effect of our proposed modules, we conduct ablation experiments on the CIFAR10dataset.
Revised: To show the effect of our proposed modules, we conduct ablation experiments on the CIFAR10dataset.

Original: Specifically, two proposed components are taken into consideration: (1) As mentioned in
Revised: Specifically, two proposed components are taken into consideration: (1) As mentioned in

Original: Section 3.3, we compare the proposed gradient assignment functions h ( t ) =
Revised: Section 3.3, we compare the proposed gradient assignment functions h ( t ) =

Original: e − βt in (12) with thecommonly used one h ( t ) =
Revised: e − βt in (12) with thecommonly used one h ( t ) =

Original: dϵ ( t ) dt .
Revised: dϵ ( t ) dt .

Original: (2) We compare the results of three different types of poolinglayers (average pooling, adjusted average pooling and max pooling) mentioned in Section 3.2.
Revised: (2) We compare the results of three different types of poolinglayers (average pooling, adjusted average pooling and max pooling) mentioned in Section 3.2.

Original: Wehave tried all combinations of gradient assignment functions and pooling layers.
Revised: Wehave tried all combinations of gradient assignment functions and pooling layers.

Original: The test accuracy ofthese different settings is shown in Tab. 3.
Revised: The test accuracy ofthese different settings is shown in Tab. 3.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: The results in Tab. 3 meets our expectation.
Revised: The results in Tab. 3 meets our expectation.

Original: For the pooling layer, maxpooling and adjusted average poolinghave much better performance thanthe average pooling.
Revised: For the pooling layer, maxpooling and adjusted average poolinghave much better performance thanthe average pooling.

Original: This accordswith the conclusion in Section 3.2 thatpooling layers keeping the property ofinvariant sum of gradients are betterthan those that do not.
Revised: This accordswith the conclusion in Section 3.2 thatpooling layers keeping the property ofinvariant sum of gradients are betterthan those that do not.

Original: The proposed gradient assignmentfunction h ( t ) =
Revised: The proposed gradient assignmentfunction h ( t ) =

Original: e − βt is also betterthan
Revised: e − βt is also betterthan

Original: the commonly used one h ( t ) =
Revised: the commonly used one h ( t ) =

Original: dϵ ( t ) dt for all three types of pooling layers.
Revised: dϵ ( t ) dt for all three types of pooling layers.

Original: In addition, as shown in Fig.
Revised: In addition, as shown in Fig.

Original: 5,h ( t ) =
Revised: 5,h ( t ) =

Original: e − βt converges faster thanh ( t ) =
Revised: e − βt converges faster thanh ( t ) =

Original: dϵ ( t ) dt in early stage.
Revised: dϵ ( t ) dt in early stage.

Original:  Conclusion and Discussion
Revised:  Conclusion and Discussion

Original: In this work, we analyze the commonly-used SNN temporal backpropagation training approach andfind that it follows the gradient assignment rule.
Revised: In this work, we analyze the commonly-used SNN temporal backpropagation training approach andfind that it follows the gradient assignment rule.

Original: We also find the average pooling layer does not obeythis rule while the max pooling layer does.
Revised: We also find the average pooling layer does not obeythis rule while the max pooling layer does.

Original: We show that the direction of the temporal gradient will bereversed when the spike kernel is decreasing and avoid it with an increasing kernel in backpropagation.
Revised: We show that the direction of the temporal gradient will bereversed when the spike kernel is decreasing and avoid it with an increasing kernel in backpropagation.

Original: Our algorithm achieves state-of-the-art performance on CIFAR10 among time-based SNN learningapproaches and successfully learns the parameters of SNN on CIFAR100 for the first time.
Revised: Our algorithm achieves state-of-the-art performance on CIFAR10 among time-based SNN learningapproaches and successfully learns the parameters of SNN on CIFAR100 for the first time.

Original: Compared with RNN-like methods, the proposed event-based learning algorithm has a lower computational cost and memory occupation when there are many time steps.
Revised: Compared with RNN-like methods, the proposed event-based learning algorithm has a lower computational cost and memory occupation when there are many time steps.

Original: Besides, our algorithm alsodoes not need bias between layers.
Revised: Besides, our algorithm alsodoes not need bias between layers.

Original: Meanwhile, gradient propagation between spikes instead of timesteps can mitigate the gradient explosion/vanishing problem along the time axis.
Revised: Meanwhile, gradient propagation between spikes instead of timesteps can mitigate the gradient explosion/vanishing problem along the time axis.

Original: However, there is still a gap between event-driven backpropagation and biological plausible learning,since event-driven backpropagation processes the spike train in reverse time, which conflicts with theonline learning in the real world and desires for future research.
Revised: However, there is still a gap between event-driven backpropagation and biological plausible learning,since event-driven backpropagation processes the spike train in reverse time, which conflicts with theonline learning in the real world and desires for future research.