Original: Training Spiking Neural Networks with Event-driven Backpropagation
Revised: Training Spiking Neural Networks with Event-driven Backpropagation

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Spiking Neural networks (SNNs) represent and transmit information by spatiotemporal spike patterns, which bring two major advantages: biological plausibilityand suitability for ultralow-power neuromorphic implementation.
Revised: Spiking Neural networks (SNNs) represent and transmit information by spatiotemporal spike patterns, which bring two major advantages: biological plausibility and suitability for ultralow-power neuromorphic implementation.

Original: Despite this, thebinary firing characteristic makes training SNNs more challenging.
Revised: Despite this, the binary firing characteristic makes training SNNs more challenging.

Original: To learn theparameters of deep SNNs in an event-driven fashion as in inference of SNNs, backpropagation with respect to spike timing is proposed.
Revised: To learn the parameters of deep SNNs in an event-driven fashion as in inference of SNNs, backpropagation with respect to spike timing is proposed.

Original: Although this event-drivenlearning has the advantages of lower computational cost and memory occupation,the accuracy is far below the recurrent neural network-like learning approaches.
Revised: Although this event-driven learning has the advantages of lower computational cost and memory occupation, the accuracy is far below the recurrent neural network-like learning approaches.

Original: Inthis paper, we first analyze the commonly used temporal backpropagation training approach and prove that the sum of gradients remains unchanged betweenfully-connected and convolutional layers.
Revised: In this paper, we first analyze the commonly used temporal backpropagation training approach and prove that the sum of gradients remains unchanged between fully-connected and convolutional layers.

Original: Secondly, we show that the max poolinglayer meets the above invariance rule, while the average pooling layer does not,which will suffer the gradient vanishing problem but can be revised to meet therequirement.
Revised: Secondly, we show that the max pooling layer meets the above invariance rule, while the average pooling layer does not, which will suffer the gradient vanishing problem but can be revised to meet the requirement.

Original: Thirdly, we point out the reverse gradient problem for time-basedgradients and propose a backward kernel that can solve this problem and keepthe property of the invariable sum of gradients.
Revised: Thirdly, we point out the reverse gradient problem for time-based gradients and propose a backward kernel that can solve this problem and keep the property of the invariable sum of gradients.

Original: The experimental results showthat the proposed approach achievesthe state-of-the-art performance on CIFAR10among time-based training methods.
Revised: The experimental results show that the proposed approach achieves state-of-the-art performance on CIFAR among time-based training methods.

Original: Also, this is the first time that the time-basedbackpropagation approach successfully trains SNN on the CIFAR100 dataset.
Revised: Also, this is the first time that the time-based backpropagation approach successfully trains SNN on the CIFAR100 dataset.

Revised: Our code is available at https://github.com/zhuyaoyu/SNN-event-driven-learning.

Original:  Introduction
Revised:  Introduction

Original: Motivated by the principles of brain computing, Spiking Neural Networks (SNNs) are consideredas the third generation of neural networks [1, 2].
Revised: Motivated by the principles of brain computing, Spiking Neural Networks (SNNs) are considered as the third generation of neural networks [1, 2].

Original: SNNs are developed to work in power-criticalscenarios, such as edge computing.
Revised: SNNs are developed to work in power-critical scenarios, such as edge computing.

Original: When run on dedicated neuromorphic chips, they can accomplishthe tasks [3, 4, 5] with ultra-low power consumption [6, 7, 8, 9, 10, 11].
Revised: When run on dedicated neuromorphic chips, they can accomplish the tasks [3, 4, 5] with ultra-low power consumption [6, 7, 8, 9, 10, 11, 12].

Original: In contrast, the last generationof neural networks – Artificial Neural Networks (ANNs) [12], generally require a large amount ofcomputation resource (e.g., GPUs).
Revised: In contrast, the last generation of neural networks – Artificial Neural Networks (ANNs) [13], generally require a large amount of computation resource (e.g., GPUs).

Original: This advantage of SNNs on power consumption largely relieson efficient event-based computations [13, 14].
Revised: This advantage of SNNs on power consumption largely relies on efficient event-based computations [14, 15].

Original: Another advantage of SNNs originates from theirbiological reality (compared to ANNs).
Revised: Another advantage of SNNs originates from their biological reality (compared to ANNs).

Original: The similarity between SNNs and biological brains providesan excellent opportunity to study how the brain computes at the neuronal circuit level [15].
Revised: The similarity between SNNs and biological brains provides an excellent opportunity to study how the brain computes at the neuronal circuit level [16].

Original: Compared with artificial neural networks, developing supervised learning algorithms for spikingneural networks requires more effort.
Revised: Compared with artificial neural networks, developing supervised learning algorithms for spiking neural networks requires more effort.

Original: The main challenge for training SNNs comes from the binarynature of spikes and the non-differentiability of the membrane potential at spike time.
Revised: The main challenge for training SNNs comes from the binary nature of spikes and the non-differentiability of the membrane potential at spike time.

Original: This difficulty in training impedes the performance of SNNs in pattern classification tasks compared to their ANNcounterparts.
Revised: This difficulty in training impedes the performance of SNNs in pattern classification tasks compared to their ANN counterparts.

Original: Existing supervised learning methods of SNNs can be grouped into two categories:
Revised: Existing supervised learning methods of SNNs can be grouped into two categories:

Original: The first category consists of recurrent neural network (RNN)-like learning algorithms.
Revised: The first category consists of recurrent neural network (RNN)-like learning algorithms.

Original: Thesealgorithms treat spiking neural networks as binary-output recurrent neural networks and handle thediscontinuities of membrane potential at spike times with continuous surrogate derivatives [16].
Revised: These algorithms treat spiking neural networks as binary-output recurrent neural networks and handle the discontinuities of membrane potential at spike times with continuous surrogate derivatives [17].

Original: Theytypically train deep SNNs with surrogate gradients based on the idea of backpropagation throughtime (BPTT) algorithm[17, 18, 19, 20, 21, 22, 23, 24, 25, 26].
Revised: They typically train deep SNNs with surrogate gradients based on the idea of backpropagation through time (BPTT) algorithm

Original: While competitive accuracies arereported on the MNIST, CIFAR-10, and even ImageNet datasets [27, 28, 29], the gradient informationis propagated each time step, whether or not a spike is emitted (as shown in Fig. 1).
Revised: While competitive accuracies are reported on the MNIST, CIFAR-10, and even ImageNet datasets [29, 30, 31], the gradient information is propagated each time step, whether or not a spike is emitted (as shown in Fig. 1).

Original: Therefore,these approaches do not follow the event-driven nature of spiking neural networks, which lose theasynchronous characteristic of SNNs and consume much power when trained on neuromorphichardware.
Revised: Therefore, these approaches do not follow the event-driven nature of spiking neural networks, which lose the asynchronous characteristic of SNNs and consume much power when trained on neuromorphic hardware.

Original: The second category is event-driven algorithms, which propagate gradient information through spikes.
Revised: The second category is event-driven algorithms, which propagate gradient information through spikes.

Original: Precise spiking timing acts an important role in this situation, and they are extensively used in suchalgorithms [30, 31, 32, 33, 34, 35, 36, 37].
Revised: Precise spiking timing acts an important role in this situation, and they are extensively used in such algorithms [32, 33, 34, 35, 36, 37, 38, 39].

Original: Classical examples include SpikeProp [30] and its variants [31, 38, 39].
Revised: Classical examples include SpikeProp [32] and its variants [33, 40, 41].

Original: These algorithms approximate the derivative of spike timing to membrane potentialas the negative inverse of the time derivative of membrane potential function.
Revised: These algorithms approximate the derivative of spike timing to membrane potential as the negative inverse of the time derivative of membrane potential function.

Original: This approximationis actually mathematically correct without preconditions [40].
Revised: This approximation is actually mathematically correct without preconditions [42].

Original: Some other works apply non-leakyintegrate-and-fire neurons to stabilize the training process [33, 41, 36].
Revised: Some other works apply non-leaky integrate-and-fire neurons to stabilize the training process [35, 38, 43].

Original: Most of these works restricteach neuron to fire at most once, which inspires [42] to take the spike time as the state of a neuron, andmodel the relation of neurons by this spike time.
Revised: Most of these works restrict each neuron to fire at most once, which inspires [44] to take the spike time as the state of a neuron, and model the relation of neurons by this spike time.

Original: As a result, the SNN is trained similarly to an ANN.
Revised: As a result, the SNN is trained similarly to an ANN.

Original: Among the methodsallowing multiple spikes trained in an event-driven fashion, the state-of-the-artmodel is TSSL-BP [37].
Revised: Among the methods trained in an event-driven fashion (not modelling the relation of spike time to train like ANNs), the state-of-the-art model is TSSL-BP [39].

Original: However, they use RNN-like surrogate gradients (a sigmoid function) toassist training.
Revised: However, they use RNN-like surrogate gradients (a sigmoid function) to assist training.

Original: It is still challenging to train SNNs in a pure event-driven fashion.
Revised: Hence, it is still challenging to train SNNs in a pure event-driven fashion.

Original: In this work, we develop a novel event-driven learning algorithm that can train high-performancedeep SNNs.
Revised: In this work, we develop a novel event-driven learning algorithm that can train high-performance deep SNNs.

Original: The main contributions of our work are as follows:
Revised: The main contributions of our work are as follows:

Original: We prove that the typical SNN temporal backpropagation training approach assigns the gradient of an output spike of a neuron to the input spikes generating it.
Revised: We prove that the typical SNN temporal backpropagation training approach assigns the gradient of an output spike of a neuron to the input spikes generating it.

Original: After summing this assignment rule altogether, we find that the sum of gradients iskept unchanged betweenlayers.
Revised: After summing this assignment rule altogether, we find that the sum of gradients is unchanged between layers.

Original: We analyze the case of the pooling layer (which does not have weights) and find that averagepooling does not keep the gradient sum unchanged, but we can modify its backward formulas to meet the requirement.
Revised: We analyze the case of the pooling layer (which does not have neurons) and find that average pooling does not keep the gradient sum unchanged, but we can modify its backward formulas to meet the requirement.

Original: Meanwhile, the max-pooling layer satisfies the rule initially.
Revised: Meanwhile, the max-pooling layer satisfies the rule initially.

Original: We point out the reverse gradient problem in event-driven learning that the direction of the temporal gradient is reversed during backpropagation when the kernel function of an input spike is decreasing.
Revised: We point out the reverse gradient problem in event-driven learning that the direction of the temporal gradient is reversed during backpropagation when the kernel function of an input spike is decreasing.

Original: Then we propose a backward kernel function that addresses this problem while keeping the sum of gradients unchanged between layers.
Revised: Then we propose a backward kernel function that addresses this problem while keeping the sum of gradients unchanged between layers.

Original: The adjusted average pooling layer and the non-decreasing backward kernel enhances theperformance of our model as well as the convergence speed.
Revised: The adjusted average pooling layer and the non-decreasing backward kernel enhances the performance of our model as well as the convergence speed.

Original: To our best knowledge, ourproposed approach achieves state-of-the-art performance on CIFAR10 among event-driven training methods (with temporal gradients) for SNNs.
Revised: To our best knowledge, our proposed approach achieves state-of-the-art performance on CIFAR10 among event-driven training methods (with temporal gradients) for SNNs.

Original: Meanwhile, our method is the firstevent-driven backpropagation approach that successfully trains SNN on the larger-scaleCIFAR100 dataset.
Revised: Meanwhile, our method is the first event-driven backpropagation approach that successfully trains SNN on the larger-scale CIFAR100 dataset.

Original:  Backgrounds and Related Work
Revised:  Backgrounds and Related Work

Original: The gradient-based learning of spiking neural networks contains two stages: the forward (inference)and the backward (learning) stages.
Revised: The gradient-based learning of spiking neural networks contains two stages: the forward (inference) and the backward (learning) stages.

Original: In the forward stage, Leaky Integrate-and-Fire (LIF) neuronsare most commonly used [20, 17, 25, 37], while other types of neurons are also applicable [30, 33].
Revised: In the forward stage, Leaky Integrate-and-Fire (LIF) neurons are most commonly used [18, 21, 26, 39], while other types of neurons are also applicable [32, 35].

Original: Typically, these neuron models can be changed to the form of the Spike Response Model 
Revised: Typically, these neuron models can be changed to the form of the Spike Response Model (SRM) [37, 45, 46], which is easily represented in an event-driven fashion.

Revised: In the backward stage, the methods used by existing works exhibits more diversity.

Revised: Here, we classify existing approaches from two dimensions: whether non-spike information is needed in discrete time steps (RNN-like) or not (event-driven) and whether the gradient represents spike scale (activation-based) or spike timing (time-based).

Original: Gradient direction 𝑡 𝑢
Revised: Gradient direction 𝑡 𝑢

Original: Output
Revised: Output

Original: Input 𝑡 𝑢 𝑢 𝑗
Revised: Input 𝑡 𝑢 𝑢 𝑗

Original: 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗
Revised: 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗 𝑢 𝑗 𝐼 𝑗

Original: Spike
Revised: Spike

Original: No spike
Revised: No spike

Original: Gradient direction a b c d 𝑡 𝑡 𝑡𝑡 0 +1 𝑡 0 +2 𝑡 0 + 𝑡𝑡 0 +1 𝑡 0 +2 𝑡 0 +
Revised: Gradient direction a b c d 𝑡 𝑡 𝑡𝑡 0 +1 𝑡 0 +2 𝑡 0 + 𝑡𝑡 0 +1 𝑡 0 +2 𝑡 0 +

Original: Output
Revised: Output

Original: Input
Revised: Input

Original: Spike train
Revised: Spike train

Original: In the backward stage, the situation is more complex.

Original: Here, we classify existing approaches fromtwo dimensions: whether non-spike information is needed in discrete time steps (RNN-like) or not(event-driven) and whether the gradient represents spike scale (activation-based) or spike timing(time-based).

Original: Event-driven learning v.s. RNN-like learning: In both forward and backward computation ofevent-driven learning, information is only carried by spikes in SNNs.
Revised: Event-driven learning v.s. RNN-like learning: In both forward and backward computation of event-driven learning, information is only carried by spikes in SNNs.

Original: Specifically, in backwardcomputation, gradient information is propagated through spikes [30, 31, 39, 33] (shown in Fig.
Revised: Specifically, in backward computation, gradient information is propagated through spikes [32, 33, 41, 35] (shown in Fig.

Original: 1a-b).
Revised: 1a-b).

Original: On the other side, in RNN-like learning, information is not only carried by spikes in backwardcomputation.
Revised: On the other side, in RNN-like learning, information is not only carried by spikes in backward computation.

Original: Especially, gradient information can be propagated through a neuron that does not emita spike in backward computation (shown in Fig.
Revised: Especially, gradient information can be propagated through a neuron that does not emit a spike in backward computation (shown in Fig.

Original: 1c-d).
Revised: 1c-d).

Original: This gradient propagation is achieved by asurrogate function [11, 16, 17, 22, 45], which is a function of the membrane potential at the currenttime step u t , and the firing threshold θ .
Revised: This gradient propagation is achieved by a surrogate function [12, 17, 18, 23, 47], which is a function of the membrane potential at the current time step u t , and the firing threshold θ .

Original: Time-based gradient v.s. activation-based gradient
Revised: Time-based gradient v.s. activation-based gradient

Original: : Time-based gradients represent the (reverse)direction that the timing of a spike should move, that is, to move leftward or rightward on the timeaxis [30].
Revised: : Time-based gradients represent the (reverse) direction that the timing of a spike should move, that is, to move leftward or rightward on the time axis [32].

Original: In backward propagation, the derivative of the firing time of a spike to the correspondingmembrane potential ∂t∂u is often approximated as − 1 ∂u∂t [30, 31], denoting how the change of membranepotential will change the spike firing time (Fig. 1b).
Revised: In backward propagation, the derivative of the firing time of a spike to the corresponding membrane potential ∂t∂u is often approximated as − 1 ∂u∂t [32, 33], denoting how the change of membrane potential will change the spike firing time (Fig. 1b).

Original: On the other side, activation-based approachesreplace the Heaviside neuron activation function Θ( · ) ( s t =
Revised: On the other side, activation-based approaches replace the Heaviside neuron activation function Θ( · ) (spike s t =

Original: Θ( u t − θ ) ) in forward propagationwith derivable functions σ ( · ) in backward propagation, whether there are spikes in the current timestep [17, 25, 29, 20].
Revised: Θ( u t − θ ) ) in forward propagation with derivable functions σ ( · ) in backward propagation, whether there are spikes in the current time step [18, 26, 31, 21].

Original: Therefore, activation-based approaches essentially regard SNNs as binary
Revised: Therefore, activation-based approaches essentially regard SNNs as binary

Original: RNNs and train them with approximated gradients, where the gradients indicate whether the valuesin the network (including the binary spikes) should be larger or smaller (Fig. 1d).
Revised: RNNs and train them with approximated gradients, where the gradients indicate whether the values in the network (including the binary spikes) should be larger or smaller (Fig. 1d).

Original: As a result, time-based gradients are event-driven by nature, since the temporal gradient could only becarried by spikes.
Revised: As a result, time-based gradients are event-driven by nature, since the temporal gradient could only be carried by spikes.

Original: Meanwhile, activation-based gradients are more suitable for the RNN-like trainingscheme since the diversity of surrogate gradients largely relies on the fact that u t ̸
Revised: Meanwhile, activation-based gradients are more suitable for the RNN-like training scheme since the diversity of surrogate gradients largely relies on the fact that u t ̸

Original: = θ in discrete timesteps [16], which no longer holds in continuous time simulation.
Revised: = θ in discrete time steps [17], which no longer holds in continuous time simulation.

Original: If we want to apply activation-basedgradients to event-driven learning, there should only be one value ∂s∂u when the membrane potentialreaches the threshold.
Revised: If we want to apply activation-based gradients to event-driven learning, there should only be one value ∂s∂u when the membrane potential reaches the threshold.

Original: Tab. 1 lists whether a gradient type can be used in a learning fashion.
Revised: Tab. 1 lists whether a gradient type can be used in a learning fashion.

Original: It should be noticed thatalthough activation-based gradient is more suitable for RNN-like learning, it is still able to be usedfor event-driven learning.
Revised: It should be noticed that although activation-based gradient is more suitable for RNN-like learning, it is still able to be used for event-driven learning.

Original: [Table]
Revised: [Table]

Original:  Methods 
Revised:  Methods 3.1 Forward Formulas

Original: 3.1 Forward Formulas

Original: We use the spike response model [1] for neurons in the network.
Revised: We use the spike response model [1] for neurons in the network.

Original: The forward propagation in thenetwork can be described as follows:
Revised: The forward propagation in the network can be described as follows:

Original: [Equation]
Revised: [Equation]

Original: Here u ( l ) i ( t ) denotes the membrane potential of neuron i in layer l at time t , w ( l ) ij denotes the weightbetween neuron j in layer l − 1 and neuron i in layer l .
Revised: Here u ( l ) i ( t ) denotes the membrane potential of neuron i in layer l at time t , w ( l ) ij denotes the weight between neuron j in layer l − 1 and neuron i in layer l .

Original: t ( l ) i,last is the time of last spike of neuron i inlayer l , and s ( l ) i ( t ) represents the spike emitted from neuron i at time t .
Revised: t ( l ) i,last is the time of last spike of neuron i in layer l , and s ( l ) i ( t ) represents the spike emitted from neuron i at time t .

Original: The function δ ( · ) is the Dirac
Revised: The function δ ( · ) is the Dirac

Original: Delta function and θ is the firing threshold.
Revised: Delta function and θ is the firing threshold.

Original: The spike response kernel ϵ ( t ) can be described by
Revised: The spike response kernel ϵ ( t ) can be described by

Original: [Equation]
Revised: [Equation]

Original: where τ m and τ s are the membrane time constant and the synapse time constant respectively.
Revised: where τ m and τ s are the membrane time constant and the synapse time constant respectively.

Original: Noticethat we do not use reset kernels as in previous works [39, 37].
Revised: Notice that we do not use reset kernels as in previous works [39, 41].

Original: Instead, we eliminate the influence ofinput spikes prior to the last output spike on membrane potentials.
Revised: Instead, we eliminate the influence of input spikes prior to the last output spike on membrane potentials.

Original: 3.2 Rethinking the Classical Time-based Backward Propagation Formula
Revised: 3.2 Rethinking the Classical Time-based Backward Propagation Formula

Original: In this subsection, we analyze the classical time-based backpropagation formula in SNNs.
Revised: In this subsection, we analyze the classical time-based backpropagation formula in SNNs.

Original: Wefirst theoretically prove that it is essentially a weighted gradient assignment mechanism amonglayers.
Revised: We first theoretically prove that the backpropagation rule essentially assigns gradients of output spikes of neurons to their input spikes.

Original: Then we check the pooling layer and show that the average pooling should be adjusted inbackpropagation to satisfy the gradient assignment mechanism, while the max pooling naturallysatisfies this mechanism.
Revised: Then we check the pooling layer and show that the average pooling should be adjusted in backpropagation to satisfy the gradient assignment mechanism, while the max pooling naturally satisfies this mechanism.

Original: Invariant sum of gradients among layers with weights.
Revised: Invariant sum of gradients among layers with weights.

Original: The most commonly used time-basedgradient backpropagation method origins from [30].
Revised: The most commonly used time-based gradient backpropagation method origins from [32].

Original: The two key approximations are as follows:
Revised: The two key approximations are as follows:

Revised: ∂t

Original: [Equation]
Revised: [Equation]

Original: where t k ( s ( l ) i ) denotes the firing time t k of neuron i in layer l , t k,last ( s ( l ) i ) is the firing time of the lastspike emitted by neuron i before time t k .
Revised: where t k ( s ( l ) i ) denotes the firing time t k of neuron i in layer l , t k,last ( s ( l ) i ) is the firing time of the last spike emitted by neuron i before time t k .

Original: ∂t k ( s ( l ) i ) ∂ · means the influence of changing other variableson the timing of a spike, and
Revised: ∂t k ( s ( l ) i ) ∂ · means the influence of changing other variables on the timing of a spike, and

Original: [Equation]
Revised: [Equation]

Original: is the influence of changing spike timing on that variable.
Revised: is the influence of changing spike timing on that variable.

Original: Combining Eqs.
Revised: Combining Eqs.

Original: 4-5 and the forward formulas, we can get an invariant equality:
Revised: 4-5 and the forward formulas, we can get an invariant equality:

Original: [Equation]
Revised: [Equation]

Original: The proof is provided in Appendix.
Revised: The proof is provided in Appendix.

Original: Eq.
Revised: Eq.

Original: 6 implies the fact that the reference time ( t = 0 ) ismeaningless, and only relative spike times matter.
Revised: 6 implies the fact that the reference time ( t = 0 ) is meaningless, and only relative spike times matter.

Original: If we increase all the spike times in layer l − 1 by1 unit along the time axis, then all the spike times in layer l are also increased by 1 unit along thetime axis.
Revised: If we increase all the spike times in layer l − 1 by 1 unit along the time axis, then all the spike times in layer l are also increased by 1 unit along the time axis.

Original: Denote the loss function as L , then the gradient of L with respect to t m ( s ( l − 1) j ) is:
Revised: Denote the loss function as L , then the gradient of L with respect to t m ( s ( l − 1) j ) is:

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: Output

Original: Input = + + = + +
Revised: = + + = + +

Original: Loss
Revised: Loss

Original: [Figure]
Revised: [Figure]

Original: 𝒍−

Original: [Figure] 𝒍 where t m,next ( s ( l − 1) j ) denotes the firing time of the next spike emitted by neuron j after timet m .
Revised: where t m,next ( s ( l − 1) j ) denotes the firing time of the next spike emitted by neuron j after time t m .

Original: Therefore, we actually decompose the gradient ∂L/∂t k ( s ( l ) i ) from layer l into (part of) aset of gradients ∂L/∂t m ( s ( l − 1) j ) of the last layer l − 1 , and keep their sum unchanged.
Revised: Therefore, we actually decompose the gradient ∂L/∂t k ( s ( l ) i ) from layer l into (part of) a set of gradients ∂L/∂t m ( s ( l − 1) j ) of the last layer l − 1 , and keep their sum unchanged.

Original: In otherwords, we assign the weighted sum ∂L/∂t k ( s ( l ) i ) by weights ∂t k ( s ( l ) i ) /∂t m ( s ( l − 1) j ) to the gradients∂L/∂t m ( s ( l − 1) j ) in the last layer, as shown in Fig. 2.
Revised: In other words, we assign the weighted sum ∂L/∂t k ( s ( l ) i ) by weights ∂t k ( s ( l ) i ) /∂t m ( s ( l − 1) j ) to the gradients ∂L/∂t m ( s ( l − 1) j ) in the last layer, as shown in Fig. 2.

Original: If we sum all the gradients together, we can get another invariant in this backpropagation rule:(cid:88)
Revised: If we sum all the gradients together, we can get another invariant in this backpropagation rule: (cid:88)

Original: [Equation]
Revised: [Equation]

Revised: which means the sum of gradients (cid:80) i (cid:80) t k

Original: [Equation]
Revised: [Equation]

Original: which means the sum of gradients (cid:80) i (cid:80) t k ∂L ∂t k ( s ( l ) i ) never changes between layers under this rule.
Revised: ( s ( l ) i ) never changes between layers under this rule.

Original: Gradient sum invariance for pooling layers.
Revised: Gradient sum invariance for pooling layers.

Original: The above function determines the gradient propagation in fully-connected and convolution layers.
Revised: The above equations determine the gradient propagation in fully-connected and convolution layers (which contain neurons).

Original: The case for pooling layers  is illustrated in Fig.
Revised: The case for pooling layers (which do not contain neurons) is illustrated in Fig.

Original: [Figure]
Revised: [Figure]

Original: In average pooling with kernel size k , the gradient of one spike (at time t ) in layer l is averagelypropagated to k × k neurons in layer l − 1 connected to it.
Revised: In average pooling with kernel size k , the gradient of one spike (at time t ) in layer l is averagely propagated to k × k neurons in layer l − 1 connected to it.

Original: Some of these k × k neurons may not emitspikes at time t .
Revised: Some of these k × k neurons may not emit spikes at time t .

Original: However, the gradients are also propagated to those neurons, which cannot furtherpropagate the gradients to the previous layers.
Revised: However, the gradients are also propagated to those neurons, which cannot further propagate the gradients to the previous layers.

Original: For instance, the two white squares (neurons) in layerl − 1 in Fig.
Revised: For instance, the two white squares (neurons) in layer l − 1 in Fig.

Original: 3 receive gradients, but they will not further propagate the gradients to layer l − 2 .
Revised: 3 receive gradients, but they will not further propagate the gradients to layer l − 2 .

Original: Thus,a part of the gradients is lost in the backpropagation of the average pooling layer, which might causethe gradient vanishing problem.
Revised: Thus, a part of the gradients is lost in the backpropagation of the average pooling layer, which might cause the gradient vanishing problem.

Original: Meanwhile, the sum of gradients carried by the spikes is not keptamong layers in this case.
Revised: Meanwhile, the sum of gradients carried by the spikes is not kept among layers in this case.

Original: We can adjust the backpropagation stage in average pooling to satisfythe gradient sum invariance requirement by increasing the multiplier from 1 /k 2 to 1 /n spike , wheren spike is the number of spikes emitted by the k × k neurons in layer l − 1 at the current time step. 
Revised: We can adjust the backpropagation stage in average pooling to satisfy the gradient sum invariance requirement by increasing the multiplier from 1 /k 2 to 1 /n spike , where n spike is the number of spikes emitted by the k × k neurons in layer l − 1 at the current time step. [Equation] [Figure]

Revised: Consider moving one of its input spike moving from time t m to t m + ∆ t for an SRM neuron.

Revised: Without loss of generality, we assume the weight multiplied on this input spike is positive.

Revised: If the next output spike is fired at time t k , moving the input spike from t m to t m + ∆ t will decrease the membrane potential of this neuron at time t k , which causes this spike to fire later.

Original:  On the contrary, in max pooling, the gradient of a spike in layer l is entirely propagated to one of thespikes emitted by its connected neurons in layer l − 1 .
Revised: As a result, the gradient direction from the output spike at time t k will be kept. In opposite, if the next output spike is fired at time t ′ k , moving the input spike from t m to t m + ∆ t will cause this spike to fire earlier, so the gradient direction from the output spike at time t ′ k will be reversed. On the contrary, in max pooling, the gradient of a spike in layer l is entirely propagated to one of the spikes emitted by its connected neurons in layer l − 1 (shown in the middle of Fig. 3).

Original: This maintains the property of the invariablesum of gradients.
Revised: This maintains the property of the invariable sum of gradients.

Original: It should be noticed that, although  max pooling is different from adjusted average pooling in discretesimulation, they are almost surely the same in the continuous simulation since (almost surely) no twospikes emit at exactly the same time in this case.
Revised: It should be noticed that, although the backpropagation stage of max pooling is different from adjusted average pooling in discrete simulation, they are almost surely the same in the continuous simulation since (almost surely) no two spikes emit at exactly the same time in this case.

Original: 3.3 Deficiencies of the Typical Time-based  PropagationFormula and a New Approach
Revised: 3.3 Deficiencies of The Typical Time-based Gradient Propagation and A New Approach

Original: 𝑡𝑡 𝑘𝑘 𝑡𝑡 𝑘𝑘′ 𝑡𝑡 𝑚𝑚 + 𝛥𝛥𝑡𝑡 𝑡𝑡 𝑚𝑚 𝜖𝜖 ∗ 𝑠𝑠 𝑡𝑡

Original: Input Output In this section, we first point out the reverse gradient problem in event-driven learning. Specifically, the gradient direction for spike timinggets wrong when the spike response kernel isdecreasing.
Revised: In this section, we first point out the reverse gradient problem in event-driven learning: the gradient direction for spike timing gets wrong when the spike response kernel is decreasing.

Original: Then we propose a backward kernelthat can not only solve the reverse gradient problem but also keep the property of the invariablesum of gradients.
Revised: Then we propose a backward kernel that can not only solve the reverse gradient problem but also keep the property of the invariable sum of gradients.

Original: The reverse gradient problem.
Revised: The reverse gradient problem.

Original: Fig.
Revised: Fig.

Original: 4 illustrates the membrane potential response of a neuron (with index i ) to one of its input spikes (witha positive weight) from presynaptic neuron j ,here we use the standard double-exponentialspike response kernel (Eq. 3).
Revised: 4 illustrates the membrane potential response of a neuron (with index i ) to one of its input spikes (with a positive weight) from presynaptic neuron j . As in equation (3), the spike response kernel is a double-exponentail function.

Original: Notice that this isnot the whole membrane potential of the neuroni as it also receives the inputs from other presynaptic neurons.
Revised: Notice that this is not the whole membrane potential of the neuron i as it also receives the inputs from other presynaptic neurons.

Original: We consider two spike times ( t kand t ′ k ) of neuron i and one spike time t m of presynaptic neuron j .
Revised: We consider two spike times ( t k and t ′ k ) of neuron i and one spike time t m of presynaptic neuron j .

Original: If  neuron i spikes at time t k ,moving the presynaptic spike from t m to t m + ∆ t will decrease membrane potential at time t k ,which means postponing the spike at time t k ( t m ↑⇒ u
Revised: If the next spike of neuron i fires at time t k , moving the presynaptic spike from t m to t m +∆ t will decrease membrane potential at time t k , which means postponing the spike at time t k ( t m ↑⇒ u

Original: [ t k ] ↓⇒ t k ↑ ).
Revised: [ t k ] ↓⇒ t k ↑ ).

Original: This result shows that if thepresynaptic neuron fires earlier, the postsynaptic neuron will fire earlier.
Revised: This result shows that if the presynaptic neuron fires earlier, the postsynaptic neuron will fire earlier in this case.

Original: Oppositely, if the  neuron spikes at  t ′ k , moving  t m to t m + ∆ t will cause a increase of u [ t ′ k ] , whichfurther moves t ′ k leftward ( t m ↑⇒ u
Revised: Oppositely, if the next spike of neuron i fires at time t ′ k , moving the input spike from t m to t m + ∆ t will cause a increase of u [ t ′ k ] , which further moves t ′ k leftward ( t m ↑⇒ u

Original: [ t ′ k ] ↑⇒ t ′ k ↓ ).
Revised: [ t ′ k ] ↑⇒ t ′ k ↓ ).

Original: As a result, if we want to move the output spikeat t ′ k leftward, we should move the input spike at t m rightward, which reverses the direction .
Revised: As a result, if we want to move the output spike at t ′ k leftward, we should move the input spike at t m rightward, which reverses the direction .

Original: Thismight cause a problem: When we want to move t ′
Revised: This might cause a problem: When we want to move t ′

Original: k leftward, we want the neuron to emit morespikes .
Revised: k leftward, we want the neuron to emit more spikes .

Original: However, in gradient backpropagation, it moves t m rightward (assume weight w  > 0 ), whichmay cause the neuron in the last layer to spike fewer, further causing neurons in the current layerto spike fewer .
Revised: However, in gradient backpropagation, it moves t m rightward (assume weight w ij > 0 ), which may cause the neuron in the last layer to spike fewer, further causing neurons in the current layer to spike fewer .

Original: More formally, we assume the neuron i receives a input spike at time t m from presynaptic neuron jwith synaptic weight w ij , then the membrane potential of neuron i at time t k is:
Revised: More formally, we assume the neuron i receives a input spike at time t m from presynaptic neuron j with synaptic weight w ij , then the membrane potential of neuron i at time t k is:

Original: [Equation]
Revised: [Equation]

Original: where ϵ ( t ) denotes the spike response kernel (Eq. 3).
Revised: where ϵ ( t ) denotes the spike response kernel (Eq. 3).

Original: C denotes the influence of other spikes, whichis not in our concern here.
Revised: C denotes the influence of other spikes, which is not in our concern here.

Original: In backward pass, according to Eqs. (4)-(5), we have
Revised: In backward pass, according to Eqs. (4)-(5), we have

Original: [Equation]
Revised: [Equation]

Original: Note that when a spike is emitted by neuron i at time t k , the slope of u i ( t ) > 0 at time t k , whichmeans − 1 du i ( t k ) /dt has a negative sign.
Revised: Note that when a spike is emitted by neuron i at time t k , the slope of u i ( t ) > 0 at time t k , which means [Equation] has a negative sign.

Original: Considering ∂ϵ ( t k − t m ) ∂t m = − dϵ ( τ ) dτ , where τ = t k − t m , we get:sign (cid:32)
Revised: Considering ∂ϵ ( [Equation] = − dϵ ( τ ) dτ , where τ = t k − t m , we get:

Original: [Equation]
Revised: [Equation]

Original: When sign (cid:0) dϵ ( τ ) /dτ (cid:1) = − 1 , which is the part of the spike response kernel that decreases (see thecase at t ′ k in Fig. 4), the gradient direction of t m can be classified into two cases: When w ij > 0 ,sign (cid:16) ∂L
Revised: When sign (cid:0) dϵ ( τ ) /dτ (cid:1) = − 1 , which is the part of the spike response kernel that decreases (see the case at t ′ k in Fig. 4), the gradient direction of t m can be classified into two cases: When w ij > 0 , sign (cid:16) [Equation]

Original: ∂t m ( s j )(cid:17)
Revised: (cid:17) [Equation] s i )

Original: = − sign (cid:16) ∂L ∂t k ( s i ) (cid:17) , which means the gradient direction is reversed. 
Revised: [Equation] , which means the gradient direction is reversed. When w ij < 0 , sign (cid:16) ∂L [Equation]

Original: When w ij < 0 ,sign (cid:16) ∂L ∂t m ( s j ) (cid:17) = sign (cid:16) ∂L ∂t k ( s i ) (cid:17) , which means the gradient direction is kept.
Revised: m ( s j [Equation] (cid:17) [Equation]

Original: In both cases, the sign of the gradient gets wrong in propagation between layers.
Revised: k [Equation] s i ) [Equation]

Revised: , which means the gradient direction is kept.

Revised: In both cases, the sign of the gradient gets wrong in propagation between layers.

Original: Thus, the commonlyused double-exponential spike response kernel is incompatible with the time-based gradient inevent-driven learning.
Revised: Thus, the commonly used double-exponential spike response kernel is incompatible with the time-based gradient in event-driven learning.

Original: A smoother gradient assigning approach.
Revised: A smoother gradient assigning approach.

Original: Inspired by the above gradient inconsistency as wellas the invariance of gradient sum, we propose a new gradient backpropagation approach here.
Revised: Inspired by the above gradient inconsistency as well as the invariance of gradient sum, we propose a new gradient backpropagation approach here.

Original: Specifically, we replace the function ∂ϵ ( t k − t m ) ∂t m in Eqs.
Revised: Specifically, we replace the function ∂ϵ ( t k − t m [Equation] ∂t m in Eqs.

Original: (4) and (5) with a new function h
Revised: (4) and (5) with a new function h

Original: ( t k − t m ) .
Revised: ( t k − t m ) .

Original: Therefore, the backpropagation formula between layers turns into:
Revised: Therefore, the backpropagation formula between layers turns into:

Original: [Equation]
Revised: [Equation]

Original: otherwise .
Revised: otherwise .

Original: To guarantee that the gradients are not reversed between layers, we should expect h ( t ) > 0 alwayshold

Original: when t > 0 .

Original: Besides, it can be see from Eq.
Revised: It can be see from Eq.

Original: 12 that ∂t k
Revised: 12 that ∂t k

Original: ( s i ) ∂t m ( s j ) will not change if we multiply h ( t )by an arbitrary constant, so we do not need to care about the scale of h ( t ) .
Revised: ( s i ) ∂t m ( s j ) will not change if we multiply h ( t ) by an arbitrary constant, so we do not need to care about the scale of h ( t ) .

Original: Meanwhile, the propertyof invariable sum of gradients is kept after this replacement.
Revised: Meanwhile, the property of invariable sum of gradients is kept after this replacement.

Revised: To guarantee that the gradients are not reversed between layers, we should expect h ( t ) > 0 always hold when t > 0 .

Original: Therefore, we choose h ( t ) =
Revised: Therefore, we choose h ( t ) =

Original: e − t τgrad to simplify the calculation, where τ grad is a tunable parameter.
Revised: e − t τgrad to simplify the calculation, where τ grad is a tunable parameter.

Original: Notice that the function h ( t ) is only used in backward propagation, which means the spike responsekernel in the forward propagation is not necessarily the integral of h ( t ) .
Revised: Notice that the function h ( t ) is only used in backward propagation, which means the spike response kernel in the forward propagation is not necessarily the integral of h ( t ) .

Original: 3.4 Overall Learning Rule
Revised: 3.4 Overall Learning Rule

Original: The loss function we use in this work is the counting loss function, which has the form L=1 N (cid:80) N out i =(cid:18)
Revised: The loss function we use in this work is the counting loss function, which has the form L =

Original: [Equation]
Revised: [Equation]

Original: (cid:16) N targeti − (cid:82) [Equation] s i ( t ) dt (cid:17)(cid:19) 2 , where N out is the number of output neurons and equalsto the number of classes, s i ( t ) represents the spike train emitted by neuron i .
Revised: 2 , where N out is the number of output neurons and equals to the number of classes, s i ( t ) represents the spike train emitted by neuron i .

Original: Besides, N targeti is thetarget of the spike number outputted by neuron i and typically we set N targeti larger when i is thecorrect answer.
Revised: Besides, N targeti is the target of the spike number outputted by neuron i and typically we set N targeti larger when i is the correct answer.

Original: During the learning process, the gradient is first propagated from the loss function to the firing time ofeach spike from the last layer to the first layer.
Revised: During the learning process, the gradient is first propagated from the loss function to the firing time of each spike from the last layer to the first layer.

Original: The formula for this stage is (please refer to Appendixfor the detailed deduction):∂L
Revised: The formula for this stage is (please refer to Appendix for the detailed deduction):

Original: [Equation]
Revised: [Equation]

Original:  where t k,next ( s ( l ) i ) is the firing time of the first spike emitted by neuron i after time t m .
Revised: ) where t k,next ( s ( l ) i ) is the firing time of the first spike emitted by neuron i after time t m .

Original: After this, the gradient to weights in each layer is calculated by summing up the multiplication of thegradients of spike firing times in the same layer and the derivative of weights with respect to spikefiring times.
Revised: After this, the gradient to weights in each layer is calculated by summing up the multiplication of the gradients of spike firing times in the same layer and the derivative of weights with respect to spike firing times.

Original: The learning rule for this stage is
Revised: The learning rule for this stage is

Original: [Equation]
Revised: [Equation]

Original:  Experiments
Revised:  Experiments

Original: In this section, we validate the effectiveness of our method on MNIST [46], Fashion-MNIST [47],
Revised: In this section, we validate the effectiveness of our method on MNIST [48], Fashion-MNIST [49],

Original: N-MNIST [48], CIFAR10 [49], and CIFAR100 [49] datasets.
Revised: N-MNIST [50], CIFAR10 [51], and CIFAR100 [51] datasets.

Revised: This section is organized as follows:

Original: We first introduce the training details, then evaluate the performance of our algorithm and compare it with the state-of-the-art event-drivenlearning approaches.
Revised: We first introduce the training details, then evaluate the performance of our algorithm and compare it with the state-of-the-art event-driven learning approaches.

Original: At last, we conduct ablation studies to illustrate the effectiveness of our proposedmodules.
Revised: At last, we conduct ablation studies to illustrate the effectiveness of our proposed modules.

Original: More details of the configurations can be found in the Appendix.
Revised: More details of the configurations can be found in the Appendix.

Original: 4.1 Training Details
Revised: 4.1 Training Details

Original: Initialization: When training in an event-driven fashion, gradient information is only carried byspikes.
Revised: Initialization: When training in an event-driven fashion, gradient information is only carried by spikes.

Original: Therefore, the gradient information will be completely blocked by a layer when there areno spikes in that layer.
Revised: Therefore, the gradient information will be completely blocked by a layer when there are no spikes in that layer.

Original: To solve this problem, we start with layers of arbitrarily initialized weightsand scale them by certain multiples, which can make the average firing rate to be a certain numberfor each layer.
Revised: To solve this problem, we start with layers of arbitrarily initialized weights and scale them by certain multiples, which can make the average firing rate to be a certain number for each layer.

Original: We obtain these multiple parameters by binary search and this strategy works well inpractice.
Revised: We obtain these multiple parameters by binary search and this strategy works well in practice.

Original: Supervisory signals: Another problem we face is that  neurons corresponding to certain classes donot fire anymore after certain training, which makes  recognizing those classes correctly impossible inthe following epochs.
Revised: Supervisory signals: Another problem we face is that output neurons corresponding to certain classes do not fire anymore after certain epochs of training. This problems makes corresponding gradients difficult to propagate in the network, further leading to these neurons no longer fire afterwards, resulting recognizing those classes correctly impossible in the following epochs.

Original: To address this problem, we utilize supervisory signals.
Revised: To address this problem, we utilize supervisory signals.

Original: For each neuron in theoutput layer corresponding to the ground-truth label, we force it to fire at the end of the simulation.
Revised: For each neuron in the output layer corresponding to the ground-truth label, we force it to fire at the end of the simulation.

Original: Experiment settings: In our experiments, we use the real-valued spike current representing the pixelintensities of the image as inputs.
Revised: Experiment settings: In our experiments, we use the real-valued spike current representing the pixel intensities of the image as inputs.

Original: We list the network architectures each work uses  in table 2.
Revised: We list the network architecture each work uses and the accuracy they achieves on each dataset in table 2.

Original: Noticethat the output layer is, by default, a fully-connected layer containing the same number of neurons asthe number of classes in the dataset, and omitted from the architecture representation.
Revised: Notice that the output layer is, by default, a fully-connected layer containing the same number of neurons as the number of classes in the dataset, and omitted from the architecture representation.

Original: We run allexperiments on a single Nvidia A100 GPU.
Revised: We run all experiments on a single Nvidia A100 GPU.

Original: 4. Comparison with the State-of-the-Art
Revised: 4. Comparison with the State-of-the-Art

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: CIFARNet: 128C3-256C3-P2-512C3-P2-1024C3-512C3-1024-

Original: [Table]

Original: VGG11: 128C3-128C3-P2-256C3-256C3-256C3-P2-512C3-512C3-512C3-P2-

Original: [Table] Tab. 2 reports the accuracies of the proposed method and other comparing methods.
Revised: Tab. 2 reports the accuracies of the proposed method and other comparing methods.

Original: The performanceof our algorithm is lower than TSSL-BP by 0.06% on the MNIST dataset and 0.01% on the N-MNISTdataset.
Revised: The performance of our algorithm is lower than TSSL-BP by 0.06% on the MNIST dataset and 0.01% on the N-MNIST dataset.

Original: However, the output of their network is real-valued postsynaptic currents while the outputof our network is binary spikes.
Revised: However, the output of their network is real-valued postsynaptic currents while the output of our network is binary spikes.

Original: In addition, they use RNN-like gradients to assist learning.
Revised: In addition, they use RNN-like gradients to assist learning.

Original: On theremaining datasets, we have achieved state-of-the-art performance among these works with temporalgradients.
Revised: On the remaining datasets, we have achieved state-of-the-art performance among these works with temporal gradients.

Original: For the Fashion-MNIST dataset, our algorithm performs 0.45% higher than the previous
Revised: For the Fashion-MNIST dataset, our algorithm performs 0.45% higher than the previous

Original: SOTA.
Revised: SOTA.

Original: On the CIFAR10 dataset, we outperform the existing SOTA by 0.47%.
Revised: On the CIFAR10 dataset, we achieve 92.45% accuracy with a 14-layer SEW-Resnet and [Table] 512C [Table]

Revised: VGG

Original: For the CIFAR100 dataset, we are the first work to successfully train SNNs with time-based gradients in an event-drivenfashion.
Revised: [Table] 128C3-128C3-P [Table] 256C

Revised: [Table]

Revised: 256C3-256C

Revised: [Table]

Revised: P

Revised: [Table]

Revised: 512C

Revised: [Table]

Revised: 512C3-512C

Revised: [Table]

Revised: P

Revised: [Table]

Revised: 92.10% with VGG11, which are all better than the current SOTA, 91.41%.

Revised: For the CIFAR100 dataset, we are the first work to successfully train SNNs with time-based gradients in an event-driven fashion.

Original: We have achieved a performance of 63.97% on this dataset.
Revised: We have achieved a performance of 63.97% on this dataset.

Original: 4.3 Ablation Studies
Revised: 4.3 Ablation Studies

Original: To show the effect of our proposed modules, we conduct ablation experiments on the CIFAR10dataset.
Revised: To show the effect of our proposed modules, we conduct ablation experiments on the CIFAR dataset.

Original: Specifically, two proposed components are taken into consideration: (1) As mentioned in
Revised: Specifically, two proposed components are taken into consideration: (1) As mentioned in

Original: Section 3.3, we compare the proposed gradient assignment functions h ( t ) =
Revised: Section 3.3, we compare the proposed gradient assignment functions h ( t ) =

Original: e − βt in (12) with thecommonly used one h ( t ) =
Revised: e − βt in (12) with the commonly used one h ( t ) =

Original: dϵ ( t ) dt .
Revised: dϵ ( t ) dt .

Original: (2) We compare the results of three different types of poolinglayers (average pooling, adjusted average pooling and max pooling) mentioned in Section 3.2.
Revised: (2) We compare the results of three different types of pooling layers (average pooling, adjusted average pooling and max pooling) mentioned in Section 3.2.

Original: Wehave tried all combinations of gradient assignment functions and pooling layers.
Revised: We have tried all combinations of gradient assignment functions and pooling layers.

Original: The test accuracy ofthese different settings is shown in Tab. 3.
Revised: The test accuracy of these different settings is shown in Tab. 3.

Original: [Table]
Revised: [Table]

Original: [Figure]

Original: The results in Tab. 3 meets our expectation.
Revised: The results in Tab. 3 meets our expectation.

Original: For the pooling layer, maxpooling and adjusted average poolinghave much better performance thanthe average pooling.
Revised: For the pooling layer, max pooling and adjusted average pooling have much better performance than the average pooling.

Original: This accordswith the conclusion in Section 3.2 thatpooling layers keeping the property ofinvariant sum of gradients are betterthan those that do not.
Revised: This accords with the conclusion in Section 3.2 that pooling layers keeping the property of invariant sum of gradients are better than those that do not.

Original: The proposed gradient assignmentfunction h ( t ) =
Revised: The proposed gradient assignment function h ( t ) =

Original: e − βt is also betterthan the commonly used one h ( t ) =
Revised: e − βt is also better than the commonly used one h ( t ) =

Original: dϵ ( t ) dt for all three types of pooling layers.
Revised: dϵ ( t ) dt for all three types of pooling layers.

Original: In addition, as shown in Fig. 5,h ( t ) =
Revised: In addition, as shown in Fig. 5, h ( t ) =

Original: e − βt converges faster thanh ( t ) =
Revised: e − βt converges faster than h ( t ) =

Original: dϵ ( t ) dt in early stage.
Revised: dϵ ( t ) dt in early [Figure] epoch [Figure]

Revised: 0 .a

Revised: cc u r a c y exponential gradient adjusted avg pooling exponential gradient avg pooling exponential gradient max pooling original gradient adjusted avg pooling original gradient avg pooling

Original:   Conclusion and Discussion
Revised: [Figure]  Conclusion and Discussion

Original: In this work, we analyze the commonly-used SNN temporal backpropagation training approach andfind that it follows the gradient assignment rule.
Revised: In this work, we analyze the commonly-used SNN temporal backpropagation training approach and find that it follows the gradient assignment rule.

Original: We also find the average pooling layer does not obeythis rule while the max pooling layer does.
Revised: We also find the average pooling layer does not obey this rule while the max pooling layer does.

Original: We show that the direction of the temporal gradient will bereversed when the spike kernel is decreasing and avoid it with an increasing kernel in backpropagation.
Revised: We show that the direction of the temporal gradient will be reversed when the spike kernel is decreasing and avoid it with an increasing kernel in backpropagation.

Original: Our algorithm achieves state-of-the-art performance on CIFAR10 among time-based SNN learningapproaches and successfully learns the parameters of SNN on CIFAR100 for the first time.
Revised: Our algorithm achieves state-of-the-art performance on CIFAR10 among time-based SNN learning approaches and successfully learns the parameters of SNN on CIFAR100 for the first time.

Original: Compared with RNN-like methods, the proposed event-based learning algorithm has a lower computational cost and memory occupation when there are many time steps.
Revised: Compared with RNN-like methods, the proposed event-based learning algorithm has a lower computational cost and memory occupation when there are many time steps.

Original: Besides, our algorithm alsodoes not need bias between layers.
Revised: Besides, our algorithm also does not need bias between layers.

Original: Meanwhile, gradient propagation between spikes instead of timesteps can mitigate the gradient explosion/vanishing problem along the time axis.
Revised: Meanwhile, gradient propagation between spikes instead of time steps can mitigate the gradient explosion/vanishing problem along the time axis.

Original: However, there is still a gap between event-driven backpropagation and biological plausible learning,since event-driven backpropagation processes the spike train in reverse time, which conflicts with theonline learning in the real world and desires for future research.Checklist For all authors...
Revised: However, there is still a gap between event-driven backpropagation and biological plausible learning, since event-driven backpropagation processes the spike train in reverse time, which conflicts with the online learning in the real world and desires for future research.

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?

Original: [Yes]

Original: (b) Did you describe the limitations of your work?

Original: [Yes](c)

Original: Did you discuss any potential negative societal impacts of your work?

Original: [Yes](d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes](b) Did you include complete proofs of all theoretical results?

Original: [Yes]3.

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?

Original: [Yes]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original:  [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No] We use identical seeds to maximize the reproducibility.
Revised:  Acknowledgements

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators? [Yes] (b) Did you mention the license of the assets? [No] The licence is accessible in the codes’
Revised: This work was supported by the National Natural Science Foundation of China Grants 62176003 and 62088102.

Original: homepage.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?

Original: [No] No need.

Original: The datasets we used in this paper are public.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?

Original: [No] No need.

Original: The datasets we used in this paper are

Original: public.

Original: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?

Original: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?

Original: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?

Original: [N/A]