Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation
Authors: Nitin Rathi
Gopalakrishnan Srinivasan
Priyadarshini Panda
Kaushik Roy
Venue:  (2020)
Keywords: spiking neural networks
ann-snn conversion
spike-based backpropagation
imagenet

Abstract: Spiking Neural Networks (SNNs) operate with asynchronous discrete events (or spikes) which can potentially lead to higher energy-efficiency in neuromorphic hardware implementations. Many works have shown that an SNN for inference can be formed by copying the weights from a trained Artificial Neural Network (ANN) and setting the firing threshold for each layer as the maximum input received in that layer. These type of converted SNNs require a large number of time steps to achieve competitive accuracy which diminishes the energy savings. The number of time steps can be reduced by training SNNs with spike-based backpropagation from scratch, but that is computationally expensive and slow. To address these challenges, we present a computationally-efficient training technique for deep SNNs. We propose a hybrid training methodology: 1) take a converted SNN and use its weights and thresholds as an initialization step for spike-based backpropagation, and 2) perform incremental spike-timing dependent backpropagation (STDB) on this carefully initialized network to obtain an SNN that converges within few epochs and requires fewer time steps for input processing. STDB is performed with a novel surrogate gradient function defined using neuron's spike time. The weight update is proportional to the difference in spike timing between the current time step and the most recent time step the neuron generated an output spike. The SNNs trained with our hybrid conversion-and-STDB training perform at $10{\times}{-}25{\times}$ fewer number of time steps and achieve similar accuracy compared to purely converted SNNs. The proposed training methodology converges in less than $20$ epochs of spike-based backpropagation for most standard image classification datasets, thereby greatly reducing the training complexity compared to training SNNs from scratch. We perform experiments on CIFAR-10, CIFAR-100 and ImageNet datasets for both VGG and ResNet architectures. We achieve top-1 accuracy of $65.19\%$ for ImageNet dataset on SNN with $250$ time steps, which is $10{\times}$ faster compared to converted SNNs with similar accuracy.

Published as a conference paper at ICLR 2020
ENABLING DEEP SPIKING NEURAL NETWORKS WITH HYBRID CONVERSION AND SPIKE TIMING DEPENDENT BACKPROPAGATION
Nitin Rathi', Gopalakrishnan Srinivasan', Priyadarshini Panda & Kaushik Roy!
School of Electrical and Computer Engineering, Purdue University DDpartment of Electrical Engineering, Yale University frathi2, srinivg purruuueeeuuuu edu, priya. -aadaayyale.dd: kaushiképurdue. edu
ABSTRACT
Spiking Neural Networks (SNNs) operate with asynchronous discrete events (or spikes) which can potentially lead to higher energy-effciency in neuromorphic hardware implementations. Many works have shown that an SNN for inference can be formed by copying the weights from trained Artificial Neural Network (ANN) and setting the firing threshold for each layer as the maximum input received in that layer. These type of converted SNNs require large number time steps to achieve competitive accuracy which diminishes the energy savings. The number of time steps spike-based backpropagation from scratch, but that is computationally expensive and slow. To address these challenges, we present computationally--fiiiii training technique for deep SNN:" We propose a hybrid training methodology: take a converted SNN and use weights and thresholds as an initialization step for spike-based backpropagation, and 2) perform incremental dependent backpropagation (STDB) on this carefully initialized network to obtain SNN that converges within few epochs and requires fewer time steps for input processing. STDB is performed with a novel surrogate gradient function defined using neuron's spike time. The weight update is proportional to the difference in spike timing between the current time step and the most recent time step the neuron generated output spike. The SNNs trained with hybrid conversion-and-93 time steps and achieve similar accuracy compared to purely converted SNNs. The proposed training methodology converges in less than 20 epochs of spike-based backpropagation for most standard image classification datasets, thereby greatly reducing the training complexity compared to training SNNs from scratch. We perform experiments on CIFAR-10, CIFAR- 100 and ImageNet datasets for both VGG and ResNet architectures. We achieve top-1 accuracy of 65.19% for ImageNet dataset on SNN with 250 time steps, which 10x faster compared to converted SNNs with similar accuracy.
be reduced by training SNNs with
training perform at 10x -5x fewer number
INTRODUCTION
In recent years, Spiking Neural Networks (SNNs) have shown promise towards enabling low-power machine intelligence with event-driven neuromorphic hardware. Founded bio-plausibility. the neurons in an SNN compute and communicate information through discrete binary events (or spikes) significant shift from the standard artificial neural networks (ANNs), which process data in real-valued (or analog) manner. The binary all-or-nothing spike- communication combined with sparse temporal processing precisely make SNNs low-power alternative to conventional ANNs. With all its appeal for power efficiency, training SNNs still remains a challenge. The discontinuous and non diffeefeeretiablee nature of spiking neuron (generally, modeled

---

Published as a conference paper at ICLR 2020
as inr--interateadd---eee (LIF), or integrate-and-fitt (IF)) poses difficulty to conduct gradient descent based backpropagation. Practically, SNNs still lag behind ANNs, in terms of performance or accuracy, in traditional learning tasks. Consequently, there has been several works over the past few years that propose different learning algorithms or learning rules for implementing deep convolutional SNNs for complex visual recognition tasks (Wu et al. [019 Hunsberger & Eliasmith Cao et al. ) Of all the techniques, conversion from ANN-10-SNN (Dichl et al, 2015 Sengupta et al. 2019, Hunsberger & Eliasmith 2015) has yielded state-of-the-ar accuracies matching deep ANN performance for Imagenet dataset on complex architectures (such as, VGG (Simonyan & Zisserman 2014) and ResNet (He et al. al.] (016)) ). In conversion, we train an ANN with ReL.U neurons using gradient descent and then convert the ANN to an SNN with IF neurons by using suitable threshold balancing (Sengupta et al,, 2019). But, SNNs obtained through conversion incur large latency of 2000 2550 time steps (measured as total number of time steps required to process a given input image The term 'time step' defines an unit of time required to process a single input spike across all layers and represents the network latency. The large latency translates to higher energy consumption during inference, thereby, diminishing the efficiency improvements of SNNs over ANNs. To reduce the latency, spike-based backpropagation rules have been proposed that perform end-to-end gradient descent training on spike data. In spike-based backpropagation methods, the non-differnntiabiliyy of the spiking neuron is handled by either approximating the spiking neuron model as continuous and differentiable (Huh & Sejnowskil (018) or by defining a surrogate gradient as continuous approximation of the real gradient (Wu et al., 2018 Bellec iet al. 2018, Neitii et al. 2019). Spike- SNN training reduces the overall latency by ~10x (for instance, 200 250 time steps required to process an input (Lee et al., 2019) but requires more training effort (in terms of total training iterations) than conversion approaches. A single feed-forward pass in ANN corresponds to multiple forward passes in SNN which is proportional to the number of time steps. In spike-based backpropagation, the backward pass requires the gradients to be integrated over the total number of time steps that increases the computation and memory complexity. The multiple-iteratien iteration training effort with exploding memory requirement (for backward pass computations) has limited the applicability of spike-bass based backpropagation methods to small datasets (like CIFAR10) on simple few-layered convolutional architectures.
2015
2016
In this work, we propose a hybrid training technique which combines ANN-SNN conversion and spike-based backpropagation that reduces the overall latency well decreases the training effort for convergence. We use ANN-SNN conversion as an initialization step followed by spike-based backpropagation incremental training (that converges to optimal accuracy with few epochs due to the precursory initialization). Essentially, our hybrid approach of taking a converted SNN and incrementally training it using backpropagation yields improved energy-efficiency as well as higher accuracy than a model trained from scratch with only conversion or only spike-based backpropagation.
In summary, this paper makes the following contributions:
We introduce a hybrid computationalll--ffiiiienn training methodology for deep SNNs. We use the weights and firing thresholds of an SNN converted from an ANN as the initialization step for spike- backpropagation. We then train this initialized network with based backpropagation for few epochs to perform inference at a reduced latency time steps.
We propose a novel spike time-dependent backpropagation (STDB, a variant of standard spike-based backpropagation) that computes surrogate gradient using neuron's spike time. The parameter update is triggered by the occurrence of spike and the gradient is computed based on the time difference between the current time step and the most recent time step the neuron generated an output spike. This is motivated from the Hebb's principle which states that the plasticity of synapse is dependent on the spiking activity of the neurons connected to the synapse.
Our hybrid approach with the novel surrogate gradient descent allows training of large-scale SNNs without exploding memory required during spike- backpropagation. We evaluate our hybrid approach on large SNNs (VGG, ResNet-like architectures) on
PSNNs process Poisson rate-coded input spike trains, wherein, each pixel in an image is converted to based spike train with the spiking frequeney proportional the pixel value
2

---

Published as a conference paper at ICLR 2020
Imagenet, CIFAR datasets and show near aso-aceuracy compared similar ANNs and converted SNNs at lower compute cost and energy.
2 SPIKE TIMING DEPENDENT BACKPROPAGATION (STDB)
In this section, we describe the spiking neuron model, derive the equations for the proposed surrogate gradient based learning, present the weight initialization method for SNN, discuss the constraints applied for ANN-SNN conversion, and summarize the overall training methodology.
53
0.00
do du
026
32
0.1
2.1 LEAKY INTEGRATE AND FIRE (LIF) NEURON MODEL.
0.14
23
40
Time from last spike (at)
The neuron model defines the dynamics of the neuron's internal state and the trigger for it to generate a spike. The differential equation
Figure 1: Surrogate gradient of the spiking neuron activation function (Eq, [1].
0.3, 3 = 0.01. The gradient is computed for each neuron and At defines the time difference between current simulation time and the last spike time of the neuron. For example, if a neuron spikes at f, = 12 its gradient will be maximum at = 12( = 0) and gradually decrease for later time steps. If the same neuron spikes later at t, = 24 its previous spike history will be overwritten and the gradient computation for = 24 onward will only consider the most recent spike. This avoids the overhead of storing all the spike history in memory.
aU dt = --U Urest) RI
(1)
is widely used to characterize the leaky-integrate-andd and-ee---n--iir (LIF) neuron model where, U is the internal state of the neuron referred as the membrane potential, Urest is the resting potential, R and I the input resistance and the current, respectively. The above equation is valid when the membrane potential is below the threshold value (V). The neuron geneartes an output spike when U>V
and U is reduced to the reset potential. This representation is described in continuous domain and more suitable for biological simulations. We modify the equation to be evaluated in discrete manner in the Pytorch framework (Wu et al. 2018). The iterative model for a single post-neuron is described by
ip = Au, + Σ wijof vo; 1-1
(2)
o.-1 --1=f1 u fuf-1
[1, - 10 otherwise
(3)
where u is the membrane potential, subscript i and represent the post- and pre-neuron, respectively, superscript t the time step, À is constant (< 1) responsible for the leak membrane potential, w is the weight connecting the pre- and post-ncuron, o is the binary output spike, and v is the firing threshold potential. The right hand side of Equation has three terms: the first term calculates the leak in the membrane potential from the previous time step, the second term integrates the input from the previous layer and adds it to the membrane potential, and the third term which is outside the summation reduces the membrane potential by the threshold value if spike is generated. This known as soft reset as the membrane potential is lowered by v compared to hard reset where the membrane potential is reduced to the reset value. Soft reset enables the spiking neuron carry forward the excess potential above the firing threshold to the following time step, thereby minimizing information loss.
3

---

Published as a conference paper at ICLR 2020
Algorithm 1 ANN-SNN conversion: initialization of weights and threshold voltages
Algorithm Initialize the neuron parameters. Membrane potential (U), last spike time (S), dropout mask (M). The initialization is performed once for every mini-batch.
Input: Trained ANN model (A), SNN model (N), Input (X)
Copy ann weights to snn for [=] to L do
Input: Input(XX), network model(N)
b_size X.b_size
NWW A.
X.height
end
X.width for /=/ to do if issintance Ni, Conv) then
Initialize threshold voltage
V < [0. ... -0
for /= to L-1 do
Ui = zeros(b Ni.out, h, w) S ones(b_size, Ni.ou,, h, w)
8 6- 0
for [=/ to to do
(-1000)
O 1-++Pp PoissonGenerator([
end
for k=/ to if k < then
else if isintance Ni. Linear) then
Ul zeros(b size, N1.out) S ones(b_size, N(.ouf) * (-1000)
// Forward (Algorithm
end
end else
else if isintance Dropout) then // Generate the dropout map that will be fixed for all time steps end
// Pre-nonlinearity (A) A + Nr(O
Mi = Nioones(Ui-..shpee
if' max(A) > v then V + max (A)
else if isiittnnee (ssitttnnee,,..uugoool then
end
// Reduce the width and height after
end
average pooling layer
end
h h/ ////eeeel_saa
end
w w/ /ee//kerellazz
VII +
end
end
end
2.2 SPIKE TIMING DEPENDENT BACKPROPAGATION (STDB) LEARNING RULE
The neuron dynamies (Equation 2 show that the neuron's state at a particular time step recurrently depends on its state in previous time steps. This introduces implicit recurrent connections in the network (Neftci et al. (019). Therefore, the learning rule has to perform the temporal credit assignment along with the spatial credit assignment. Credit assignment refers to the process of assigning credit or blame to the network parameters according to their contribution to the loss function. Spatial credit assignment identifies structural network parameters (like weights), whereas temporal credit assignment determines which past network activities contributed to the loss function. Gradient-descent learning solves both credit assignment problem: spatial credit assignment is performed by distributing error spatially across all layers using the chain rule of derivatives, and temporal credit assignment is done by unrolling the network in time and performing backpropagation through time (BPTT) using the same chain rule of derivatives (Werbos et al., 1990). In BPTT, the network is unrolled for all time steps and the final output is computed as the sum of outputs from each time step. The loss function is defined on the summed output.
The dynamics of the neuron in the output layer is described by Equation (4). where the leak part is removed (A 1) and the neuron only integrates the input without firing. This eliminates the difficulty of defining the loss function on spike count (Lee al. 2019).
th
+
WiyOj
(4)
The number of neurons in the output layer is the same as the number of categories in the classification task. The output of the network is passed through softmax layer that outputs a probability distribution. The loss function is defined the cross-entropy between the true output and the
4

---

Published as a conference paper at ICLR 2020
network's 's predicted distribution.
L = Σ y.oog(p))
(5)
P: Σk=1 e
(6)
L is the loss function, y the true output, the prediction, T the total number of time steps, u² the accumulated membrane potential of the neuron in the output layer from all time steps, and N the number of categories in the task. For deeper networks and large number of time steps the truncated version of the BPTT algorithm the loss is computed at some time step t' before T based the potential accumulated till t'. The loss backpropagated to all layers and the loss gradients are computed and stored. At this point, the history of the computational graph is cleaned to memory. The subsequent computation of to get
used to avoid memory issues. In the truncated version
loss gradients at later time steps (2f', 3t', ...I) are summed together with the gradient the final gradient. The optimizer updates the parameters T based the sum of the gradients. Gradient descent learning has the objective of minimizing the loss function. This is achieved by backpropagating the error and updating the parameters opposite the direction of the derivative. The derivative of the loss function w.r.t. to the membrane potential of the neuron in the final layer is described by,
aL du.
= V/s
To compute the gradient current time step, the membrane potential at last time step (u/- in Equation 4) is considered as an input quantity. Therefore, gradient descent updates the network parameters Wiy of the output layer as,
W6 Wis - nA
(8)
Σ OL OW
Σ ÖL duf duf OW!
ØL Duf
du! T
AWG =
(9)
OW!
where is the learning rate, and W represents the copy of the weight used for computation at time step t. In the output layer the neurons do not generate spike, and hence, the issue of non-differentiabbiiy is not encountered. The update of the hidden layer parameters is described
by,
AW Σ
ÔL OW! t
Σ ÖL dot duf do dut Ow?
(10)
where of is the thresholding function (Equation whose derivative w.r.t of is zero everywhere and not defined at the time of spike. The challenge of discontinuous spiking nonlincarity is resolved by introducing a surrogate gradient which is the continuous approximation of the real gradient.
do Ouf
ae -BAr
(11)
where a and are constants, At is the time difference between the current time step (t) and the last time step the post-neuron generated spike (t.). It is an integer value whose range is from zero to the total number of time steps (T)).
Al (1- AKT, e
(12)
The values of a and 3 are selected depending on the value of T. If T is large B is lowered to reduce the exponential decay so a spike can contribute towards gradients for later time steps. The value of is also reduced for large T' because the gradient can propagate through many time steps. The gradient is summed at each time step and thus large a may lead to exploding gradient. The surrogate gradient can be pre-computed for all values of At and stored in a look-up table for faster computation. The parameter updates are triggered by the spiking activity but the error gradients are still non-zero for time steps following the spike time. This enables the algorithm avoid the 'dead
5

---

Published as a conference paper at ICLR 2020
Algorithm 3 Training an SNN with surrogate gradient computed with spike timing. The network is composed of L layers. The training proceeds with mini-batch size (balch_size)
Input: Mini-batch of input (X) target (Y) pairs, network model (N), initial weights (W), threshold voltage (V)
U, M =
[Algorit [Algorithm]
// Forward propagation
for /= to T do
O PoissonnGenerator/a
for /= to 1.-/ do
isintance (Ni, [Cone, Linear)) then
// accumulate the output of previous layer in U, soft reset when spike occurs Vi O
01 = XU, W
1/ generate the output (+1) if exceeds V
0 STDD Vi, i)
1/ store the latest spike times for each neuron
$ = 1] =
end
else if isintance Nr, Pool) then Of = N==;
end
else if iisintannee Ni, Dropout) then
Of = 0/ M2
end
end
= -U W20
end
// Backward Propagation
Compute 8L from the cross-entropy loss function using BPTT BÜr for 1=7 to / do for |=L-] to I do
Compute 80
based on if N₁ linear, conv, pooling, etc.
aL |0?
aL 80% 80 oU.
8L 80
ae-888
end
end
neuron' problem, where no learning happens when there is no spike. Fig. D shows the activation gradient for different values of At. the gradient decreases exponentially for neurons that have not been active for a long time. In Hebbian models of biological learning, the parameter update is plasticity (STDP)
activity dependent. This is experimentally observed in spike-timing-depentee learning rule which modulates the weights for pair of neurons that spike within a time window (Song et al. 2000
3 SNN WEIGHT INITIALIZATION
A prevalent method of constructing SNNs for inference is ANN-SNN conversion (Diehl al. 2015 Sengupta et al. 2019). Since the network is trained with analog activations it does not suffer from the non-differentiablit issue and can leverage the training techniques of ANNs. The conversion process has major drawback: it suffers from long inference latency (~22500 time steps) as mentioned in Section [ As there no provision optimize the parameters after conversion based on spiking activity, the network can not leverage the temporal information of the spikes. In this work, we propose use the conversion process as an initialization technique for STDB. The converted weights and thresholds serve as good initialization for the optimizer and the STDB learning rule is applied for temporal and spatial credit assignment.

---

Published as a conference paper at ICLR 2020
Algorithm exxlainn the ANN-SNN conversion process, The threshold voltages in SNN needs to be adjusted based on the ANN weights. Sengupta et al. (2019) showed two ways to achieve this: weight-normalizztion and fureshold-balameiin a normalization factor and threshold is set to 1, whereas in threshold-balancing the weights are unchanged and the threshold is set to the normalization factor. Both have similar effect and either method and the normalization
In weightt-oommalization the weights are scaled by
can be used to set the threshold. We employ the threshold-balancin factor is calculated as the maximum output of the corresponding convolution/inear layer in SNN. The maximum is calculated over a mini-batch of input for all time steps.
There are several constraints imposed on training the ANN for the conversion process (Sengupta et al. 2019] Dichl et al. 2013). The neurons are trained without the bias term because the bias term in SNN has an indirect effect on the threshold voltage which increases the difficulty of threshold balancing and the process becomes more prone to conversion loss. The absence bias term eliminates the use of Batch Normalization (loffe & Szegedy 2015) a regularizer in ANN since it biases the input of each layer to have zero mean. As an alternative, Dropout (Srivastava et al. 2014) is used for both ANN and SNN training. The implementation of
Input Data
[333
Pre Processing
ReLUMi
Drapout
Corv (3x3, /1)
Threshold Balancing
ReLU/F
Drapout
regularizer
(3x3,
Dropout in SNN is further discussed in Section The pooling operation is widely used in ANN to reduce the convolution map size. There are two popular variants: max pooling and average pooling (Boureau et al., 2010). Max (Average) pooling outputs the maximum (average) value in the kernel space of the neuron's activations. In SNN, the activations are binary and performing max pooling will result significant information loss for the next layer, we adopt the average pooling for both ANN and SNN (Dichl et al. 2015).
ReLU
AvgP Pool
Conv (3x3, /1)
Basic Block
ReLL
Threshoid -1
Drepout
(3x3,
Adaition
ReLUMF
4 NETWORK ARCHITECTURES
In this section, we describe the changes made the VGG Figure 2: Residual architecture for (Simonyan & Zisserman (014) and residual architecture (He) SNN
al. 2016) for hybrid learning and discuss the process of threshold computation for both the architectures.
4.1 VGG ARCHITECTURE
The threshold balaneing is performed for all layers except the input and output layer in VGG architecture. For every hidden convolution/ineer layer the maximum inpuf) to the neuron is computed over all time steps and set threshold for that layer. The threshold assignment is done sequentially as described in Algorithm [ The threshold computation for all layers can not be performed in parallel (in one forward pass) because in the forward method (Algorithm we need the threshold at each time step to decide if the neuron should spike or not.
4.2 RESIDUAL ARCHITECTURE
Residual architectures introduce shortcut connections between layers that are not next to each other. In order to minimize the ANN-SNN conversion loss various considerations were made by (Sengupta et al. 2019). The original residual architecture proposed by He et al.) (2016) uses an initial convolution layer with wide kernel (7 (7,7, stride 2). For conversion. this is replaced by a pre- prepprocessing block consisting of series of three convolution layer (3x3, stride 1) with dropout layer in between (Fig. _ The threshold balancing mechanism is applied to only these three layers and the layers in the basic block have unity threshold.
input a neuron is the weighted sum of spkies from pre-neurons Σ, Mig0j
7

---

Published as a conference paper at ICLR 2020
Table 1: Classification results (Top-1) for CIFARIO, CIFAR100 and ImageNet data sets. Column-1 shows the network architecture. Column-2 shows the ANN accuracy when trained under the constraints as described in Section 3 Column-3 shows the SNN accuracy for T 2500 when converted from a ANN with threshold balancing. Column-4 shows the performance of the same converted SNN with lower time steps and adjusted thresholds. Column-5 shows the performance after training the Column-4 network with STDB for less than 20 epochs.
Architecture
ANN
ANN-SNN Conversion (T' -2000
ANN-SNN Conversion (reduced time steps)
Hybrid Training
(ANN-SNN Conversion + STDB)
CIFARIO
VGG5
87.88%
87.64%
84.56% (T' 75)
86.91% (T = 75)
VGG9
91.45%
90.98%
87.31% (T 100)
90.54% (T 100)
VGG16
92.81%
92.48%
90.2% (T 100)
91.13% (T 100)
ResNet8
91.35%
91.12%
89.5% (T 200)
91.35% (T =200)
ResNet20
93.15%
92.94%
91.12% (T 250)
92.22% (T =250)
CIFAR100
VGG11
71.21%
70.94%
65.52% (T 125)
67.87% (T 125)
ImageNet
ResNet34
70.2%
65.1%
56.87% (T = 250)
61.48% (T 2200)
VGG16
69.35%
68.12%
62.73% (T = 250)
63.19% (T 250)
5
OVERALL TRAINING ALGORITHM
Algorithm defines the process to initialize the parameters (weights, thresholds) of SNN based on ANN-SNN conversion. Algorithm and show the mechanism of training the SNN with STDB Algorithm initializes the neuron parameters for every mini-batch, whereas Algorithm performs the forward and backward propagation and computes the credit assignment. The threshold voltage for all neurons in layer is same and is not altered in the training process. For each dropout layer we initialize a mask (M) for every mini-batch of inputs. The function of dropout is to randomly drop certain number of inputs in order avoid overfitting. In case of SNN, inputs are represented as a spike train and we want to keep the dropout units same for the entire duration the input. Thus, random mask (M) is initialized (Algorithm 2 for every mini-batch and the input is element-wise multiplied with the mask to generate the output of the dropout layer (Lee et al. 2019). The Poisson generator function outputs Poisson spike train with rate proportional to the pixel value in the input. A random number is generated at every time step for each pixel in the input image. The random number compared with the normalized pixel value and if the random number less than the pixel value an output spike is generated. This results in a Poisson spike train with rate equivalent to the pixel value if averaged over long time. The weighted sum of the input accumulated in the membrane potential of the first convolution layer. The STDB function compares the membrane potential and the threshold of that layer to generate an output spike. The neurons that output a spike their corresponding entry in S is updated with current time step (t). The last spike time is initialized with a large negative number (Algorithm 2 to denote that at the beginning the last spike happened at negative infinity time. This is repeated for all layers until the last layer. For last layer the inputs are accumulated over all time steps and passed through a softmax layer to compute the multi-class probability. The cross-entropy loss function is defined the output of the softmax and the weights are updated by performing the temporal and spatial credit assignment according to the STDB rule.
2
6 EXPERIMENTS
We tested the proposed training mechanism on image classification tasks from CIFAR (Krizhevsky et al. 2009) and ImageNet (Deng al. 2009) datasets. The results are summarized in Table
8

---

Published as a conference paper at ICLR 2020
VGG16 CIFAR10 (1500 SAMPLES)
35
Average #Spikes (100 time-steps)
JANN-SNN Conversion (Acc: 89 20)
30
SNN STDB (Aoc: 91.87)
V=155
V=3399
25
20
15
v=
v=0.12
V=31
10
V=
v=22
v=
V=0.94
v
-
v=0.8
6
conv1
conv2
conv3
conv4
conv6
convé
conv7
convé
convi
conv 10
conv 11
convi 12
conv 13
linear
linearz
Figure 3: Average number of spikes for each layer in a VGG16 architecture for purely converted
SNN and SNN trained with hybrid technique. The converted SNN and SNN trained with hybrid technique achieve an accuracy of 89.20% and 91.87%, respectively, for the randomly selected 1500 samples from the test set. Both the networks were inferred for 100 time steps and 'v" represents the threshold voltage for each layer obtained during the conversion process (Algorithm) I
CIFAR10: The dataset consists of labeled 60, 000 images of 10 categories divided into training (50, 000) and testing (10, 000) set. The images are of size 32 x2 with RGB channels.
CIFAR100: The dataset is similar to CIFAR10 except that it has 100 categories.
ImageNet: The dataset comprises of labeled high-resolution 1.2 million training images and 50, 000 validation images with 1000 categories.
7 ENERGY- -DELAY PRODUCT ANALYSIS OF SNNs
A single spike in an SNN consumes a constant amount of energy (Cao et al. 2015). The first order analysis of energy-delay product of an SNN dependent on the number of spikes and the total number of time steps. Fig. 3 shows the average number of spikes in each layer when evaluated for 1500 samples from CIFARIO testset for VGG16 architecture. The average is computed by summing all the spikes in a layer over 100 time steps and dividing by the number of neurons in that layer. For example, the average number of spikes the 10th layer is 5.8 for both the networks, which implies that over 100 time step period each neuron in that layer spikes 5.8 times on average over all input samples. Higher spiking activity corresponds to lower energy-efficiency. The average number of spikes is compared for converted SNN and SNN trained with conversion-and-STDDi. The SNN trained with conversion-and-SS
has 1.5x less number of average spikes over all layers under iso conditions (time steps, threshold voltages, inputs, etc.) and achieves higher accuracy compared to the converted SNN. The converted SNNs when simulated for larger time steps further degrade the energy-delay product with minimal increase in accuracy (Sengupta al. 2019).
8 RELATED WORK
Bohte et al. (000)) proposed a method to directly train on SNN by keeping track of the membrane potential of spiking neurons only at spike times and backpropagating the error at spike times based on only the membrane potential. This method is not suitable for networks with sparse activity due to the 'dead neuron' problem: no learning happens when the neurons do not spike. In our work, we need one spike for the learning to start but gradient contribution continues in later time steps as shown in Fig. Zenke & Ganguli (2018) derived a surrogate gradient based method on the membrane potential of spiking neuron at single time step only. The error was backpropagated at only one time step and only the input at that time step contributed to the gradient. This method neglects the effect of earlier spike inputs. In approach, the error is backpropagated for every time step and the weight update is performed on the gradients summed over all time steps. [Shresthal & Orchard (2018) proposed gradient function similar to the one proposed in this work. They
9

---

Published as a conference paper at ICLR 2020
Table 2: Comparion of work with other SNN models CIFARIO and ImageNet datasets
Model
Dataset
Training Method
Architecture
Accuracy
Time-steps
Hunsberger
CIFARIO
ANN-SNN Conversion
2Conv, Liincr
82.95%
6000
& Eliasmith
(0155)
Cao et al. (2015)
CIFAR1O
ANN-SNN Conversion
3Conv, 2Linear
77.43%
400
Sengupta
CIFARIO
ANN-SNN
VGG16
91.55%
2500
et al. (2019)
Conversion
Lee et al.
CIFARI0
Spiking BP
VGG9
90.45%
100
(019))
Wu et al.
CIFARIO
Surrogate Gradient
5Conv, 2Linea
90.53%
12
(0199)
This work
CIFARIO
Hybrid
VGG16
91.13%
100
Training
92.02%
200
Sengupta
ImageNet
ANN-SNN
VGG16
69.96%
2500
et al. (2019)
Conversion
This work
ImageNet
Hybrid
VGG16
65.19%
250
Training
used the difference between the membrane potential and the threshold compute the gradient compared to the difference in spike timing used in this work. The membrane potential continuous value whereas the spike time is integer value bounded the number of time steps. Therefore, gradients that depend on spike time can be pre-computed and stored in a look-up table for faster computation. They evaluated their approach on shallow architectures with two convolution layer for MNIST dataset. In this work, we trained deep SNNs with multiple stacked layers for complex calssification tasks. /Wu al.) (2018) performed backpropagation through time on SNN with surrogate gradient defined on the membrane potential. The surrogate gradient was defined as piece-wise linear exponential function of the membrane potential. The other surrogate gradients proposed in the literature are all computed on the membrane potential (Nefici et al. (011)). Lee et al. (2199) approximated the neuron output as continuous low-pass filtered spike train. They used this approximated continuous value to perform backpropagation. Most of the works in the literature direct training of SNN or conversion based methods have been evaluated on shallow architectures for simple classification problems. In Table 2 we compare model with the models that reported accuracy on CIFAR10 10 and ImageNet dataset. [Wu et al. (219) achieved convergence in 12 time steps by using dedicated encoding layer capture the input precision. It is beyond the scope of this work to compute the hardware and energy implications of such encoding layer. Our model performs better than all other models at far fewer number of time steps.
CONCLUSIONS
The direct training of SNN with backpropagation is computationally expensive and slow, whereas ANN-SNN conversion suffers from high latency. To address this issue we proposed a hybrid training technique for deep SNNs. We took an SNN converted from ANN and used its weights and thresholds as initialization for spike-based backpropagation of SNN. We then performed spike-based backpropagation on this initialized network to obtain an SNN that can perform with fewer number of time steps. The number of epochs required to train SNN was also reduced by having good initial starting point. The resultant trained SNN had higher accuracy and lower number spikes/inference compared to purely converted SNNs at reduced number of time steps. The backpropagation through time was performed with surrogate gradient defined using neuron's 's spike time that captured the temporal information and helped in reducing the number of time steps. We tested our algorithm CIFAR and ImageNet datasets and achieved state-of-the-ee performance with fewer number of time steps.
10

---

Published as a conference paper at ICLR 2020
ACKNOWLEDGMENTS
This work was supported in part by the National Science Foundation, in part by Vannevar Bush Faculty Fellowship, and in part by C-BRIC, one of six centers in JUMP, Semiconductor Research Corporation (SRC) program sponsored by DARPA.
REFERENCES
Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-ferm memory and learning-to-leam in networks of spiking neurons. In Advances in Neural Information Processing Systems, 787-797, 2018.
Sander M Bohte, Joost N Kok, and Johannes A La Poutré. Spikeprop: backpropagation for networks of spiking neurons. In ESANN, pp. 419-424, 2000.
Y-Lan Boureau, Jean Ponce, and Yann LeCun. A theoretical analysis of feature pooling in visual recognition. In Proceedings of the 27th international conference on machine learning (ICMI-10), pp. 111-118, 2010.
Yongqiang Cao, Yang Chen, and Deepak Khosla. Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision, 113(1):54-66, 2015.
J. Deng, W. Dong, R. Socher, L.-J. Li. K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hicrarchical Image Database. In CVPR09, 2009.
Peter U Dichl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, and Michael Pfeiffer. Fast-clasiifying, high-accuracy spiking deep networks through weight and threshold balancing. In 2015 International Joint Conference Neural Networks (IJCNV), pp. 1-8. IEBE, 2015.
Peter U Dichl, Guido Zarrella, Andrew Cassidy, Bruno U Pedroni, and Emre Neftci. Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware. In 2016 IEEE International Conference on Rebooting Computing (ICRC), pp. 1-8.
IEEE, 2016.
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Dongsung Huh and Terrence J Sejnowski. Gradient descent for spiking neural networks. In Advances in Neural Information Processing Systems, pp. 1433-1443, 2018.
Eric Hunsberger and Chris Eliasmith. Spiking deep networks with lif neurons. arXiv preprint arXiv:1510.08829, 2015.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint 1xXiv:1502.031666 2015.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Chankyu Lee, Syed Shakib Sarwar, and Kaushik Roy. Enabling spike-based backpropagation in state-of-the-art deep neural network architectures. arXiv preprint arXiv:1903.06379 2019.
Emre O Neftci, Hesham Mostafa, and Friedemann Zenke Surrogate gradient learning spiking neural networks. arXiv preprint arXiv:1901.09944 2019.
Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in neuroscience, 13, 2019.
Sumit Bam Shrestha and Garrick Orchard. Slayer: Spike layer error reassignment in time. In
Advances in Neural Information Processing Systems, pp. 1412-1421, 2018.
11

---

Published as a conference paper at ICLR 2020
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 2014.
Sen Song, Kenneth D Miller, and Larry F Abbout. Competitive hebbian learning through spike-timing-dependenn synaptic plasticity. Nature neuroscience, 3(9):919, 2000.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-199 1958, 2014.
Paul Werbos et al. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78(10):1550-1560, 1990.
Yujie Wu, Lei Deng. Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience. 12, 2018.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 1311-1318, 2019.
Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural computation, 301)):511-1541, 2018
12

---

Published as a conference paper at ICLR 2020
A COMPARISONS WITH OTHER SURROGATE GRADIENTS
The transfer function of the spiking neuron a step function and its derivative is zero everywhere except at the time of spike where is not defined. In order to perform backpropagation with spiking neuron several approximations are proposed for the gradient function (Bellec et al.] 2018) Zenke & Ganguli, 2018; [Shrestha Orchard, 2018] Wu et al., (018).. These approximations are either linear or exponential function of (u Vi), where is the membrane potential and Vú the threshold voltage (Fig. 4.. These approximations surrogate gradient or pseudo-derivative. In this work, proposed approximation that is computed using the spike timing of the neuron (Equation I.I. We compare our proposed approximation with the following surrogate gradients:
0.3
Linaar Exponential
0e
Zu
0.1
-1
-0.5
0
0.5
Vi)
referred
Figure 4: Linear and Exponential approximation of the gradient of the spiking neuron (step function).
do
du = a max {0, -lu Vil3
(13)
do ae (u-Vil 8u.
(14)
where is the binary output of the neuron, u is the membrane potential, V2 is the threshold potential, and 3 are constants. Equation 13] and Equation represen the linear and exponential approximation of the gradient, respectively. We employed these approximations in the hybrid training for a VGG9 network for CIFAR1O dataset. All the approximations (Equation 11 13 and 14) produced similar results in terms of accuracy and number of epochs for convergence. This shows that the term At (Equation]) [1p is a good replacement for lu - Vil (Equation 14). The behaviour of At and tu Vzl is similar, i.e., is small closer to the time of spike and increases as we move away from the spiking event. The advantage of using At is that its domain is bounded by the total number be pre-computed and stored in table for faster access during training. This is not possible for membrane potential because real value computed based the stochastic inputs and previous state of the neuron which is not known before hand. The exact benefit in energy from the pre-computation is dependent on the overall system architecture and evaluating it is beyond the scope of this paper.
of time steps (Equation (2]. Hence, all possible values of gradients
13

---

Published as a conference paper at ICLR 2020
B COMPARISONS OF SIMULATION TIME AND MEMORY REQUIREMENTS
The simulation time and memory requirements for ANN and SNN are very different. SNN requires much more resources iterate over multiple time steps and store the membrane potential for each neuron. Fig. 5 shows the training and inference time and memory requirements for ANN, SNN trained with backpropagation from scratch, and SNN trained with the proposed hybrid technique. The performance was evaluated for VGG16 architecture trained for CIFARIO 10 dataset. SNN trained from scratch and SNN trained with hybrid wwweeerioonanndSTTeT One epoch of ANN training (inference) takes 0.57 (0.05) minutes and 1.47 (1.15) GB of GPU memory. On the other hand, one epoch of SNN training (inference) takes (11.39) minutes and 9.36 (1.37) GB of GPU memory for same hardware and mini-batch size. ANN and SNN trained from scratch reached convergence after 250 epochs. The hybrid technique requires 250 epochs of ANN training and 20 epochs of spike- based backpropagation. The hybrid training technique is one order of magnitude faster than training SNN from scratch. The memory requirement for hybrid need to perform fine-tuning with spike-based backpropagation.
are evaluated for 100 time steps.
technique is same SNN
Training Time and Memory
Inference Time and Memory
100
100
10
10
ANN
SNN
Hybrid
ANN
SNN
Hybrid
Time
Memory
Time
Memory
Figure 5: Training and Inference time and memory for ANN, SNN trained with backpropagation from scratch, and SNN trained with hybrid technique. All values are normalized based on ANN values. The y-axis in log scale. The performance was evaluated on one Nvidia GeForce RTX 2080 TU102 GPU with 11 GB of memory. All the networks were trained for VGG16 architecture, CIFAR1O dataset, 100 time steps, and mini-batch size of ANN and SNN require 250 epochs of training from scratch, hybrid conversion-ant and-STDB based training requires 250 epochs of ANN training followed by 20 epochs of baske-based backpropagation.
14


Reviews:
[{"review_id": "SklebdhWoB", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper examines combining two approaches of obtaining a trained spikingneural network (SNN). The first approach of previous work is converting the weights of a trained artificial neural network (ANN) with a given architecture, to the weights and thresholds of a SNN, and the second approach uses a surrogate gradient to train an SNN with backpropagation. The true novelty of the paper seems to be in showing that combining the two approaches sequentially, trains a SNN that requiresfewer timesteps to determine an output which achieves state of the art performance. This is summarized by Table 1. However, it does not mention how many epochs it takes to train an SNN from scratch, nor compare this to the total training time (ANN training + SNN fine-tuning) of their approach. They also claim a novel spike-time based surrogate gradient function (eq. 11), but it is very practicallysimilar to the ones explored in the referenced Wu. et al 2018 (eq. 27 for instance), and these should be properly contrasted showing that this novel surrogate function is actually helpful (the performance/energy efficiency might only come from the hybrid approach). The authors argue for SOTA performance in Table 2, but the comparison to other work doesn’t clearly separate their performance from the otherlisted works; For example the accuracy gain against Lee et al.,2019 only comes from the architecture being VGG16 as opposed to VGG9, as can be seen from comparing with the VGG9 architecture from Table 1, furthermore they take the sameamount of timesteps, which is supposed to be the principle gain of this work.\n\nSome small suggestions that are independent from the above:\n\n1.The most similar or relevant version of equation (2) in previous work could be referenced nearby for context.\n\n2.The last sentence of the first paragraph on p.4 “the outputs from each copy...” is confusing. Are you just meaning to describe BPTT? \n\n3.Typos: sec7 4th line “neruons”, sec 2.2 “both the credit” (remove “the”)\n\n---------------\nFollowing the author response I have upgraded my rating.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "S1gQnUOTFS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a fine-tuning method of models converted from standard encoding and SGD training to Spike/NN's. \nThe key point of the paper is that directly training S/NN's with spike-back-prop is slow and inefficient, while directly inferencing with converted models is also inefficient due to the large integration window required to get a good estimate of the spiking neuron potential. The authors claim, and to a good extent show that, their proposed method is best of both worlds: train the models efficiently with standard encoding / SGD, this is something we know works and scale well, then convert and fine-tune with spike-backprop to get models that perform well under a shorter integration window, and thus are more efficient at inference time. The intuition is that models can achieve shorter integration windows while keeping good results because, under the assumptions made by the proposed algorithm, the fine-tuning is effectively unrolling neuron dynamics that can be trained with BPPT, in a way similar to LSTM/Recurrent models. In that case, since model dynamics are taken into account during fine-tuning, it results in better performance even under shorter time-windows. This is an interesting concept, since the training doesn't only consider a mean-field estimate of the spike-activation, but it looks at  spiking neuron dynamics with an higher granularity. The paper is well written, clear and easy to understand. Results are comparatively competitive and code is made available.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Bye9F-4CFS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes methods to initialize and train spiking NNs (SNNs) as an alternative to ANNs, not driven primarily by improved loss or generalization, but by energy efficiency improvements derived from timing-event based sparse operation rather than asynchronous sweeps. The backpropagation method introduced is sensible, as are the experiments on known datasets to show its effectiveness. The paper is well written (apart from the miniscule Figure 3 containing the main result). \nI recommend acceptance, with caveats: the energy performance is actually not directly calculated, but speculatively estimated, it depends on the computational architecture chosen to implement the respective networks. I point out that ANNs need to be trained first to properly initialize an SNN, so the relative training effort claimed is less impressive, but energy performance does count in actual operational practice - training is (or should) be a small fraction of that.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: After the rebuttal, all reviewers rated this paper as a weak accept. 
The reviewer leaning towards rejection was satisfied with the author response and ended up raising their rating to a weak accept.  The AC recommends acceptance.
Decision: Accept (Poster)