SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes
Authors: Johannes C. Thiele
Olivier Bichler
Antoine Dupret
Venue:  (2020)
Keywords: spiking neural network
neuromorphic engineering
backpropagation

Abstract: Event-based neuromorphic systems promise to reduce the energy consumption of deep neural networks by replacing expensive floating point operations on dense matrices by low energy, sparse operations on spike events. While these systems can be trained increasingly well using approximations of the backpropagation algorithm, this usually requires high precision errors and is therefore incompatible with the typical communication infrastructure of neuromorphic circuits. In this work, we analyze how the gradient can be discretized into spike events when training a spiking neural network. To accelerate our simulation, we show that using a special implementation of the integrate-and-fire neuron allows us to describe the accumulated activations and errors of the spiking neural network in terms of an equivalent artificial neural network, allowing us to largely speed up training compared to an explicit simulation of all spike events. This way we are able to demonstrate that even for deep networks, the gradients can be discretized sufficiently well with spikes if the gradient is properly rescaled. This form of spike-based backpropagation enables us to achieve equivalent or better accuracies on the MNIST and CIFAR10 datasets than comparable state-of-the-art spiking neural networks trained with full precision gradients. The algorithm, which we call SpikeGrad, is based on only accumulation and comparison operations and can naturally exploit sparsity in the gradient computation, which makes it an interesting choice for a spiking neuromorphic systems with on-chip learning capacities.

Published as a conference paper at ICLR 2020
SPIKEGRAD: AN ANN-EQUIVALENT COMPUTA- TION MODEL FOR IMPLEMENTING BACKPROPAGA- TION WITH SPIKES
Johannes Thiele, Olivier Bichler & Antoine Dupret CEA, LIST
91191 Gifetur-Yvette, France
(Johannes thiele, olivier. wiceerricchheeee
ae http://r..b.b...ee..aaeeeeeeeeee
fr
ABSTRACT
Event-based neuromorphic systems promise to reduce the energy consumption of deep neural networks by replacing expensive floating point operations dense matrices by low energy, sparse operations on spike events. While these systems be trained increasingly well using approximations of the backpropagation algorithm, this usually requires high precision errors and is therefore incompatible with the typical communication infrastructure of neuromorphic circuits. In this work, we analyze how the gradient can be discretized into spike events when training a spiking neural network. To accelerate our simulation, we show that using a special implementation of the integrate-and-Fe fire neuron allows us to de- scribe the accumulated activations and errors of the spiking neural network in terms of equivalent artificial neural network, allowing training compared to an explicit simulation of all spike events. This way we are able demonstrate that even for deep networks, the gradients can be discretized sufficiently well with spikes if the gradient is properly rescaled. This form spike-based backpropagation enables us achieve equivalent or better accuracies on the MNIST and CIFARIU 10 datasets than comparable state-of-the-ar spiking neural networks trained with full precision gradients. The algorithm,, which call SpikeGrad, is based on accumulation and comparison operations and can naturally exploit sparsity in the gradient computation, which makes it interesting choice for spiking neuromorphic systems with on-chip learning capacities.
to largely speed up
INTRODUCTION
Spiking neural networks (SNNs) new generation of artificial neural network models (Maass 1997) that try to harness properties of biological neurons to build energy efficient spiking neuro- morphic systems. Processing in traditional artificial neural networks (ANNs) is based on parallel processing of operations on dense tensors of fixed length. In contrast to this, spiking neuromorphic systems communicate with asynchronous events, which allows dynamic, data dependent computation that can exploit high temporal and spatial sparsity.
The recent years have seen large number of approaches devoted to optimization of spiking neural networks with the backpropagation algorithm, either by converting ANNs to SNNs (Dichl et al. 2015
Esser al. [016 Rueckauer al.) 2017 Sengupta et al. 2019) or by simulating spikes exp sxpliclly
in the forward pass and optimizing these dynamics with floating point gradients (Lee et al. 2016
Yin et al. 2017 Wu et al. 2018h Severa al. 2019, Jin al. 2018 Bellec et al. 2018 Zenke & Ganguli, 2018 Shrestha & Orchard 2018). These methods aim to optimize SNNs for efficient inference, and backpropagation
performed offline on a standard computing system. It would however be desirable to also enable on-chip learning in neuromorphic chips using the power of the backpropagation algorithm, and to maintain the advantages of spike-based processing also in the error propagation phase.
Previous work on the implementation of backpropagation with spikes is mostly concerned with biological plausibility. A non-spiking version of biologically inspired backpropagation is presented

---

Published as a conference paper at ICLR 2020
by Sacramento et al. (2018). Guerguiev et al. (2017) Neftci et al. (017) and Samadi et al. (2017)
introduce spike- bikee-baedd versions of the backpropagation algorithm using variants of (direct) feedback alignment (Lillicrap et al. 2016 Nøkland| 2016). The exact backpropagation algorithm, which
backpropagates through symmetric weights, might however be required to achieve good inference performance on large-scale deep neural networks (Baldi & Sadowski 2016; Bartunov et al. (0188) O'Connor & Welling (2016) and Thiele et al. (2019) present implementations of standard backpropa- gation where the gradient is coded into spikes and propagated through symmetric weights. In the same spirit, our work is mostly concerned with exploiting spike based information encoding for energy efficient processing, which means that inference performance and operational simplicity will be preferred over biological plausibility and complex neuron models.
We demonstrate how backpropagation can be seamlessly integrated into the spiking neural network framework by using second accumulation compartment that discretizes the error into spikes. By additionally weighting the activity counters by the learning rate, we obtain system that is able to perform learning and inference based on accumulations and comparisons alone. As for the forward pass, this allows us to use the dynamic precision computation provided by the discretization of all operations into spike events, and to exploit the sparsity of the gradient. Using similar reasoning as Binas et al. (2016) and Wu et al. (2019) have applied to forward propagation in SNNs, we show that the system obtained in this way can be mapped an integer activation ANN whose activations equivalent to the accumulated neuron responses for both the forward and the backward propagation phase. This allows us simulate training of large-scale SNNs efficiently on graphics processing units (GPUs), using their equivalent ANN. Additionally, in contrast to conversion methods that approximate pre-trained ANNs with SNNs, this method guarantees that the inference precision of the SNN will be equivalent to the ANN. In contrast to O'Connor & Welling (2016), this is true for any number of spikes and arbitrary spike order. We demonstrate classification accuracies equivalent or superior to existing implementations of SNNs trained with full precision gradients. and comparable to the precision of standard ANNS using similar topologies. This is the first time competitive classification performances are reported on the CIFAR1O and CIFAR 100 datasets using a sarge-scale SNN where both training and inference are fully implemented with spikes. To the best of our knowledge, our work provides for the first time a demonstration of how the sparsity of the gradient during backpropagation could be exploited within a large-scale SNN processing structure.
2 THE SpikeGrad ALGORITHM
We begin with the description of SpikeGrad, the spike-based backpropagation algorithm. The algorithm mainly consists of special implementations of the integrate-and-fir (IF) neuron model, that are used to encode and propagate information in the forward and backward pass.
For each training example/mini bbttt, integration is performed from t = 0 to t = T for the forward pass and from = T At to T in the backward pass. Since no explicit time is used in the algorithm, At represents symbolically the (very short) time between the arrival of an incoming spike and the response of the neuron, which is only used here to describe causality.
Spike discretization of the forward pass For forward propagation, the architecture is described by multiple layers (labeled by I [0, L)) of IF neurons with integration variable V/((t and threshold Of:
V1(t At) Vi (ns((t) Σ Eu Vi(0) =0: b/.
The variable wef is the weight and by bias value. The spike activation function st( E (-1,0, is a function which triggers signed spike event depending on the internal variables of the neuron. It will be shown later that the specific choice of the activation function is fundamental for the mapping to an equivalent ANN, After neuron has fired, its integration variable is decremented or incremented by the threshold value Og, which is represented by the second term on the r.h.s. of equation D
As representation of the neuron activity, we use a trace (1) which accumulates spike information over a single example:
(t At) = 2f(1) + (t/(().
By weighting the activity with the learning rate 17 we avoid performing a multiplication when weighting the input with the learning rate for the weight update equation g
2

---

Published as a conference paper at ICLR 2020
Implementation of implicit ReLU and surrogate activation function derivative It is possible to define an implicit activation function based on how the neuron variables affect the spike activation function (t). In our implementation, use the following fully symmetric function to represent linear activation functions (used for instance in pooling layers):
if (1) 2 Oh Jim (V!(0) = -1 if (1) S - otheewise
(3)
The following function corresponds to the rectified linear unit (ReLU) activation function:
1
if (t( O
$
..RLLU (V((0)
(1) < O2 and xx 0.
(4)
0
otherwise
The pscudo-derivative of the activation function is denoted symbolically by S We use 5 (("" = 1 for the linear case. For the ReLU, surrogate of the form:
if (T) or =T( >
s R.RU (T)
(5)
otherwise
These choices will be motivated in the following sections. Note that the derivatives depend only on the final states of the neurons at time T.
Diseretization of the error into spikes For gradient backpropagation, we introduce second compartment with threshold Op in each neuron, which integrates error signals from higher layers. The process discretizes errors in the same fashion as the forward pass discretizes an input signal into sequence of signed spike signals:
( + Ar) = U(() - 0bp ={(() + Σ 1+1 ((11(((11))
whi of (1).
To this end, we introduce ternary error spike activation function 2(( E (- which is defined in analogy to equation 3 using the eттоr integration variable (t) and the backpropagation threshold Opp- The error then obtained by gating this ternarized variable 2t()) with one of the surrogate activation function derivatives of the previous section (linear ReLU):
(t) (T))
This ternary spike signal is backpropagated through the weights to the lower layers and also applied the update rule of the weight increment accumulator why:
wh, + At) (1) 6(())
which is triggered every time an error spike (equation 7 is backpropagated. The weight updates are accumulated during error propagation and applied after propagation is finished to update each weight simultaneously. In this way, the backpropagation of errors and the weight update will, exactly forward propagation, only involve additions and comparisons of floating point numbers.
The SpikeGrad algorithm can also be expressed in an event-based formulation, described in algorithms and 3 This formulation is closer to how the algorithm would be implemented an actual SNN hardware implementation of the IF firing dynamics.
Loss function and error scale We use the cross entropy loss function in the final layer applied to the softmax of the total integration (T) (no spikes are triggered in the top layer during inference). This requires more complex operations than accumulations, but is negligible if the number classes is small. To make sure that sufficient emror spikes are triggered in the top layer, and that error spikes arrive even in the lowest layer of the network, we apply a scaling factor a to the error values before transferring them to U. This scaling factor also implicitly sets the precision of the gradient, since higher number of spikes means that a large range of values can be represented. To counteract the relative increase of the gradient scale, the learning rates have to be rescaled by factor l/a.
3

---

Published as a conference paper at ICLR 2020
Input encoding As pointed out in Rueckauer al. 2017)) and Wu al. (2018c), it is crucial to maintain the full precision of the input image to obtain good performances on complex standard benchmarks with SNNs. One possibility is to encode the input in large number of spikes (Sengupta et al. 2019). Another possibility, which has been shown to require a much lower number of spikes in the network, is multiply the input values directly with the weights of the first layer (just like in a standard ANN). The drawback is that the first layer then requires multiplication operations. The additional cost of this procedure may however be negligible if all other layers can profit from spike-based computation. This problematic does not exist for stimuli which are natively encoded in spikes.
Algorithm Backward
Algorithm 1 Forward
function BACKPROPAGATE(( ()
function PROPAGATE((4, (,),,
U - U + 8
w [+1
V + V + w
2 - +(() 5: < z S
error activation function
s/ 4 (V æ spike activation function
if' : # 0 then
if o + 0 then
V + V s OE
U! + - o Obp for j in layer I - connected to do
a < x + 77s
for k in + connected to i do PROPAGATE([1 + ().
BACKPROPAGATE(! w/y + w/y - -8 x,
1, i], 8))
Algorithm 3 Training of single example/batch
init: V +- b. U +- 0, x 0, w +- variables in bold describe all neurons in network/layer while input spikes sf"h do for k in / = 0 receiving s do PROPAGATE((). (0, k i), (T)
>> spikes corresponding to training input
S' + SV(V U + a ac/avz
> calculate surrogate derivatives > calculate classfication error
for i in = do while |U-4] 2 bp do BACKPROPAGATTE(L,. -1,0) w0 f- w + w
> backpropagate error spikes last layer receives error b update weights with weight update accumulator
FORMULATION OF THE EQUIVALENT ANN
The simulation of the temporal dynamics of spikes in SpikeGrad requires large number of time steps or events if activation or error values are large. It would therefore be extremely beneficial if we were able to map the SNN an equivalent ANN that can be trained much faster on standard hardware. In this section, we demonstrate that it is possible to find such an ANN using the forward and backward propagation dynamics of the SpikeGrad IF neuron model described in the previous section.
Spike discretization error We start our analysis with equation We reorder the terms and sum over the increments (v) (t + At) - (t) every time the integration variable is changed either by spike that arrives at time t. E [o, T] via connection j. or by a spike that is triggered at [0, 27. With the initial conditions vy' = by, s0(0) = 0, we obtain the final value VV!
time t
V'(T)- = Σ AV' 0 Σ $ (1;) + Σ w sj
Σ
-1))
(9)
!
By defining the total transmitted output of a neuron as S
= En s1(1)) we obtain:
1 VI(T) = S St
s =
1
Σ w/ S - -6²
(10)
4

---

Published as a conference paper at ICLR 2020
The same reasoning be applied to backpropagation of the gradient. We define the summed responses over error spikes times Tj E [T + At.
7 as Z ==
Eng 2/ (T;)) to obtain:
1 (T)
Z
Z
1
Σ
++ W
F
(11)
bp
bp
(Tt)
Σ
$ 1+ (T) (TT) St (T)
(12)
In both equation 10 and equation 11 the terms S and E are equivalent to the output of an ANN with signed integer inputs S and E 1/0g and l/eu, are implicit scaling factors of activations and gradients. If gradients shall not be explicitly rescaled, backpropagation requires Obp On- The values of the residual integrations 1/eu (T) and U therefore represent the spike discretization error SDEa S S or SDEbp == Z Zj between the ANN outputs S and z! and the accumulated SNN outputs S and Z Since we know that (T) e (-On, θn) and (T) E (-Obp, Ob,), this gives bounds of [SDE#l < 1 and |SDEapl < 1.
So far we can only represent linear functions. We now consider an implementation where the ANN applies a ReLU activation function instead. The SDE in this case is:
SDE ReLU
S..
(13)
We can calculate the error by considering that equation forces the neuron in one of two regimes
(note that a > 0 <+ S > 0): In one case, S - 0, (i(T) < Og (this includes (T) < -gg).
This implies 8 I/e V (T) and therefore [SDE ReLU (or even [SDF 0 V (T) S 0).
In the other case, S O, (t) E (-On, Og), where equation 4 equivalent to equation 3
This equivalence motivates the choice of equation as a surrogate derivative for the SNN: the condition (V) 0 or (T) > 0) can be seen to be equivalent to (T) > 0, which defines the derivative of a ReLU. Finally, for the total weight increment Awty. it can be seen from equation and equation that:
(T - An) = n
Awly (T) (1)
(14)
which is exactly the weight update formula of an ANN defined on the accumulated variables. We have therefore demonstrated that the SNN can be represented by an ANN by replacing all S and Z by S and Σ and applying the corresponding activation function directly on these variables. The error that will be caused by this substitution compared to using the accumulated variables S and Z of an SNN is described by the SDE. This ANN can now be used for training of the SNN on GPUs. The SpikeGrad algorithm formulated on the variables implemented on a dedicated, event-based spiking neural network hardware implementation of the IF neurons. We will now demonstrate how the SDE can be further reduced to obtain ANN and SNN that are exactly equivalent.
5 and x represents the algorithm that would be
Response equivalence For large number of spikes, the SDE may be negligible compared to the activation of the ANN. However, in a framework whose objective is to minimize the number of spikes emitted by each neuron, this error can have potentially large impact.
One option to reduce the error between the ANN and the SNN output is to constrain the ANN during training integer values. One possibility is to round the ANN outputs:
Σ w + 6/ )
s round = rounddSs [s]] = round
1 +
(15)
The round function here rounds to the next integer value, with boundary cases rounded away from zero. This behavior can be implemented in the SNN by a modified spike activation function which is applied after the full stimulus has been propagated. To obtain the exact response as the ANN, we have to take into account the current value of S and modify the threshold values:
if VT) Gr/2 or (S) > 0, V'(T)) 6m/2) if V((T)) 0n/2 or ( < 0, VI(T) = -Ow/2) (16)
1rees (V! s) 8/
otherwise
5

---

Published as a conference paper at ICLR 2020
Because this spike activation function is applied only to the residual values, we call it the residual spike activation ionvaationfuncctonn function. The function is applied to a layer after all spikes have been propagated with the standard spike activation function (equation 3 equation 4 We start with the lowest layer and propagate all residual spikes to the higher layers, which use the standard activation function. We then proceed with setting the next layer residual mode and propagate the residual spikes. This is continued until we arrive at the last layer of the network.
By considering all possible rounding scenarios, it can be seen that equation 16 indeed implies:
S + s; rese (V) = round|s + V/ round
(17)
The same principle can be applied to obtain integer-rounded error propagation:
)
z zlrround = round
[z]] = round
w
(18)
Op
ki
We have to apply the following modified spike activation function in the SNN after the full error has been propagated by the standard error spike activation function:
if (T) > 0be/2 or (Z) 2 0, U,(T) = 02//2)
1.rCs (U.(T, z
if < (2 z 0, (T 0bp/2) otherwise
(19)
which implies:
z + r. (U)(T). round[Z + U/e., round[Z]].
(20)
We have therefore shown that the SNN will after each propagation phase have exactly the same accumulated responses as the corresponding integer activation ANN. The same principle can be applied to obtain other forms of rounding (e.g. floor and ceil), if equation 16 and equation T9 are modified accordingly.
Computational complexity estimation Note that we have only demonstrated the equivalence of the accumulated neuron responses. However, for each of the response values, there is a large number of possible combinations of 1 and - values that lead the same response. The computational complexity of the event-based algorithm depends therefore on the total number of these events. The best possible case when the accumulated response value S is represented by exactly [S spikes. In the worst case, large number of additional redundant spikes is emitted which sum up to 0. The maximal number of spikes each layer is bounded by the largest possible integration value that can be obtained. This depends on the maximal absolute weight value w max. the number of connections N and the number of spike events n² each connection receives, which is given by the maximal value of the previous layer (or the imput in the first layer):
1 OH
mimin
|
No. Wmxx Taax
(21)
The same reasoning applies backpropagation. Our experiments show that for input encodings where the input is provided in a continuous fashion, and weight values that are much smaller than the threshold value, the deviation from the best case scenario is rather small. This is because in this case the sub-threshold integration allows to average out the fluctuations in the signal. This way the firing rate stays rather close its long term average and few redundant spikes are emitted. For the total number of spikes n in the full network on the CIFAR1O test set, we obtain empirically nl--iann//ea Aminn 0.035.
EXPERIMENTS
Classification performance Tables and compare the state-of-the-an results for SNNs on the MNIST and CIFAR10 datasets. It can be seen that in both cases, our results competitive with respect to the state-of-the results of other SNNs trained with high precision gradients. Compared to results using the same topology, our algorithm performs at least equivalently.

---

Published as a conference paper at ICLR 2020
Table 1: Comparison of different state-of-the-art spiking CNN architectures on MNIST. "idiicates that the same topology 2282881555--2244055----------- was used.
Architecture
Method
Rec. Rate (max(meanistd))
Wu et al. 2018b)*
BP float gradient
99.42%
Rueckauer et al.
2017
CNN converted to SNN
99.44%
Jin et al. (2018)
BP float gradient
99.49%
This work*
BP float gradient
99.48 (9.48/99.36 + 0.066% 99.53 + 0.06
This work*
BP spike gradient
Table 2: Comparison of different state-of-the-art spiking CNN architectures CIFARIO. indicates that the same topology 114x::/222322260.332222222222222
was used.
Architecture
Method
Rec. Rate (max(mean-std])
Rueckauer et al. (2017)
CNN converted SNN (with BN)
90.85%
Sengupta et al. (2019)
VGG- 16 converted to SNN
91.55%
Wu et al. (2018c)*
BP float gradient (no NeuNorm)
89.32%
This work
BP float gradient
89.72 89.38 + 0.25 %
This work*
BP spike gradient
89.99 89.49 0.28
The final classification performance of the network function of the error scaling term o in the final layer can be seen in figure [ Previous work on low bitwidth gradients (Zhou et al.) 2018] found that gradients usually require higher precision than both weights and activations. Our results also indicate that a certain minimum number of error spikes is necessary to achieve convergence. This strongly depends on the depth of the network and if enough spikes are triggered to provide sufficient gradient signal in the bottom layers. For the CIFARIO 10 network, convergence becomes unstable for approximately a < 300. If the number of operations large enough for convergence, the required precision for the gradient does not seem to be extremely high. On the MNIST task, the difference in test performance between a gradient rescaled by a factor of 50 and a gradient rescaled by a factor of 100 becomes insignificant. In the CIFAR10 task, this is true for rescaling by 400 or 500. Also the results obtained with the float precision gradients in tables] [ and demonstrate the same performance, given the range of the error.
To investigate the performance of SpikeGrad on databases with a larger number of classes, we also ran an additional experiment on the CIFARIOK dataset. Using the exactly same architecture for CIFARI0 10 with a = 500 (besides the final layer that is increased to 100 classes), we obtain a maximal classification score of 64.40%. Running the same experiment with only the forward pass encoded in spikes, but floating point gradients, we obtain 64.69%. This result could probably be improved by adapting the architecture better to the dataset. However, it demonstrates that coding the gradient into spikes does not lead to a large precision loss even in this scenario.
Sparsity in backpropagated gradient To evaluate the potential efficiency of the spike coding scheme relative to an ANN, we use the metric of relative synaptic operations. A synaptic operation corresponds
(MAC) in the case of an ANN, and simple accumulation (ACC) in the case of an SNN. This metric allows to compare networks based on their fundamental operation. The advantage of this metric the fact that it does not depend on the exact implementation of the operations (for instance the number of bits used to represent each number). Since ACC is however generally cheaper and easier to implement than a MAC, we can be sure that an SNN is more efficient in terms of its operations than the corresponding ANN if the number of ACCs is smaller than the number of MAC' (without considering potential hardware overheads of event-basen based computing).
cattiply-accumulad
Numbers were obtained with the integer activations of the equivalent ANN to keep simulation times tractable. As previously explained, the integer response of the equivalent ANN represents the best case scenario for the SNN, i.e. the activation encoding with the lowest number of spikes. The actual
7

---

Published as a conference paper at ICLR 2020
(a)
10-1
0.07%
0.25%
mii.. SynOps pns/connecion
SynOps /ess//oncciioon
20, 99.32 4 0.01%
0r 400)
89
0.28%
10-1
50, 99.38 + 06%
1r 500,
89 43 0.25%
10-2
100, 99.36 + 0.06%
10-4
10-3
100
epoch
epoch
Figure I: Number of relative synaptic operations during backpropagation for different error scaling factors function of the epoch. Numbers based activation values of the equivalent integer activation ANN. Test performance with error is given for each a. (a) MNIST. (b) CIFARIO.
(a)
(b)
10a
min. SynOps/oome fconnection
spoci
min. SynOps SynOps/connen feonnettnoo
epoch:
spodt
10
epoch::
10
spoch:
35
opoch: 205
10
10-2
10-4
10-3
10-3
fel layer
pony22
6:3
[el
conv3
corv2
layer
Figure 2: Number of relative synaptic operations during backpropagation in each layer (connections in direction of backpropagation) for different epochs. Numbers are based on activation values the equivalent integer activation ANN. (a) MNIST with a 100. (b) CIFAR10 with a = 500.
number of events and synaptic operations in SNN may therefore slightly deviate from these numbers.
In [i it can be seen that the number of operations decreases with increasing inference precision of the network. This a result of the decrease of error in the classification layer, which leads to the demonstrates how the number of operations during the backpropagation phase is distributed in the layers of the network (the float input bottom layer and average pooling layers were omitted). While propagating deeper down the network, the relative number of operations decreases and the error becomes increasingly sparse. This tendency is consistent during the whole training process for different epochs.
emission of a smaller number of error spikes. Figure 2
DISCUSSION AND CONCLUSION
Using spike-based propagation of the gradient, we demonstrated that the paradigm of event-based information propagation can be easily translated to the backpropagation algorithm. We have not only shown that competitive inference performance can be achieved, but also that gradient propagation seems particularly suitable to leverage spike-based processing by exploiting high signal sparsity. For both forward and backward propagation, SpikeGrad requires a similar event- based communication infrastructure between neurons. which simplifies possible spiking hardware implementation. One restriction of our algorithm is the need for negative spikes, which could be problematic for some neuromorphic hardware platforms.
8

---

Published as a conference paper at ICLR 2020
In particular the topology used for CIFARIO classification is rather large for the given task. We decided to use the same topologies as other state-of-the-art SNNs to allow for better comparison. The relatively large number of parameters of these topologies may to certain extent explain the very low number of relative synaptic operations we observe during backpropagation. It has been demonstrated that the number of parameters in many ANNS can be strongly reduced without significant impact on performance (Han et al.) 2016). It remains to show that the spike coding approach provides sufficient gains also in topologies that are optimized for minimal computation and memory requirements. This seems at least possible, since the reduction of operation count in SNNs is mainly caused by two factors: the first factor actual sparsity, that means the number of zeros that are produced by thresholding and the ReLU activation function. The second factor is the more fine grained distribution of activation and error values. most values are represented by small integers, only a few spikes are necessary to encode them. The consequence is that even topologies that are compressed and therefore have lower sparsity may be able to profit from spike coding if most of the activation or error values small. This is typically the case for networks quantized to low precision integer values (see for instance (Zhou et al. 2018)) or Wu et al. (2018a)) and could lead to interesting synergies between our based computation model and ANN quantization methods.
Our analysis the potential energy efficiency of SpikeGrad a dedicated SNN architecture is based on counting the number of basic operations that performed the synapses of neuron. This does not take into account the potential hardware overhead that is introduced by the implementation of the IF neuron model and the event-based processing scheme. If the low operation count can in the end be translated into higher processing speed or increased energy efficiency, compared to normal ANN, depends on the particular SNN hardware implementation. Such dedicated neuromorphic hardware is the subject of current research efforts (see for instance Merolla et al. (2014) and Davies et al. (2018). ) We assumed in this work that such system might be built and used operation count as our principal metric, since it enables us easily approximate the energy consumption of spiking neuromorphic chip if the average cost of such an operation known. Our work shows that a hardware implementation of SrikeGrad might profit sufficiently from the low operation count that energy speed improvements could be possible at least in certain cases.

---

Published as a conference paper at ICLR 2020
REFERENCES
Pierre Baldi and Peter Sadowski. A theory of local learning, the learning channel, and the optimality of backpropagation. Neural Networks, 38:51-74, 2016.
Sergey Bartunov, Adam Santoro, Blake A. Richards, Geoffrey Hinton, and Timothy P. Lillicrap. Assessing the Scalability of Biologically-Motivateee Deep Learning Algorithms and Architectures. In Advances Neural Information Processing Systems (NIPS), 2018.
Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-term memory and learning-to-lean in networks of spiking neurons. In Advances in Neural Information Processing Systems (NIPS), 2018.
Jonathan Binas, Giacomo Indiveri, and Michael Pfeiffer. Deep counter networks for asynchronous brent-based processing. ar I. 1000700v1, NIPS 2016 workshop "Computing with Spikes". 2016.
M. Davies, N. Srinivasa, T. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain, Y. Liao, C. Lin, A. Lines, R. Liu, D. Mathaikutty, S. McCoy, A. Paul, Tse, G. Venkataramanan, Y. Weng, A. Wild, Y. Yang, and H. Wang. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1):82 99, January 2018.
Peter U. Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, and Michael Pfeiffer. Fast- Classifying, High-Accuracy Spiking Deep Networks Through Weight and Threshold Balancing. In International Joint Conference on Neural Networks (IJCNN), 2015.
Steven K. Esser, Paul A. Merolla, John V. Arthur, Andrew S. Cassidy, Rathinakumar Appuswamy, Alexander Andreopoulos, David J. Berg, Jeffrey L. McKinstry, Timothy Melano, Davis R. Barch, Carmelo di Nolfo, Pallab Datta, Armon Amir, Brian Taba, Myron D. Flickner, and Dharmendra Modha. Convolutional networks for fast, energy-efficient neuromorphic computing. Proceedings of the National Academy of Sciences,
(104)):11441-11
2016.
Jordan Guerguiev, Timothy P. Lillicrap, and Blake Richards. Towards deep learning with segregated dendrites. eLife, 2017.
Song Han, Huizi Mao, and William J. Dally. Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffmann Coding. In International Conference on Learning Representations (ICLR), 2016.
Yingyezhe Jin, Peng Li, and Wenrui Zhang. Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks. In Advances in Neural Information Processing Systems (NIPS), pp. 7005-7015, 2018.
Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer. Training Deep Spiking Neural Networks Using Backpropagation. Frontiers in Neuroscience, (10:508), 2016.
Timothy P. Lillicrap, Daniel Cownden, Douglas B. Tweed, and Colin J. Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature Communications, 2016.
Wolfgang Maass. Networks of Spiking Neurons: The Third Generation of Neural Network Models. Electronic Colloquium on Computational Complexity, 19)166911177, 1997.
Paul A. Merolla, John V. Arthur, Rodrigo Alvarez-Icaza, Andrew S. Cassidy, Jun Sawada, Filipp Akopyan, Bryan L. Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, Bernard Brezzo, Ivan Vo, Steven K. Esser, Rathinakumar Appuswamy, Brian Taba, Arnon Amir, Myron D. Flickner, William P. Risk, Rajit Manohar, and Dharmendra S. Modha. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345(6197):668-775 2014.
Emre O. Neftci, Charles Augustine, Paul Somnath, and Georgios Detorakis. Event-Driven Random Backpropagation: Enabling Neuromorphic Deep Learning Machines. Frontiers in Neuroscience, 11(324), 2017.
Arild Nøkland. Direct Feedback Aligmment Provides Learning in Deep Neural Networks. In Advances in Neural Information Processing Systems (NIPS), 2016.
10

---

Published as a conference paper at ICLR 2020
Peter O' O'Connor and Max Welling. Deep Spiking Networks. arXiv:16022.88323222 NIPS 2016 workshop "Computing with Spikes". 2016.
Bodo Rueckauer, Iulia-Alexanee Alexandaaa Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Con- version of Continuous- Valued Deep Networks to Efficient Event-Driven Networks for Image Classification. Frontiers in Neuroscience, 1 11(682), 2017.
Joo Sacramento, Rui Costa, Y Bengio, and Walter Senn. Dendritic cortical microcircuits approximate the backpropagation algorithm. In Advances in Neural Information Processing Systems (NIPS), 2018.
Arash Samadi, Timothy P. Lillicrap, and Douglas B. Tweed. Deep Learning with Dynamic Spiking Neurons and Fixed Feedback Weights. Neural Computation, -29, 2017.
Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. Going Deeper in Spiking Neural Networks: VGG and Residual Architectures. Frontiers in Neuroscience, 13:95, 2019.
William Severa, Craig M. Vineyard, Ryan Dellana, Stephen J. Verzi, and James B. Aimone. Training deep neural networks for binary communication with the Whetstone method. Nature Machine Intelligence, (1):86-94, 2019.
Sumit Bam Shrestha and Garrick Orchard. Slayer:
Spike layer error reassignment in time. In Advances in Neural Information Processing Systems (NIPS), pp. 1412-1421, 2018.
Johannes C. Thiele, Olivier Bichler, Antoine Dupret, Sergio Solinas, and Giacomo Indiveri. A Spiking Network for Inference Relations Trained with Neuromorphic Backpropagation. In (to appear) International Joint Conference Neural Networks (IJCNN, arXiv: 1903.09344,,,011
Jibin Wu, Yansong Chua, Malu Zhang, Qu Yang. Guoqi Li. and Haizhou Li. Deep Spiking Neural 2019.
Network with Spike Count based Learning Rule.
Yujei Wu, Lei Deng, Guoqi Li, Chen Feng, and Luping Shi. Training and Inference with Integers in Deep Neural Networks. In International Conference on Learning Representations (ICLR), Vancouver, Canada, number 12:331, 2018a.
Yujei Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks. Frontiers in Neuroscience, (12:331), 2018b.
Yujei Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Direct Training for Spiking Neural Networks: Faster, Larger, Better. 10X9::88090057990 2018c.
Shihui Yin, Shreyas K. Venkataramanaiah, Gregory K. Chen, Ram Krishnamurthy, Yu Cao, Chaitali Chakrabarti, and Jae-sun Seo. Algorithm and Hardware Design Discrete-Time Spiking Neural Networks Based on Back Propagation with Binary Activations.
2017.
Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural Computation, (30):1514-1511, 2018
Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv:1606.067600 2018.
A APPENDIX
B
ADDITIONAL EXPLANATIONS
B. ROUND FUNCTION
It can be seen that the residual spike activation function covers indeed all possible cases. In particular, the boundary cases are rounded correctly: for V22 = θh, we obtain S S, 0.5. For S; 2 0. this should be rounded to round S; and for S; < 0, we should obtain round[S] S. Similarly,
11

---

Published as a conference paper at ICLR 2020
for Vi = obtain S; 0.5. For S 0, this should be rounded to round S and for S; O, we should obtain round = S..
The same reasoning applies to error propagation.
C ADDITIONAL ARCHITECTURE DETAILS
C.I AVERAGE POOLING
All pooling layers are average pooling layers, since these are much easier implement spiking neural network. Average pooling can simply be implemented as an IF neuron with constant weights:
(22)
P
where p is the number of neurons in the pooling window of the neuron.
C.2 DROPOUT
Dropout can be implemented by an additional gating variable which randomly removes some neurons during forward and backward propagation. During learning, the activations of the other neurons have be rescaled by the inverse of the dropout probability 1/(1 Pdrop). This can be implemented in the SNN framework by rescaling the threshold values by (1 Pdrop)
C.3 MOMENTUM
While momentum not required for the basic algorithm, we used it to speed up training. SNN implementation, this could be implemented by adding additional accumulators that save past gradient information.
C.4
PARAMETER PRECISION
All real valued variables coded with 32 bit floating point variables.
D EXPERIMENTS
D.1 COMPUTING FRAMEWORK
All experiments are performed with custom CUDA/cuDNN accelerated C++ code. Training is performed on RTX 2080 Ti graphic cards.
D.2
ERROR BARS
For all experiments, the means, errors and maximal values are calculated over 20 simulation runs.
D.3 PREPROCESSING
No preprocessing is used on the MNIST dataset. We separate the training set of size 60000 into 50000 training and 10000 validation examples, which are used to monitor convergence. Testing performed on the test set of 10000 examples.
For CIFAR10, the values of color channels are divided by 255 and then rescaled by factor of 20 to trigger sufficient activation in the network. The usual preprocessing and data augmentation is applied. For data augmentation images are padded with the image mean value by two pixels on each side and random slices of 32 32 extracted. Additionally, images flipped randomly along the vertical axis. We separate the training set of size 50000 into 40000 training and 10000 validation examples, which used to monitor convergence. Testing is performed on the test set of 10000 examples.
Final scores obtained without retraining the validation set.
12

---

Published as a conference paper at ICLR 2020
Table 3: Parameters used for training of MNIST.
Parameter
Value
Epochs
60
Batch size
128
OF
1.0
Op
1.0
Base learning rate 7)
0.1
Momentum
0.9
Decay policy
mutliply by 0.1 every 20 epochs
Dropout (fel only)
0.5
Table 4: Parameters used for training of CIFARIO.
Parameter
Value
Epochs
300
Batch size
16
Oi
1.0
Obp
1.0
Base learning rate 77
0.001
Momentum
0.9
Decay policy
multiply by after 150 epochs
Dropout (all except pool and top)
0.2
D.4 HYPERPARAMETERS
The hyperparameters for training can be seen in tables B and
The maximal inference performances in the results were achieved with a 100 for MNIST and 400 for CIFARI0.
13


Reviews:
[{"review_id": "rJlYxLacKr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "EDIT After Rebuttal: My understanding of the contributions of this paper has improved. I now increase my score to a weak accept.\n\nThis paper proposes a new backpropagation algorithm learning algorithm \"SpikeGrad\" for Spike-based neural network paradigm. Simulating this algorithm on a classical hardware would require a lot of time-steps. To circumvent this, they show how to construct a corresponding artificial neural net (that can be trained using the traditional gradient based algorithms) which is equivalent to the spiking neural net. Using this equivalence they simulate a large scale SNN on many real-world dataset (first paper to do so). In particular, they use MNIST and CIFAR-10 for this purpose. They show that training a fixed architecture using their method is comparable to other prior work which uses high-precision gradients to train them. They also show how to exploit sparsity of the gradient in the back propagation for SNN.\n\nThis paper is hard-to-follow for someone not familiar with the background material. In particular, without looking at prior literature it was hard to understand that \"integrate and fire neuron model\" is essentially the feedforward mechanism for the SNN. I would suggest the authors make this a bit more explicit. Moreover, it would serve the structuring of the paper to have a formal \"Preliminaries\" section, where all known stuff goes. It was hard to discern what is new in this paper, and what is from prior work and these are mixed in section 2. For instance, section 2 states \"SpikeGrad\" algorithm; but the main contribution (ie., the back propagation algorithm) only appears in the middle of this section. Likewise, I think section 3 can be arranged better. In particular, the equivalence is a \"formal\" statement and thus, could be stated as a theorem followed by a proof. It will also make it explicit as to what does it mean by an \"equivalent\" network. In fact, it is still not clear to me at this point what that statement means. Could you please elaborate this in the rebuttal? \n\nRegarding the conceptual contribution of this paper, if I understood things correctly, the main claim is that they give a new way to train SNN whose performance on MNIST and CIFAR-10 is comparable to other works. The second contribution is that they give the equivalence between ANN and SNN (point above). It is also unclear to me what the point regarding the sparse gradient in the backpropagation in the experimental section is trying to make? Could you please clarify this in the rebuttal as well?\n\nAt this point, the writing of this paper leaves me with many unanswered questions that needs to be addressed before I can make a more informed decision. Please provide those in the rebuttal and based on those will update my final score. But with my current understanding of this paper, I think this does not meet the bar. The contributions in this paper do not seem too significant.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "rJgnwZSTYH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper shows how SNNs can integrate backpropagation with a second accumulation module that discretizes errors into spikes. In other words, the authors show how to translate event-based information propagation (used by SNNs) into a backpropagation method, which is the main contribution. The description of establishing the equivalence between an ANN and SNN in Section 3 is mostly well done. They perform empirical studies on MNIST and CIFAR-10 to demonstrate the effectiveness of SpikeGrad. \n\n\n= Main Concerns =\n\n1. It seems not clear to me why it is a good idea to introduce a second compartment with a threshold in each neuron as described in Eqn. 6. \n2. I very much like the idea of \"translating\" an SNN into an ANN. I'm a bit confused about the computational complexity estimation of the SNN. In particular, it is not clear to me what is the practical implication of {n-n_min}/n_min < 0.035. Furthermore, in https://openreview.net/forum?id=rkg6PhNKDr, for ANNs on CIFAR-10, freezing 80% of the VGG19 parameters from the third epoch onwards only results in 0.24% drop in accuracy. I wonder if the advantage of SNN over ANN is still huge in this case. \n3. I do not think experiments on MNIST are very useful, as the task is a toy task. I would suggest running at least one more experiment on CIFAR-100 or TinyImagenet. \n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "BkgT1iYcYS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a first framework of large-scale spiking neural network that exploits the the sparsity of the gradient during backpropagation, to save training energy. \nLater, it provides detailed analysis to show the equivalence of accumulated response and the corresponding integer activation ANN.\n\nThe paper is clearly written. The forward and backward process with the spike activation and error activation function respectively to save energy is clearly demonstrated. The response equivalence of the proposed architecture and integer ANNs provides theoretical gurantee for the good performance in training accuracy.  \n\nMy only concern is the lack of empirical support for the energy saving of the proposal. In order to show the effectiveness of the proposal, the authors should also provide time consumptions of the SNN and normal ANN. A mere comparison on sparsity doesn't really show the advantage of the proposal, since there is some computational overhead. For a system-level improvement, it's not sufficient to show the epoch-operation relation.\nIf the authors could provide wall clock time comparisons, I will consider raising my score. \n\n==========\nI find the response of the authors reasonable and address some of my concerns. Therefore I'm willing to raise my score to 6. ", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a learning framework for spiking neural networks that exploits the sparsity of the gradient during backpropagation to reduce the computational cost of training. The method is evaluated against prior works that use full precision gradients and shown comparable performance. Overall, the contribution of the paper is solid, and after a constructive rebuttal cycle, all reviewers reached a consensus of weak accept. Therefore, I recommend accepting this submission.
Decision: Accept (Poster)