Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation

1 INTRODUCTION . In recent years , Spiking Neural Networks ( SNNs ) have shown promise towards enabling low-power machine intelligence with event-driven neuromorphic hardware . Founded on bio-plausibility , the neurons in an SNN compute and communicate information through discrete binary events ( or spikes ) a significant shift from the standard artificial neural networks ( ANNs ) , which process data in a real-valued ( or analog ) manner . The binary all-or-nothing spike-based communication combined with sparse temporal processing precisely make SNNs a low-power alternative to conventional ANNs . With all its appeal for power efficiency , training SNNs still remains a challenge . The discontinuous and non-differentiable nature of a spiking neuron ( generally , modeled 1https : //github.com/nitin-rathi/hybrid-snn-conversion as leaky-integrate-and-fire ( LIF ) , or integrate-and-fire ( IF ) ) poses difficulty to conduct gradient descent based backpropagation . Practically , SNNs still lag behind ANNs , in terms of performance or accuracy , in traditional learning tasks . Consequently , there has been several works over the past few years that propose different learning algorithms or learning rules for implementing deep convolutional SNNs for complex visual recognition tasks ( Wu et al. , 2019 ; Hunsberger & Eliasmith , 2015 ; Cao et al. , 2015 ) . Of all the techniques , conversion from ANN-to-SNN ( Diehl et al. , 2016 ; 2015 ; Sengupta et al. , 2019 ; Hunsberger & Eliasmith , 2015 ) has yielded state-of-the-art accuracies matching deep ANN performance for Imagenet dataset on complex architectures ( such as , VGG ( Simonyan & Zisserman , 2014 ) and ResNet ( He et al. , 2016 ) ) . In conversion , we train an ANN with ReLU neurons using gradient descent and then convert the ANN to an SNN with IF neurons by using suitable threshold balancing ( Sengupta et al. , 2019 ) . But , SNNs obtained through conversion incur large latency of 2000−2500 time steps ( measured as total number of time steps required to process a given input image2 ) . The term ‘ time step ’ defines an unit of time required to process a single input spike across all layers and represents the network latency . The large latency translates to higher energy consumption during inference , thereby , diminishing the efficiency improvements of SNNs over ANNs . To reduce the latency , spike-based backpropagation rules have been proposed that perform end-to-end gradient descent training on spike data . In spike-based backpropagation methods , the non-differentiability of the spiking neuron is handled by either approximating the spiking neuron model as continuous and differentiable ( Huh & Sejnowski , 2018 ) or by defining a surrogate gradient as a continuous approximation of the real gradient ( Wu et al. , 2018 ; Bellec et al. , 2018 ; Neftci et al. , 2019 ) . Spike-based SNN training reduces the overall latency by ∼10× ( for instance , 200 − 250 time steps required to process an input ( Lee et al. , 2019 ) ) but requires more training effort ( in terms of total training iterations ) than conversion approaches . A single feed-forward pass in ANN corresponds to multiple forward passes in SNN which is proportional to the number of time steps . In spike-based backpropagation , the backward pass requires the gradients to be integrated over the total number of time steps that increases the computation and memory complexity . The multiple-iteration training effort with exploding memory requirement ( for backward pass computations ) has limited the applicability of spike-based backpropagation methods to small datasets ( like CIFAR10 ) on simple few-layered convolutional architectures . In this work , we propose a hybrid training technique which combines ANN-SNN conversion and spike-based backpropagation that reduces the overall latency as well as decreases the training effort for convergence . We use ANN-SNN conversion as an initialization step followed by spike-based backpropagation incremental training ( that converges to optimal accuracy with few epochs due to the precursory initialization ) . Essentially , our hybrid approach of taking a converted SNN and incrementally training it using backpropagation yields improved energy-efficiency as well as higher accuracy than a model trained from scratch with only conversion or only spike-based backpropagation . In summary , this paper makes the following contributions : • We introduce a hybrid computationally-efficient training methodology for deep SNNs . We use the weights and firing thresholds of an SNN converted from an ANN as the initialization step for spike-based backpropagation . We then train this initialized network with spike-based backpropagation for few epochs to perform inference at a reduced latency or time steps . • We propose a novel spike time-dependent backpropagation ( STDB , a variant of standard spike-based backpropagation ) that computes surrogate gradient using neuron ’ s spike time . The parameter update is triggered by the occurrence of spike and the gradient is computed based on the time difference between the current time step and the most recent time step the neuron generated an output spike . This is motivated from the Hebb ’ s principle which states that the plasticity of a synapse is dependent on the spiking activity of the neurons connected to the synapse . • Our hybrid approach with the novel surrogate gradient descent allows training of large-scale SNNs without exploding memory required during spike-based backpropagation . We evaluate our hybrid approach on large SNNs ( VGG , ResNet-like architectures ) on 2SNNs process Poisson rate-coded input spike trains , wherein , each pixel in an image is converted to a Poisson-distribution based spike train with the spiking frequency proportional to the pixel value Imagenet , CIFAR datasets and show near iso-accuracy compared to similar ANNs and converted SNNs at lower compute cost and energy . 2 SPIKE TIMING DEPENDENT BACKPROPAGATION ( STDB ) . In this section , we describe the spiking neuron model , derive the equations for the proposed surrogate gradient based learning , present the weight initialization method for SNN , discuss the constraints applied for ANN-SNN conversion , and summarize the overall training methodology . 2.1 LEAKY INTEGRATE AND FIRE ( LIF ) NEURON MODEL . The neuron model defines the dynamics of the neuron ’ s internal state and the trigger for it to generate a spike . The differential equation τ dU dt = − ( U − Urest ) +RI ( 1 ) is widely used to characterize the leaky-integrate-and-fire ( LIF ) neuron model where , U is the internal state of the neuron referred as the membrane potential , Urest is the resting potential , R and I are the input resistance and the current , respectively . The above equation is valid when the membrane potential is below the threshold value ( V ) . The neuron geneartes an output spike when U > V and U is reduced to the reset potential . This representation is described in continuous domain and more suitable for biological simulations . We modify the equation to be evaluated in a discrete manner in the Pytorch framework ( Wu et al. , 2018 ) . The iterative model for a single post-neuron is described by uti = λu t−1 i + ∑ j wijo t j − vot−1i ( 2 ) ot−1i = { 1 , if ut−1i > v 0 , otherwise ( 3 ) where u is the membrane potential , subscript i and j represent the post- and pre-neuron , respectively , superscript t is the time step , λ is a constant ( < 1 ) responsible for the leak in membrane potential , w is the weight connecting the pre- and post-neuron , o is the binary output spike , and v is the firing threshold potential . The right hand side of Equation 2 has three terms : the first term calculates the leak in the membrane potential from the previous time step , the second term integrates the input from the previous layer and adds it to the membrane potential , and the third term which is outside the summation reduces the membrane potential by the threshold value if a spike is generated . This is known as soft reset as the membrane potential is lowered by v compared to hard reset where the membrane potential is reduced to the reset value . Soft reset enables the spiking neuron to carry forward the excess potential above the firing threshold to the following time step , thereby minimizing information loss . Algorithm 1 ANN-SNN conversion : initialization of weights and threshold voltages Input : Trained ANN model ( A ) , SNN model ( N ) , Input ( X ) // Copy ann weights to snn for l=1 to L do Nl.W ← Al.W end // Initialize threshold voltage to 0 V ← [ 0 , · · · , 0 ] L−1 for l=1 to L-1 do v ← 0 for t=1 to T do Ot0 ← PoissonGenerator ( X ) for k=1 to l do if k < l then // Forward ( Algorithm 3 ) end else // Pre-nonlinearity ( A ) A← Nl ( Otk−1 ) if max ( A ) > v then v ← max ( A ) end end end end V [ l ] ← v end Algorithm 2 Initialize the neuron parameters . Membrane potential ( U ) , last spike time ( S ) , dropout mask ( M ) . The initialization is performed once for every mini-batch . Input : Input ( X ) , network model ( N ) b size = X.b size h = X.height w = X.width for l=1 to L do if isintance ( Nl , Conv ) then Ul = zeros ( b size , Nl.out , h , w ) Sl = ones ( b size , Nl.out , h , w ) ∗ ( −1000 ) end else if isintance ( Nl , Linear ) then Ul = zeros ( b size , Nl.out ) Sl = ones ( b size , Nl.out ) ∗ ( −1000 ) end else if isintance ( Nl , Dropout ) then // Generate the dropout map that will be fixed for all time steps Ml = Nl ( ones ( Ul−1.shape ) ) end else if isintance ( Nl , AvgPool ) then // Reduce the width and height after average pooling layer h = h//kernel size w = w//kernel size end end 2.2 SPIKE TIMING DEPENDENT BACKPROPAGATION ( STDB ) LEARNING RULE . The neuron dynamics ( Equation 2 ) show that the neuron ’ s state at a particular time step recurrently depends on its state in previous time steps . This introduces implicit recurrent connections in the network ( Neftci et al. , 2019 ) . Therefore , the learning rule has to perform the temporal credit assignment along with the spatial credit assignment . Credit assignment refers to the process of assigning credit or blame to the network parameters according to their contribution to the loss function . Spatial credit assignment identifies structural network parameters ( like weights ) , whereas temporal credit assignment determines which past network activities contributed to the loss function . Gradient-descent learning solves both credit assignment problem : spatial credit assignment is performed by distributing error spatially across all layers using the chain rule of derivatives , and temporal credit assignment is done by unrolling the network in time and performing backpropagation through time ( BPTT ) using the same chain rule of derivatives ( Werbos et al. , 1990 ) . In BPTT , the network is unrolled for all time steps and the final output is computed as the sum of outputs from each time step . The loss function is defined on the summed output . The dynamics of the neuron in the output layer is described by Equation ( 4 ) , where the leak part is removed ( λ = 1 ) and the neuron only integrates the input without firing . This eliminates the difficulty of defining the loss function on spike count ( Lee et al. , 2019 ) . uti = u t−1 i + ∑ j wijoj ( 4 ) The number of neurons in the output layer is the same as the number of categories in the classification task . The output of the network is passed through a softmax layer that outputs a probability distribution . The loss function is defined as the cross-entropy between the true output and the network ’ s predicted distribution . L = − ∑ i yilog ( pi ) ( 5 ) pi = eu T i∑N k=1 e uTk ( 6 ) L is the loss function , y the true output , p the prediction , T the total number of time steps , uT the accumulated membrane potential of the neuron in the output layer from all time steps , and N the number of categories in the task . For deeper networks and large number of time steps the truncated version of the BPTT algorithm is used to avoid memory issues . In the truncated version the loss is computed at some time step t′ before T based on the potential accumulated till t′ . The loss is backpropagated to all layers and the loss gradients are computed and stored . At this point , the history of the computational graph is cleaned to save memory . The subsequent computation of loss gradients at later time steps ( 2t′ , 3t′ , ... T ) are summed together with the gradient at t′ to get the final gradient . The optimizer updates the parameters at T based on the sum of the gradients . Gradient descent learning has the objective of minimizing the loss function . This is achieved by backpropagating the error and updating the parameters opposite to the direction of the derivative . The derivative of the loss function w.r.t . to the membrane potential of the neuron in the final layer is described by , ∂L ∂uTi = pi − yi ( 7 ) To compute the gradient at current time step , the membrane potential at last time step ( ut−1i in Equation 4 ) is considered as an input quantity . Therefore , gradient descent updates the network parameters Wij of the output layer as , Wij = Wij − η∆Wij ( 8 ) ∆Wij = ∑ t ∂L ∂W tij = ∑ t ∂L ∂uTi ∂uTi ∂W tij = ∂L ∂uTi ∑ t ∂uTi ∂W tij ( 9 ) where η is the learning rate , and W tij represents the copy of the weight used for computation at time step t. In the output layer the neurons do not generate a spike , and hence , the issue of non-differentiability is not encountered . The update of the hidden layer parameters is described by , ∆Wij = ∑ t ∂L ∂W tij = ∑ t ∂L ∂oti ∂oti ∂uti ∂uti ∂W tij ( 10 ) where oti is the thresholding function ( Equation 3 ) whose derivative w.r.t to u t i is zero everywhere and not defined at the time of spike . The challenge of discontinuous spiking nonlinearity is resolved by introducing a surrogate gradient which is the continuous approximation of the real gradient . ∂oti ∂uti = αe−β∆t ( 11 ) where α and β are constants , ∆t is the time difference between the current time step ( t ) and the last time step the post-neuron generated a spike ( ts ) . It is an integer value whose range is from zero to the total number of time steps ( T ) . ∆t = ( t− ts ) , 0 < ∆t < T , ∆t Z ( 12 ) The values of α and β are selected depending on the value of T . If T is large β is lowered to reduce the exponential decay so a spike can contribute towards gradients for later time steps . The value of α is also reduced for large T because the gradient can propagate through many time steps . The gradient is summed at each time step and thus a large α may lead to exploding gradient . The surrogate gradient can be pre-computed for all values of ∆t and stored in a look-up table for faster computation . The parameter updates are triggered by the spiking activity but the error gradients are still non-zero for time steps following the spike time . This enables the algorithm to avoid the ‘ dead Algorithm 3 Training an SNN with surrogate gradient computed with spike timing . The network is composed of L layers . The training proceeds with mini-batch size ( batch size ) Input : Mini-batch of input ( X ) - target ( Y ) pairs , network model ( N ) , initial weights ( W ) , threshold voltage ( V ) U , S , M = InitializeNeuronParameters ( X ) [ Algorithm 2 ] // Forward propagation for t=1 to T do Ot0 = PoissonGenerator ( X ) for l=1 to L-1 do if isintance ( Nl , [ Conv , Linear ] ) then // accumulate the output of previous layer in U , soft reset when spike occurs U tl = λU t−1 l +WlO t l−1 − Vl ∗O t−1 l // generate the output ( +1 ) if U exceeds V Otl = STDB ( U t l , Vl , t ) // store the latest spike times for each neuron Stl [ O t l == 1 ] = t end else if isintance ( Nl , AvgPool ) then Otl = Nl ( O t l−1 ) end else if isintance ( Nl , Dropout ) then Otl = O t l−1 ∗Ml end end U tL = λU t−1 L +WLO t L−1 end // Backward Propagation Compute ∂L∂UL from the cross-entropy loss function using BPTT for t=T to 1 do for l=L-1 to 1 do Compute ∂L ∂Otl based on if Nl is linear , conv , pooling , etc . ∂L ∂Utl = ∂L ∂Otl ∂Otl ∂Utl = ∂L ∂Otl ∗ αe−βStl end end neuron ’ problem , where no learning happens when there is no spike . Fig . 1 shows the activation gradient for different values of ∆t , the gradient decreases exponentially for neurons that have not been active for a long time . In Hebbian models of biological learning , the parameter update is activity dependent . This is experimentally observed in spike-timing-dependent plasticity ( STDP ) learning rule which modulates the weights for pair of neurons that spike within a time window ( Song et al. , 2000 ) .

Summary: This paper examines combining two approaches of obtaining a trained spikingneural network (SNN). The first approach of previous work is converting the weights of a trained artificial neural network (ANN) with a given architecture, to the weights and thresholds of a SNN, and the second approach uses a surrogate gradient to train an SNN with backpropagation. The true novelty of the paper seems to be in showing that combining the two approaches sequentially, trains a SNN that requiresfewer timesteps to determine an output which achieves state of the art performance. This is summarized by Table 1. However, it does not mention how many epochs it takes to train an SNN from scratch, nor compare this to the total training time (ANN training + SNN fine-tuning) of their approach. They also claim a novel spike-time based surrogate gradient function (eq. 11), but it is very practicallysimilar to the ones explored in the referenced Wu. et al 2018 (eq. 27 for instance), and these should be properly contrasted showing that this novel surrogate function is actually helpful (the performance/energy efficiency might only come from the hybrid approach). The authors argue for SOTA performance in Table 2, but the comparison to other work doesn’t clearly separate their performance from the otherlisted works; For example the accuracy gain against Lee et al.,2019 only comes from the architecture being VGG16 as opposed to VGG9, as can be seen from comparing with the VGG9 architecture from Table 1, furthermore they take the sameamount of timesteps, which is supposed to be the principle gain of this work.