SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes

1 INTRODUCTION . Spiking neural networks ( SNNs ) are a new generation of artificial neural network models ( Maass , 1997 ) that try to harness properties of biological neurons to build energy efficient spiking neuromorphic systems . Processing in traditional artificial neural networks ( ANNs ) is based on parallel processing of operations on dense tensors of fixed length . In contrast to this , spiking neuromorphic systems communicate with asynchronous events , which allows dynamic , data dependent computation that can exploit high temporal and spatial sparsity . The recent years have seen a large number of approaches devoted to optimization of spiking neural networks with the backpropagation algorithm , either by converting ANNs to SNNs ( Diehl et al. , 2015 ; Esser et al. , 2016 ; Rueckauer et al. , 2017 ; Sengupta et al. , 2019 ) or by simulating spikes explicitly in the forward pass and optimizing these dynamics with floating point gradients ( Lee et al. , 2016 ; Yin et al. , 2017 ; Wu et al. , 2018b ; c ; Severa et al. , 2019 ; Jin et al. , 2018 ; Bellec et al. , 2018 ; Zenke & Ganguli , 2018 ; Shrestha & Orchard , 2018 ) . These methods aim to optimize SNNs for efficient inference , and backpropagation is performed offline on a standard computing system . It would however be desirable to also enable on-chip learning in neuromorphic chips using the power of the backpropagation algorithm , and to maintain the advantages of spike-based processing also in the error propagation phase . Previous work on the implementation of backpropagation with spikes is mostly concerned with biological plausibility . A non-spiking version of biologically inspired backpropagation is presented by Sacramento et al . ( 2018 ) . Guerguiev et al . ( 2017 ) , Neftci et al . ( 2017 ) and Samadi et al . ( 2017 ) introduce spike-based versions of the backpropagation algorithm using variants of ( direct ) feedback alignment ( Lillicrap et al. , 2016 ; Nøkland , 2016 ) . The exact backpropagation algorithm , which backpropagates through symmetric weights , might however be required to achieve good inference performance on large-scale deep neural networks ( Baldi & Sadowski , 2016 ; Bartunov et al. , 2018 ) . O ’ Connor & Welling ( 2016 ) and Thiele et al . ( 2019 ) present implementations of standard backpropagation where the gradient is coded into spikes and propagated through symmetric weights . In the same spirit , our work is mostly concerned with exploiting spike based information encoding for energy efficient processing , which means that inference performance and operational simplicity will be preferred over biological plausibility and complex neuron models . We demonstrate how backpropagation can be seamlessly integrated into the spiking neural network framework by using a second accumulation compartment that discretizes the error into spikes . By additionally weighting the activity counters by the learning rate , we obtain a system that is able to perform learning and inference based on accumulations and comparisons alone . As for the forward pass , this allows us to use the dynamic precision computation provided by the discretization of all operations into spike events , and to exploit the sparsity of the gradient . Using a similar reasoning as Binas et al . ( 2016 ) and Wu et al . ( 2019 ) have applied to forward propagation in SNNs , we show that the system obtained in this way can be mapped to an integer activation ANN whose activations are equivalent to the accumulated neuron responses for both the forward and the backward propagation phase . This allows us to simulate training of large-scale SNNs efficiently on graphics processing units ( GPUs ) , using their equivalent ANN . Additionally , in contrast to conversion methods that approximate pre-trained ANNs with SNNs , this method guarantees that the inference precision of the SNN will be equivalent to the ANN . In contrast to O ’ Connor & Welling ( 2016 ) , this is true for any number of spikes and arbitrary spike order . We demonstrate classification accuracies equivalent or superior to existing implementations of SNNs trained with full precision gradients , and comparable to the precision of standard ANNs using similar topologies . This is the first time competitive classification performances are reported on the CIFAR10 and CIFAR100 datasets using a large-scale SNN where both training and inference are fully implemented with spikes . To the best of our knowledge , our work provides for the first time a demonstration of how the sparsity of the gradient during backpropagation could be exploited within a large-scale SNN processing structure . 2 THE SpikeGrad ALGORITHM We begin with the description of SpikeGrad , the spike-based backpropagation algorithm . The algorithm mainly consists of special implementations of the integrate-and-fire ( IF ) neuron model , that are used to encode and propagate information in the forward and backward pass . For each training example/mini-batch , integration is performed from t = 0 to t = T for the forward pass and from t = T + ∆t to t = T in the backward pass . Since no explicit time is used in the algorithm , ∆t represents symbolically the ( very short ) time between the arrival of an incoming spike and the response of the neuron , which is only used here to describe causality . Spike discretization of the forward pass For forward propagation , the architecture is described by multiple layers ( labeled by l ∈ [ 0 , L ] ) of IF neurons with integration variable V li ( t ) and threshold Θff : V li ( t+ ∆t ) = V l i ( t ) −Θffsli ( t ) + ∑ j wlijs l−1 j ( t ) , V l i ( 0 ) = b l i . ( 1 ) The variable wlij is the weight and b l i a bias value . The spike activation function s l i ( t ) ∈ { −1 , 0 , 1 } is a function which triggers a signed spike event depending on the internal variables of the neuron . It will be shown later that the specific choice of the activation function is fundamental for the mapping to an equivalent ANN . After a neuron has fired , its integration variable is decremented or incremented by the threshold value Θff , which is represented by the second term on the r.h.s . of equation 1 . As a representation of the neuron activity , we use a trace xli ( t ) which accumulates spike information over a single example : xli ( t+ ∆t ) = x l i ( t ) + ηs l i ( t ) . ( 2 ) By weighting the activity with the learning rate η we avoid performing a multiplication when weighting the input with the learning rate for the weight update equation 8 . Implementation of implicit ReLU and surrogate activation function derivative It is possible to define an implicit activation function based on how the neuron variables affect the spike activation function sli ( t ) . In our implementation , we use the following fully symmetric function to represent linear activation functions ( used for instance in pooling layers ) : sl , lini ( V li ( t ) ) : = 1 if V li ( t ) ≥ Θff −1 if V li ( t ) ≤ −Θff 0 otherwise . ( 3 ) The following function corresponds to the rectified linear unit ( ReLU ) activation function : sl , ReLUi ( V li ( t ) , x l i ( t ) ) : = 1 if V li ( t ) ≥ Θff −1 if V li ( t ) ≤ −Θff and xli ( t ) > 0 0 otherwise . ( 4 ) The pseudo-derivative of the activation function is denoted symbolically by S ′l i . We use S ′l , lin i ( T ) = 1 for the linear case . For the ReLU , we use a surrogate of the form : S ′l , ReLU i ( T ) : = { 1 if V li ( T ) > 0 or x l i ( T ) > 0 0 otherwise . ( 5 ) These choices will be motivated in the following sections . Note that the derivatives depend only on the final states of the neurons at time T . Discretization of the error into spikes For gradient backpropagation , we introduce a second compartment with threshold Θbp in each neuron , which integrates error signals from higher layers . The process discretizes errors in the same fashion as the forward pass discretizes an input signal into a sequence of signed spike signals : U li ( t+ ∆t ) = U l i ( t ) −Θbpzli ( t ) + ∑ k wl+1ki δ l+1 k ( t ) . ( 6 ) To this end , we introduce a ternary error spike activation function zli ( t ) ∈ { −1 , 0 , 1 } which is defined in analogy to equation 3 using the error integration variable U li ( t ) and the backpropagation threshold Θbp . The error is then obtained by gating this ternarized variable zli ( t ) with one of the surrogate activation function derivatives of the previous section ( linear or ReLU ) : δli ( t ) = z l i ( t ) S ′l i ( T ) . ( 7 ) This ternary spike signal is backpropagated through the weights to the lower layers and also applied in the update rule of the weight increment accumulator ωlij : ωlij ( t+ ∆t ) = ω l ij ( t ) − δli ( t ) xl−1j ( T ) , ( 8 ) which is triggered every time an error spike ( equation 7 ) is backpropagated . The weight updates are accumulated during error propagation and are applied after propagation is finished to update each weight simultaneously . In this way , the backpropagation of errors and the weight update will , exactly as forward propagation , only involve additions and comparisons of floating point numbers . The SpikeGrad algorithm can also be expressed in an event-based formulation , described in algorithms 1 , 2 and 3 . This formulation is closer to how the algorithm would be implemented in an actual SNN hardware implementation of the IF firing dynamics . Loss function and error scale We use the cross entropy loss function in the final layer applied to the softmax of the total integration V Li ( T ) ( no spikes are triggered in the top layer during inference ) . This requires more complex operations than accumulations , but is negligible if the number of classes is small . To make sure that sufficient error spikes are triggered in the top layer , and that error spikes arrive even in the lowest layer of the network , we apply a scaling factor α to the error values before transferring them to ULi . This scaling factor also implicitly sets the precision of the gradient , since a higher number of spikes means that a large range of values can be represented . To counteract the relative increase of the gradient scale , the learning rates have to be rescaled by a factor 1/α . Input encoding As pointed out in Rueckauer et al . ( 2017 ) and Wu et al . ( 2018c ) , it is crucial to maintain the full precision of the input image to obtain good performances on complex standard benchmarks with SNNs . One possibility is to encode the input in a large number of spikes ( Sengupta et al. , 2019 ) . Another possibility , which has been shown to require a much lower number of spikes in the network , is to multiply the input values directly with the weights of the first layer ( just like in a standard ANN ) . The drawback is that the first layer then requires multiplication operations . The additional cost of this procedure may however be negligible if all other layers can profit from spike-based computation . This problematic does not exist for stimuli which are natively encoded in spikes . Algorithm 1 Forward function PROPAGATE ( [ l , i , j ] , s ) V li ← V li + s · wlij sli ← sli ( V li , xli ) . spike activation function if sli 6= 0 then V li ← V li − sli ·Θff xli ← xli + ηsli for k in l + 1 connected to i do PROPAGATE ( [ l + 1 , k , i ] , sli ) Algorithm 2 Backward function BACKPROPAGATE ( [ l , i , k ] , δ ) U li ← U li + δ · w l+1 ki zli ← zli ( U li ) . error activation function δli ← zli · S ′l i if zli 6= 0 then U li ← U li − zli ·Θbp for j in layer l − 1 connected to i do BACKPROPAGATE ( [ l − 1 , j , i ] , δli ) ωlij ← ωlij − δli · x l−1 j Algorithm 3 Training of single example/batch init : V ← b , U ← 0 , x← 0 , ω ← 0 . variables in bold describe all neurons in network/layer while input spikes sini do for k in l = 0 receiving sini do . spikes corresponding to training input PROPAGATE ( [ 0 , k , i ] , sini ) S′ ← S′ ( V , x ) . calculate surrogate derivatives UL ← α · ∂L/∂ V L . calculate classfication error for i in l = L do while |ULi | ≥ Θbp do . backpropagate error spikes BACKPROPAGATE ( [ L , i , − ] , 0 ) . last layer receives no error w ← w + ω . update weights with weight update accumulator

Summary: This paper shows how SNNs can integrate backpropagation with a second accumulation module that discretizes errors into spikes. In other words, the authors show how to translate event-based information propagation (used by SNNs) into a backpropagation method, which is the main contribution. The description of establishing the equivalence between an ANN and SNN in Section 3 is mostly well done. They perform empirical studies on MNIST and CIFAR-10 to demonstrate the effectiveness of SpikeGrad.