Title: Direct Training of SNN using Local Zeroth Order Method
Venue: NeurIPS 2023
Accepted: yes
Keywords: Spiking Neural Network

Zeroth Order

Surrogate Gradient
Abstract: Spiking neural networks are becoming increasingly popular for their low energy requirement in real-world tasks with accuracy comparable to traditional ANNs. SNN training algorithms face the loss of gradient information and non-differentiability due to the Heaviside function in minimizing the model loss over model parameters. To circumvent this problem, the surrogate method employs a differentiable approximation of the Heaviside function in the backward pass, while the forward pass continues to use the Heaviside as the spiking function. We propose to use the zeroth-order technique at the local or neuron level in training SNNs, motivated by its regularizing and potential energy-efficient effects and establish a theoretical connection between it and the existing surrogate methods. We perform experimental validation of the technique on standard static datasets (CIFAR-10, CIFAR-100, ImageNet-100) and neuromorphic datasets (DVS-CIFAR-10, DVS-Gesture, N-Caltech-101, NCARS) and obtain results that offer improvement over the state-of-the-art results. The proposed method also lends itself to efficient implementations of the back-propagation method, which could provide 3-4 times overall speedup in training time. The code is available at \url{https://github.com/BhaskarMukhoty/LocalZO}.

Direct Training of SNN using Local Zeroth Order Method

Abstract

Spiking neural networks are becoming increasingly popular for their low energy requirement in real-world tasks with accuracy comparable to traditional ANNs. SNN training algorithms face the loss of gradient information and non-differentiability due to the Heaviside function in minimizing the model loss over model parameters. To circumvent this problem, the surrogate method employs a differentiable approximation of the Heaviside function in the backward pass, while the forward pass continues to use the Heaviside as the spiking function. We propose to use the zerothorder technique at the local or neuron level in training SNNs, motivated by its regularizing and potential energy-efficient effects and establish a theoretical connection between it and the existing surrogate methods. We perform experimental validation of the technique on standard static datasets (CIFAR-10, CIFAR-100, ImageNet100) and neuromorphic datasets (DVS-CIFAR-10, DVS-Gesture, N-Caltech-101, NCARS) and obtain results that offer improvement over the state-of-the-art results. The proposed method also lends itself to efficient implementations of the backpropagation method, which could provide 3-4 times overall speedup in training time. The code is available at https://github.com/BhaskarMukhoty/LocalZO .

1 Introduction

Biological neural networks are known to be significantly more energy efficient than their artificial avatars - the artificial neural networks (ANN). Unlike ANNs, biological neurons use spike trains to communicate and process information asynchronously. [27] To closely emulate biological neurons, spiking neural networks (SNN) use binary activation to send information to the neighbouring neurons when the membrane potential exceeds membrane threshold. The event-driven binary activation simplifies the accumulation of input potential and reduces the computation burden when the spikes are sparse. Specialized neuromorphic hardware [11] is designed to carry out such event-driven and sparse computations in an energy-efficient way [32, 20].

2

There are broadly three categories of training SNNs: ANN-to-SNN conversion, unsupervised and supervised. The first one is based on the principle that parameters for SNN can be inferred from the corresponding ANN architecture [6, 15, 5]. Although training SNNs through this method achieves performance comparable to ANNs, it suffers from the long latency needed in SNNs to emulate the corresponding ANN or from retraining of ANNs required to achieve near lossless conversion [10]. The unsupervised training is biologically inspired and uses local learning to adjust the SNN parameters [14]. Although it is the most energy-efficient one among the three as it is implementable on neuromorphic chips [11], it still lags in its performance compared to ANN-to-SNN conversion and supervised training.

Finally, supervised training is a method of direct training of SNNs by using back-propagation (through time). As such, it faces two main challenges. The first is due to the nature of SNNs, or more precisely, due to the Heaviside activation of neurons (applied to the difference between the membrane potential and threshold). As the derivative of the Heaviside function is zero, except at zero where it is not defined, back-propagation does not convey any information for the SNN to learn [16]. One of the most popular ways to circumvent this drawback is to use surrogate methods, where a derivative of a surrogate function is used in the backward pass during training. Due to their simplicity, surrogate methods have been widely used and have seen tremendous success in various supervised learning tasks [36, 28]. However, large and complex network architectures, the time-recursive nature of SNNs, and the fact that the training is oblivious of the sparsity of spikes in SNNs make surrogate methods quite time and energy-consuming.

Regarding regularization or energy efficiency during direct training of SNNs, only a few methods have been proposed addressing these topics together or separately, most of which deal with the forward propagation in SNNs. For example, [1] uses stochastic neurons to increase energy efficiency during inference. More recently, [42] uses regularization during the training to increase the sparsity of spikes, reducing the computational burden and energy consumption. Further, [7] performs the forward pass on a neuromorphic chip, while the backward pass is performed on a standard GPU. Although these methods improve the SNN models' performance, they do not significantly reduce the computational burden or provide the potential to do so. On the other hand, [31] introduces a threshold for surrogate gradients (or suggests using only a surrogate with bounded support). However, introducing gradient thresholds has the drawback of limiting the full potential of surrogates during training.

This paper proposes a direct training method for SNNs based on the zeroth order technique. We apply it locally, at the neuronal level - hence dubbed Local Zeroth Order (LOCALZO) - with twofold benefits: regularization, which comes as a side-effect of the introduced randomness that is naturally associated with this technique, as well as a threshold for gradient backpropagation in the style of [31] which translates to potential energy-efficient training when properly implemented.

We summarize the main contributions of the paper as follows:

We introduce zeroth order techniques in SNN training at a local level. We provide extensive theoretical properties of the method, relating it to the surrogate gradients via the internal distributions used in LOCALZO.

We experimentally demonstrate the main properties of LOCALZO: its superior performance compared to baselines when it comes to generalizations, its ability to simulate arbitrary surrogates as well as its property to speed up the training process, which translates to energy-efficient training.

2 Background

2.1 Spiking neuron dynamics

An SNN consists of Leaky Integrate and Fire neurons (LIF) governed by differential equations in continuous time [19]. They are generally approximated by discrete dynamics given in the form of

recurrence equations,

where u ( l ) i [ t ] denote the membrane potential of i -th neuron in the layer l at time-step (discrete) t , which recurrently depends upon its previous potential (with scaling factor β < 1 ) and spikes x ( l -1) j [ t ] received from the neurons of previous layers weighted by w ij . The neuron generates binary spike x ( l ) i [ t ] whenever the membrane potential exceeds threshold u th , represented by the Heaviside function h , followed by a reset effect on the membrane potential.

To implement the back-propagation of training loss through the network, one must obtain a derivative of the spike function, which poses a significant challenge in its original form represented as:

where we denote u := u ( l ) i [ t ] -u th . To avoid the entire gradient becoming zero, known as the dead neuron problem, the surrogate gradient method (referred to as SURROGATE) redefines the derivative using a surrogate:

Here, the function g ( u ) can be, for example, the derivative of the Sigmoid function (see section 4.4), but in general, one takes a scaled probability density function as a surrogate (see Section 4.2 for more details).

2.2 Motivation

Classically, the purpose of dropout in ANNs is to prevent a complex and powerful network from over-fitting the training data, which consequently implies better generalization properties [3]. In the forward pass, one usually assigns to each neuron in a targeted layer a probability of being 'switched-off' during both forward and backward passes, and this probability does not change during the training. Moreover, the 'activity' of the neuron, however we may define it, does not affect whether the neuron will be switched on or off.

Our motivation comes along these lines: how to introduce a dropout-like regularizing effect in the training of SNNs, but keeping in mind the temporal dimension of the data, as well as the neuron activity at that particular moment (heuristically, a more active neuron would be kept 'on' with a high probability (randomness of the dropout) while a less active one would be 'switched off', again with high probability, in a sense to be made precise shortly). Generally speaking, our idea consists of the following two steps: 1) For each spiking neuron of our SNN network, measure how active the neuron is in the forward pass at each time step t . Here, we define the activity based on how far the current membrane potential of the neuron u [ t ] is from the firing threshold u th (this idea comes from [31]). However, unlike in [31] where the sole distance is the determining factor, we introduce the effect of randomness via a fixed PDF, say λ , sample z from it and say the neuron is active at time t if | u [ t ] -u th | < c | z | , where c is some upfront fixed constant. 2) In the backward pass at the time t , if the neuron is dubbed active, we will apply some surrogate function g ( u [ t ] -u th ) ; otherwise, we will take the surrogate to be 0 (hence, switching off the propagation of gradients through the neuron in the latter case).

Having said this, we ask ourselves the final question: can we have a systematic way of choosing functions λ and g so that the expected surrogate we use (with respect to λ ) equals the one we chose upfront? A simple yet elegant solution that satisfies all of the above comes with zeroth order methods.

Zeroth order technique is a popular gradient-free method [26], well studied in neural networks literature. To briefly introduce it, we consider a function f : R d → R , that we intend to minimize

using gradient descent, for which the gradient may not be available or even undefined. The zerothorder method estimates the gradients using function outputs: given a scalar δ > 0 , the 2-point ZO is defined as

where, z ∼ λ is a random direction with E z ∼ λ [ ∥ z ∥ 2 ] = 1 and ϕ ( d ) is a dimension dependent factor, with d being the dimension. However, to approximate the full gradient of f up to a constant squared error, we need an average of O ( d ) samples of G 2 , which becomes computationally challenging when d is large, such as the number of learnable parameters of the neural network. Though well studied in the literature, properties of 2-point ZO are known only for the continuous functions [29, 4]. In the present context, we will apply it locally to the Heaviside function that produces the outputs of spiking neurons, and we justify this by providing the necessary theoretical background for doing so.

3 The LOCALZO algorithm

Applying ZO on a global scale is challenging due to the large dimensionality of neural networks[25]. Since the non-differentiability of SNN is introduced by the Heaviside function at the neuronal level, we apply the 2-point ZO method on h : R →{ 0 , 1 } itself,

where u = u ( l ) i [ t ] -u th and z is sampled from some distribution λ . We may average the 2-point ZO gradient over a few samples z k so that the LOCALZO derivative of the spike function is defined as:

where, the number of samples, m , is a hyper-parameter to the LOCALZO method. We implement this at the neuronal level of the back-propagation routine, where the forward pass uses the Heaviside function, and the backward pass uses equation (6). Note that the gradient dx i [ t ] dt being non-zero naturally determines the active neurons of the backward pass (as was discussed in Section 2.2), which can be inferred from the forward pass through the neuron. Algorithm 1 gives an abstract representation of the process at a neuronal level, which hints that the backward call is redundant when the neuron has a zero gradient.

4 Theoretical Properties of LOCALZO

4.1 General ZO function

For the theoretical results around LOCALZO, we consider a more general function than what was suggested by eqn. 5, in the form

where the new constant α is an integer different from 0, while δ is a positive real number (so, for example, setting α = 1 in (7), we obtain (5)).

The integer α is somewhat a normalizing constant, which allows obtaining different surrogates as the expectation of function G 2 ( u ; z, δ ) when z is sampled from suitable distributions.

In practice, taking α = ± 1 will suffice to account for most of the surrogates found in the literature. The role of δ is somewhat different, as it controls the 'shape' of the surrogate (narrowing it and stretching around zero). The role of each constant will be more evident from what follows (see section 4.4).

̸

4.2 Surrogate functions

Definition 4.1. We say that a function g : R → R ≥ 0 is a surrogate function (gradient surrogate) if it is even, non-decreasing on the interval ( -∞ , 0) and c := ∫ ∞ -∞ g ( z ) dz < ∞ .

Note that the integral ∫ ∞ -∞ g ( z ) dz is convergent (as g ( z ) is non-negative), but possibly can be ∞ and the last condition means that the function 1 c g ( t ) is a probability density function. The first two conditions, that is, requirements for the function to be even and non-decreasing, are not essential but rather practical and consistent with examples from SNN literature.

Note that the function G : R → [0 , 1] , defined as G ( t ) := 1 c ∫ t -∞ g ( z ) dz is the corresponding cumulative distribution function (for PDF 1 c g ( t ) ). Moreover, it is not difficult to see that its graph is 'symmetric' around point (0 , 1 2 ) (or in more precise terms, G ( t ) = 1 -G ( -t ) ), hence G ( t ) can be seen as an approximation of Heaviside function h ( t ) . Then, its derivative d dt G ( t ) = 1 c g ( t ) can serve as an approximation of the 'derivative' of h ( t ) , or in other words, as its surrogate, which somewhat justifies the terminology.

Finally, one may note that 'true' surrogates would correspond to those functions g for which c = 1 . However, the reason we allow c to be different from 1 is again practical and simplifies the derivation of the results that follow. We note once again that allowing general c is in consistency with examples used in the literature.

4.3 Surrogates and ZO

To be in line with classic results around the ZO method and gradient approximation of functions, we pose ourselves two fundamental questions: What sort of functions in variable u can be obtained as the expectation of G 2 ( u ; z, δ ) when z is sampled from a suitable distribution λ , and, given some function g ( u ) , can we find a distribution λ such that we obtain g ( u ) in the expectation when z is sampled from λ ?

Two theorems that follow answer these questions and are the core of this section. The main player in both of the questions is the expected value of G 2 ( u ; z, δ ) , so we start by analyzing it more precisely. Let λ be a distribution, λ ( t ) its PDF for which we assume that it is even and that ∫ ∞ 0 z α λ ( z ) dz < ∞ . Then, we may write

It becomes apparent from eqn. (8) that E z ∼ λ [ G 2 ( u ; z, δ )] has some properties of surrogate functions (it is even and non-decreasing on R < 0 ). The proofs of the following results are detailed in the appendix A.

Lemma 1. Assume further that ∫ ∞ 0 z α +1 λ ( z ) dz < ∞ . Then, E z ∼ λ [ G 2 ( u ; z, δ )] is a surrogate function.

Theorem 2. Let λ be a distribution and λ ( t ) its corresponding PDF. Assume that integrals ∫ ∞ 0 t α λ ( t ) dt and ∫ ∞ 0 t α +1 λ ( t ) dt exist and are finite. Let further ˜ λ be the distribution with corresponding PDF function

where c is the scaling constant (such that ∫ ∞ -∞ ˜ λ ( z ) dz = 1 ). Then,

For our next result, which answers the second question we asked at the beginning of this section, note that a surrogate function is differentiable almost everywhere, which follows from the Lebesgue theorem on the differentiability of monotone functions. So, taking derivatives here is understood in an 'almost everywhere' sense.

3

2

1

0

1

2

3

u

=

ui

vth

0.0

0.2

0.4

0.6

0.8

1.0

z

N

(0, 1)

h

h

,

=1

h

,

=0.5

3

2

1

0

1

2

3

u

=

ui

vth

0.0

0.2

0.4

0.6

0.8

1.0

z

([

3,

3])

h

h

,

=1

h

,

=0.5

3

2

1

0

1

2

3

u

=

ui

vth

0.0

0.2

0.4

0.6

0.8

1.0

z

Laplace

(0,

1

2

)

h

h

,

=1

h

,

=0.5

Theorem 3. Let g ( u ) be a surrogate function. Suppose further that c = -2 δ 2 ∫ ∞ 0 1 z α g ′ ( zδ ) dz < ∞ and put λ ( z ) = -δ 2 cz α g ′ ( zδ ) (so that λ ( z ) is a PDF). Then,

4.4 Application of Theorem 2 and 3

Next, we spell out the results of Theorem 2 applied to some standard distributions, with α = 1 . For clarity, all the distributions' parameters are chosen so that the scaling constant of the resulting surrogate is 1. One may consult Figure 1 for the visual representation of the results, while the details are provided in the appendix A.1. Recall that the standard normal distribution N (0 , 1) has PDF of the form 1 √ 2 π exp( -z 2 2 ) . Consequently, it is straightforward to obtain

In appendix, A.1, we further derive surrogates when z is sampled from Uniform and Laplace distribution.

We recall that Theorem 3 provides a way to derive distributions for arbitrary surrogate functions ( that satisfy the conditions of the theorem). Consider the Sigmoid surrogate function, where the differentiable Sigmoid function approximates the Heaviside [43]. The corresponding surrogate gradient is given by,

Observe that g ( u ) satisfies our definition of a surrogate ( g ( u ) being even, non-decreasing on ( -∞ , 0) and ∫ ∞ -∞ g ( u ) du = 1 < ∞ ). Thus, according to Theorem 3, we have c = -2 δ 2 ∫ ∞ 0 g ′ ( tδ ) t dt = δ 2 k 2 a 2 where, a := √ 1 0 . 4262 . The corresponding PDF is given by

Observe that the temperature parameter k comes from the surrogate to be simulated, while δ is used by LOCALZO. Appendix A.2 provides calculations and distribution corresponding to the popular Fast-sigmoid surrogate, followed by a description of the inverse sampling method that can be used to simulate sampling for arbitrary distributions using the uniform distribution.

4.5 Expected back-propagation threshold for LOCALZO

To study the energy efficiency of the LOCALZO method when training SNNs, we compute the expected threshold ˜ B th for the activity of the neurons, i.e. the expectation of the quantity | z | δ when z is sampled from a distribution λ . It is used in the experimental section when comparing our method with the alternative energy-efficient method [31]. The expected threshold values are presented in Table 1 ( m denotes the number of samples used in (6)), while the details of the derivations can be found in A.3.

0

1000

2000

3000

4000

Number of gradient updates

0.5

1.0

1.5

2.0

Loss

SparseGrad

LocalZO

0

1000

2000

3000

4000

Number of gradient updates

2.0

2.5

3.0

3.5

4.0

4.5

Speedup

SparseGrad

LocalZO

0

1000

2000

3000

4000

Number of gradient updates

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Percentage of active neurons (%)

Second Hidden Layer, SparseGrad

5 Experiments

5.1 General Performance of LOCALZO

First, we evaluate the generalization performance of LOCALZO as a substitute for the surrogate method on standard static image datasets such as CIFAR-10, CIFAR-100[22], ImageNet-100[12] and neuromorphic datasets such as DVS-CIFAR-10[24], DVS-Gesture[2], N-Caltech-101[30], N-CARS[37]. More specifically, by substituting surrogate gradients with gradients computed with our method, we com-

bined LOCALZO with contemporary state-of-the-art methods in direct training of SNNs, such as tDBN [45] and TET [13]. We opted for these two techniques due to their high performance and different natures (the former is a batch normalization technique, while the latter introduces auxiliary loss in training). We refer to the combined methods as LocalZO+tDBN and LocalZO+TET, respectively. Following the recent results, we choose ResNet-19[45] architecture for CIFAR datasets, SEW-ResNet34[17] for ImageNet-100, and VGGSNN[13] architecture for the neuromorphic datasets. Table2 summarizes these results, where LOCALZO is implemented with m = 5 , δ = 0 . 5 , except for DVSCIFAR-10 where we have used m = 1 . Table 7 reports the detailed training hyper-parameters for each dataset.

Results on Static Datasets : The static datasets CIFAR-10, CIFAR-100, and ImageNet-100, have the number of classes mentioned in the dataset name, while each class respectively have (5000, 1000), (500, 100), and (1300, 50) train and test images. We use constant encoding to supply the images to the SNN network, with standard latencies as mentioned in Table 2. We train separate models for different latencies and report the test results respectively. The results of CIFAR datasets are reported with cutout augmentation following TET[13], while for ImageNet-100, we use standard data augmentation (random resized crop, random horizontal flip), without and with the ImageNet Policy[9]. We note that in conjunction with LOCALZO, both tDBN and TET improve their performance significantly for all the latencies. For example, in CIFAR-10, LOCALZO improves TET between 0 . 9 -1% for different latencies, while for CIFAR-100, it improves TET by 2 . 5 -3 . 5% . The ImageNet-100 results reported in Table 2 were further enhanced to 83.33% with m = 20 and batch size 72.

Results on Neuromorphic Datasets : Events of the neuromorphic datasets are collected into event frames of dimension (2 × H × W ) where H and W stand for the height and width of the frame and are resized to (2 × 48 × 48) . The temporal events are collected into a fixed number (10) of frames (a.k.a. bins), treated as the effective temporal dimension for the SNN. The experiments are reported without and with standard data augmentation. For DVS-CIFAR-10, the TET result is re-computed to avoid an obscure frame preparation step, which is replaced by an open-source routine. We obtain results superior to the state-of-the-art for DVS-CIFAR-10[13] and DVS-Gesture[17]. For N-Caltech-101 and NCARS, the improvements are 1.2% and 2.4%, respectively, compared to the state-of-the-art[18, 35].

⋆ our implementation, † pre-trained with ImageNet, ‡ 83.33 % with m = 20

5.2 Performance on Energy Efficient Implementation

In the energy-efficient implementation of the back-propagation [31], the optimization of the network weights takes place in a layer-wise fashion through the unrolling of recurrence of equation (1) w.r.t time. As the active neurons of each layer for every time step are inferred from the forward pass, gradients of only active neurons are required to be saved for the backward pass, hence saving the computation requirement of the backward pass. One may refer to [31] for further details of this implementation framework. To compare, we supply SPARSEGRAD method the surrogate approximated by LOCALZO, as per section 4.4. The SPARSEGRAD algorithm also requires a backpropagation threshold parameter, B th , to control the number of active neurons participating in the back-propagation. We supply it the expected back-propagation threshold ˜ B th of LOCALZO as obtained in sections 4.5. We follow the same experimental setting as in [31] for a fair comparison. We use a fully connected LIF neural network with two hidden layers of 200 neurons each and input and output layers. We train every model for 20 epochs and report the average training and test accuracies computed over five trials. We compute the speedup of SPARSEGRAD and LOCALZO, with respect to the full surrogate without truncation, that uses standard back-propagation. The backward speedup (Back.) captures the number of times the backward pass of a gradient update is faster, while the overall speedup (Over.) considers the total time for the forward and the backward pass and then computes the speedup. The speedup reported is averaged over all the gradient updates and the experimental trials.

We compare the performance of the algorithms on three datasets: 1) Neuromorphic-MNIST (NMNIST) [30], which consists of static images of handwritten digits (between 0 and 9) converted to temporal spiking data using visual neuromorphic sensors; 2) Spiking Heidelberg Digits (SHD) [8], a neuromorphic audio dataset consisting of spoken digits (between 0 and 9) in English and German language, totalling 20 classes. To challenge the generalizability of the learning task, 81% of test inputs of this dataset are new voice samples, which are not present in the training data; 3) Fashion-MNIST (FMNIST) [41] dataset is converted using temporal encoding to convert static gray-scale images based on the principle that each input neuron spikes only once, and a higher intensity spike results in an earlier spike.

Table 3 provides a comparison of the algorithms, using surrogates corresponding to the Normal and Sigmoid, with δ = 0 . 05 and m = 1 . For the normal distribution, we supply SPARSEGRAD algorithm the backpropagation threshold ˜ B th obtained in Table 1. In the section 4.4, we derived distributions corresponding to the Sigmoid surrogate. We use inverse transform sampling (see A.2.3), and take the temperature parameter k = a/δ ≈ 30 . 63 so that c = δ 2 k 2 a 2 = 1 and supply SPARSEGRAD method the corresponding back-propagation threshold, ˜ B th = 0 . 766 δ . The LOCALZO method offers better test accuracies than SPARSEGRAD, with a slight compromise in speedup due to the sampling of random variable z . The difference between training and test accuracies for the SHD dataset can be attributed to the unseen voice samples in the test data[8].

Figure 2 shows the training loss, overall speedup, and percentage of active

neurons after each gradient step for the Sigmoid surrogate. The sparseness of active neurons (under 0.6%) explains the reduced computational requirement that translates to the speedup.

We further implement LOCALZO with δ = 0 . 5 , z ∼ Normal (0 , 1) to train a CNN architecture (Input- 16C5BN-LIF-MP2-32C5-BN-LIF-MP2-800FC-10) and compare it with the corresponding surrogate gradient algorithm. Figure 3 shows the corresponding sparsity of the methods by plotting the number of zero elements at a neuronal level. The plot suggests that during the training, LOCALZO exhibits higher sparsity gradients than the surrogate method.

Ablation study: Table4 further shows the test accuracy of the LOCALZO method and overall speed-up for a wide range of values of m with z ∼ Normal (0 , 1) . Like Table 3, the experiments are repeated five times and mean test accuracy is reported along with standard deviation (Std.).

0

200

400

600

800

Number of Gradient updates

0.00

0.05

0.10

0.15

0.20

Fraction of zero neurons

Surrogate

LocalZO

In general, by increasing m , the method approximates the surrogate better, still offers the regularizing effect and potentially improves the generalization, but also requires more computation. Larger m leads to more non-zero gradients at the neuronal level in the backward pass, reducing overall speedup. On the other hand, smaller m introduces higher randomness (less 'controlled'), still yielding regularization, which helps obtain better generalization, as well as potential speed-up. In conclusion, m should be treated as a hyper-parameter, its value depending on the training setting itself. In our experiments, we chose m = 1 or 5 for most of the experiments, as a proof of concept, but also because it offers a nice balance between the speed-up and performance.

6 Discussions

We propose a new direct training algorithm for SNNs that establishes a formal connection between the standard surrogate methods and the zeroth order method applied locally to the neurons. The method introduces systematic randomness in the training that helps in better generalization. The method simultaneously lends itself to efficient back-propagation. We experimentally demonstrate the efficiency of the proposed method in terms of speed-up obtained in training under specialized implementations and its top generalization performance when combined with other training methods, ameliorating their respective strengths.

References

Mohammed Alawad, Hong-Jun Yoon, and Georgia Tourassi. Energy efficient stochastic-based deep spiking neural networks for sparse datasets. In 2017 IEEE International Conference on Big Data (Big Data) , pages 311-318, 2017.

Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. A low power, fully event-based gesture recognition system. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 7243-7252, 2017.

Pierre Baldi and Peter J Sadowski. Understanding dropout. Advances in neural information processing systems , 26, 2013.

Albert S Berahas, Liyuan Cao, Krzysztof Choromanski, and Katya Scheinberg. A theoretical and empirical comparison of gradient approximations in derivative-free optimization. Foundations of Computational Mathematics , 22(2):507-560, 2022.

Tong Bu, Wei Fang, Jianhao Ding, PengLin Dai, Zhaofei Yu, and Tiejun Huang. Optimal annsnn conversion for high-accuracy and ultra-low-latency spiking neural networks. In International Conference on Learning Representations , 2021.

Yongqiang Cao, Yang Chen, and Deepak Khosla. Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision , 113(1):54-66, 2015.

Benjamin Cramer, Sebastian Billaudelle, Simeon Kanya, Aron Leibfried, Andreas Gr¨ ubl, Vitali Karasenko, Christian Pehle, Korbinian Schreiber, Yannik Stradmann, Johannes Weis, et al. Surrogate gradients for analog neuromorphic computing. Proceedings of the National Academy of Sciences , 119(4):e2109194119, 2022.

Benjamin Cramer, Yannik Stradmann, Johannes Schemmel, and Friedemann Zenke. The heidelberg spiking data sets for the systematic evaluation of spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems , 2020.

Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501 , 2018.

Simon Davidson and Steve B. Furber. Comparison of artificial and spiking neural networks on digital hardware. Frontiers in Neuroscience , 15, 2021.

Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro , 38(1):82-99, 2018.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu. Temporal efficient training of spiking neural network via gradient re-weighting. arXiv preprint arXiv:2202.11946 , 2022.

Peter U Diehl and Matthew Cook. Unsupervised learning of digit recognition using spiketiming-dependent plasticity. Frontiers in computational neuroscience , 9:99, 2015.

Peter U Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, and Michael Pfeiffer. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In 2015 International Joint Conference on Neural Networks (IJCNN) , pages 1-8. ieee, 2015.

Jason K Eshraghian, Max Ward, Emre Neftci, Xinxin Wang, Gregor Lenz, Girish Dwivedi, Mohammed Bennamoun, Doo Seok Jeong, and Wei D Lu. Training spiking neural networks using lessons from deep learning. arXiv preprint arXiv:2109.12894 , 2021.

Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timoth´ ee Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems , 34:21056-21069, 2021.

Daniel Gehrig, Antonio Loquercio, Konstantinos G Derpanis, and Davide Scaramuzza. Endto-end learning of representations for asynchronous event-based data. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5633-5643, 2019.

Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski. Neuronal dynamics: From single neurons to networks and models of cognition . Cambridge University Press, 2014.

Seijoon Kim, Seongsik Park, Byunggook Na, and Sungroh Yoon. Spiking-yolo: spiking neural network for energy-efficient object detection. In Proceedings of the AAAI conference on artificial intelligence , volume 34, pages 11270-11277, 2020.

Youngeun Kim, Yuhang Li, Abhishek Moitra, Ruokai Yin, and Priyadarshini Panda. Sharing leaky-integrate-and-fire neurons for memory-efficient spiking neural networks. arXiv preprint arXiv:2305.18360 , 2023.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

Alexander Kugele, Thomas Pfeil, Michael Pfeiffer, and Elisabetta Chicca. Efficient processing of spatio-temporal data streams with spiking neural networks. Frontiers in Neuroscience , 14:439, 2020.

Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: an eventstream dataset for object classification. Frontiers in neuroscience , 11:309, 2017.

Yuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng, Yongqing Hai, and Shi Gu. Differentiable spike: Rethinking gradient-descent for training spiking neural networks. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems , volume 34, pages 23426-23439. Curran Associates, Inc., 2021.

Sijia Liu, Pin-Yu Chen, Bhavya Kailkhura, Gaoyuan Zhang, Alfred O Hero III, and Pramod K Varshney. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine , 37(5):43-54, 2020.

Zachary F Mainen and Terrence J Sejnowski. Reliability of spike timing in neocortical neurons. Science , 268(5216):1503-1506, 1995.

Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Processing Magazine , 36(6):51-63, 2019.

Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics , 17(2):527-566, 2017.

Garrick Orchard, Ajinkya Jayawant, Gregory K Cohen, and Nitish Thakor. Converting static image datasets to spiking neuromorphic datasets using saccades. Frontiers in neuroscience , 9:437, 2015.

Nicolas Perez-Nieves and Dan Goodman. Sparse spiking gradient descent. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems , volume 34, pages 11795-11808. Curran Associates, Inc., 2021.

Michael Pfeiffer and Thomas Pfeil. Deep learning with spiking neurons: opportunities and challenges. Frontiers in neuroscience , page 774, 2018.

Nitin Rathi and Kaushik Roy. Diet-snn: Direct input encoding with leakage and threshold optimization in deep spiking neural networks. arXiv preprint arXiv:2008.03658 , 2020.

Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy. Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. arXiv preprint arXiv:2005.01807 , 2020.

Simon Schaefer, Daniel Gehrig, and Davide Scaramuzza. Aegnn: Asynchronous event-based graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12371-12381, 2022.

Sumit B Shrestha and Garrick Orchard. Slayer: Spike layer error reassignment in time. Advances in neural information processing systems , 31, 2018.

Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagorce, and Ryad Benosman. Hats: Histograms of averaged time surfaces for robust event-based object classification. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 17311740, 2018.

Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience , 12:331, 2018.

Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI conference on artificial intelligence , volume 33, pages 1311-1318, 2019.

Zhenzhi Wu, Hehui Zhang, Yihan Lin, Guoqi Li, Meng Wang, and Ye Tang. Liaf-net: Leaky integrate and analog fire network for lightweight and efficient spatiotemporal information processing. IEEE Transactions on Neural Networks and Learning Systems , 33(11):6249-6262, 2021.

Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 , 2017.

Yulong Yan, Haoming Chu, Yi Jin, Yuxiang Huan, Zhuo Zou, and Lirong Zheng. Backpropagation with sparsity regularization for spiking neural network learning. Frontiers in Neuroscience , 16, 2022.

Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural computation , 30(6):1514-1541, 2018.

Wenrui Zhang and Peng Li. Temporal spike sequence learning via backpropagation for deep spiking neural networks. Advances in Neural Information Processing Systems , 33:12022-12033, 2020.

Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with directly-trained larger spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 11062-11070, 2021.

Appendix: Direct Training of SNN using Local Zeroth Order Method

Appendix A Proofs of theoretical results

Lemma 1. Assume further that ∫ ∞ 0 z α +1 λ ( z ) dz < ∞ . Then, E z ∼ λ [ G 2 ( u ; z, δ )] is a surrogate function.

Proof. Based on our remark above, the only thing left to prove is that the integral ∫ ∞ -∞ E z ∼ λ [ G 2 ( u ; z, δ )] du is finite. To this end, we have (by using equation (8))

which proves the lemma, as by assumptions the resulting integral is finite.

Theorem 2. Let λ be a distribution and λ ( t ) its corresponding PDF. Assume that integrals ∫ ∞ 0 t α λ ( t ) dt and ∫ ∞ 0 t α +1 λ ( t ) dt exist and are finite. Let further ˜ λ be the distribution with corresponding PDF function

where c is the scaling constant (such that ∫ ∞ -∞ ˜ λ ( z ) dz = 1 ). Then,

Proof. We have

which coincides with (8).

For our following result, note that a surrogate function is differentiable almost everywhere, which follows from the Lebesgue theorem on the differentiability of monotone functions. So, taking derivatives here is understood in an 'almost everywhere' sense.

Theorem 3. Let g ( u ) be a surrogate function. Suppose further that c = -2 δ 2 ∫ ∞ 0 1 z α g ′ ( zδ ) dz < ∞ and put λ ( z ) = -δ 2 cz α g ′ ( zδ ) (so that λ ( z ) is a PDF). Then,

Proof. Let us assume that u ≥ 0 (the other case is similar). Then,

which after change of variables u = δz becomes g ( u ) and finishes our proof.

A.1 Obtaining full-surrogates on Expectation

We demonstrate performance of LOCALZO over different distributions of z , such as standard Normal, Uniform ([ √ 3 , √ 3]) and Laplace (0 , 1 √ 2 ) , for m ∈ { 1 , 5 } and δ = 0 . 05 . The distributions are of unit variance so that parameter δ is comparable across the methods. We supply SPARSEGRAD algorithm the back-propagation threshold ˜ B th obtained in Table 1. Table 5 shows the performance of the methods on the N-MNIST dataset in terms of accuracy and speedup. The LOCALZO method obtains better train and test accuracies for all cases with a slight compromise in the speedup, except for the uniform distribution where it offers better speedup for m = 1 compared to the SPARSEGRAD method.

A.1.1 From standard Gaussian

Recall that the standard normal distribution N (0 , 1) has PDF of the form 1 √ 2 π exp( -z 2 2 ) . Consequently, it is straightforward to obtain

A.1.2 From Uniform Continuous

Consider the PDF of a continuous uniform distribution:

where a < b are some real numbers. For the distribution to be even and the resulting scaling constant of the surrogate to be 1 (which translates to E [ z ] = 0 and E [ z 2 ] = 1 , respectively) we set, a = - √ 3 , b = √ 3 . Then,

A.1.3 From Laplacian Distribution

The PDF of Laplace distribution is given by:

with mean µ and variance 2 b 2 . Setting, b = 1 √ 2 and µ = 0 and using (5) we obtain,

A.2 Simulating a specific Surrogate

A.2.1 Sigmoid

Consider the Sigmoid surrogate function, where the Heaviside is approximated by the differentiable Sigmoid function [43]. The corresponding surrogate gradient is given by,

and,

Observe that g ( u ) satisfies our definition of a surrogate ( g ( u ) being even, non-decreasing on ( -∞ , 0) and ∫ ∞ -∞ g ( u ) du = 1 < ∞ ). Thus, according to Theorem 3, we have

where, a := √ 1 0 . 4262 . The corresponding PDF is given by

Note that the temperature parameter k comes from the surrogate to be simulated, while δ is used by LOCALZO. We compute the expected back-propagation threshold of SPARSEGRAD for m = 1 as, ˜ B th = δ E z ∼ λ [ | z | ] , with,

A.2.2 Fast Sigmoid

Consider also the Fast Sigmoid surrogate gradient [43, 31] that avoids computing the exponential function in Sigmoid to obtain the gradient:

We choose α = -1 (note that α = 1 does not work in this case) and apply theorem 3 so that,

The PDF is then given by

To compute the expected back-propagation threshold, we note,

The above integral does not converge. However, if we consider finite support [-a, a], we may compute, 2 k ∫ a 0 x 2 (1+ x ) 3 dx

A.2.3 Inverse Transform Sampling

To simulate a given surrogate in LOCALZO, one needs to sample from the corresponding distribution described by the PDF λ . Given a sample r ∼ Unif ([0 , 1]) and the inverse CDF Λ -1 of the distribution, the inverse sampling technique returns, Λ -1 ( r ) , as a sample from the distribution. Suppose the inverse CDF is not computable analytically from the PDF (or not implementable practically). In that case, we may choose a finite support over which the PDF is evaluated at a sufficiently dense set of points and compute the discretized CDF using the Riemann sum. The inverse discretized CDF is then computed empirically and stored as a list for a finite number of points (spaced regularly) between [0 , 1] . Sampling from the uniform distribution then amounts to randomly choosing the indices of the list and picking the corresponding inverse CDF values.

A.3 Expected Back-propagation Thresholds

In what follows, m is the number of samples used in (6), while k is the index of a particular sample. To compute the expected back-propagation threshold, we observe that a neuron is inactive in LOCALZO back-propagation if,

Assume z k ∼ λ , where λ ( t ) denotes the PDF of the sampling distribution, with the corresponding CDF denoted by F z k . The PDF, ˜ λ , of the random variable | z k | is given by

The corresponding CDF is obtained by integrating the previous expression,

If we denote the PDF of the random variable t as ˆ λ , we obtain

Finally, the expected back-propagation threshold takes the form

In the cases of distributions used in experimental sections, the previous expression simplifies. Table 1 gives the numerical values for some particular m . To obtain an expected back-propagation threshold, we would like to evaluate:

For the standard normal distribution, λ = Normal (0 , 1) we have, F | z | ( t ) = erf( t √ 2 ) giving,

For uniform continuous, λ = Unif ([ - √ 3 , √ 3]) , we have, F | z | ( t ) = t √ 3 giving,

For Laplace distribution, λ = Laplace (0 , 1 √ 2 ) , we have, F | z | ( t ) = 1 -exp( - √ 2 t ) ,

Appendix B Additional Results

B.1 Computational Speedup in SPARSEGRAD

The SPARSEGRAD frame-work derives speedup by performing the back-propagation in a layerwise fashion. To summarize their finding, let us start with eqn. 1.

which, after unfolding the recurrence and using the fact that u ( l ) i [0] = 0 , can be restated as:

Further note that,

0

1000

2000

3000

4000

Number of gradient updates

0.5

1.0

1.5

2.0

Loss

SparseGrad

LocalZO

0

1000

2000

3000

4000

Number of gradient updates

0.5

1.0

1.5

2.0

Loss

SparseGrad

LocalZO

0

1000

2000

3000

4000

Number of gradient updates

0.5

1.0

1.5

2.0

Loss

SparseGrad

LocalZO

0

1000

2000

3000

4000

Number of gradient updates

10 2

10 3

SparseGrad

LocalZO

0

1000

2000

3000

4000

Number of gradient updates

10 2

10 3

SparseGrad

LocalZO

0

1000

2000

3000

4000

Number of gradient updates

10 2

10 3

SparseGrad

LocalZO

0

1000

2000

3000

4000

Number of gradient updates

2.75

3.00

3.25

3.50

3.75

4.00

4.25

4.50

Overall Speedup

SparseGrad

LocalZO

0

1000

2000

3000

4000

Number of gradient updates

2.25

2.50

2.75

3.00

3.25

3.50

3.75

4.00

4.25

Overall Speedup

SparseGrad

LocalZO

0

1000

2000

3000

4000

Number of gradient updates

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

5.5

Overall Speedup

SparseGrad

LocalZO

where, the input trace , ∑ t -1 k =0 β t -k -1 x ( l -1) j [ k ] can be computed in a layer-wise fashion from the forward propagation. Note that, by ignoring the reset mechanism in the gradient the computation, the gradient of the loss can be written as:

where, the term ∂l [ t ] ∂x [ t ] is coming from the next layer, the term ∂x [ t ] ∂u [ t ] is the given by 5, so it can be computed in the forward propagation along with the input trace . Thus, whenever, many neurons are inactive, i.e., ∂x [ t ] ∂u [ t ] = 0 the SPARSEGRAD frame-work can reduce the computation burden of the back-propagation.

B.2 Further comparison with SPARSEGRAD

In the section 4.4, we derived surrogates corresponding to distributions and distributions corresponding to popular surrogates. We implement LOCALZO, with δ = 0 . 05 for different distribution

such as Normal (0 , 1) , Laplace (1 , 1 √ 2 ) , and Unif ([ - √ 3 , √ 3]) . They all have a unit variance to ensure δ is comparable across the distributions. The corresponding back-propagation thresholds for SPARSEGRAD are derived in Table 1.

For the Sigmoid surrogate, we take the temperature parameter k = a/δ ≈ 30 . 63 so that c = δ 2 k 2 a 2 = 1 for the corresponding PDF derived in eqn.14. We supply SPARSEGRAD method the corresponding back-propagation threshold, ˜ B th = 0 . 766 δ , as obtained in eqn. 15.

For the Fast Sigmoid surrogate, we choose k = 100 following [31] so that c = 2 k . To compute the expected back-propagation threshold, we consider finite support [ -10 , 10] used in the inverse transform sampling of z and evaluate ˜ B th = 0 . 0461 . Table 5 reports accuracies obtained by LOCALZO and SPARSEGRAD across various sampling distributions, with m = 1 . LOCALZO offers better accuracies compared to SPARSEGRAD across all the distributions, with a slight compromise in speed-up in most cases. For uniform distribution, LOCALZO offers even better speed-up than SPARSEGRAD.

Table 6 reports the details of the comparison over the N-MNIST dataset for sampling z from various distributions with m = 5 . LOCALZO method obtains better test accuracies. The SPARSEGRAD method is supplied with the respective surrogate and back-propagation threshold as computed in

Optimizer: Adam with betas: (0.9; 0.999), Rate Scheduler: cosine annealing

Tab.1. The LOCALZO method achieves better accuracies than SPARSEGRAD for all the distributions, though the speed-up is reduced due to higher sampling cost at m = 5 .

In table 8, we further provide the hyper-parameters for the comparison in the SPARSEGRAD framework, which are replicated from their work. The FMNIST data uses latency encoding, where each input pixel x , is converted to a single spike, the spike timing is determined by:

Table 7 gives the hyper-parameter setting for comparison in general framework reported in Tab. 2.

B.3 Dropout effect in LOCALZO

A fixed neuron has an active gradient for m = 1 , only if | u i [ t ] -u th | < | z | δ as described in eqn5. Observe that u i [ t ] is specific to an input data point and time-step. The same neuron can be inactive for another data-point, or at a different time-step. Moreover, a neuron can have a non-zero gradient with respect to a single data point, even if the spikes are zero. Fixing a neuron and time-step, we plot the distribution of membrane potential (minus membrane threshold), spikes, and non-zero gradients over different data points across batches, where we implement LOCALZO on MNIST data with δ = 0 . 5 . The plot captures the sparsity of spikes and sparsity of neuron gradients for LOCALZO.

1.75

1.50

1.25

1.00

0.75

0.50

0.25

0.00

(

ui

[

t

]

uth

) in 100 evenly spaced bins

0

50

100

150

200

250

count

t=1

2.0

1.5

1.0

0.5

0.0

0.5

(

ui

[

t

]

uth

) in 100 evenly spaced bins

0

50

100

150

200

250

count

t=2

2.5

2.0

1.5

1.0

0.5

0.0

(

ui

[

t

]

uth

) in 100 evenly spaced bins

0

50

100

150

200

count

t=4

x[t]=0

x[t]=1

0

1000

2000

3000

4000

5000

6000

count

Spike, t=1

x[t]=0

x[t]=1

0

1000

2000

3000

4000

5000

6000

count

Spike, t=2

x[t]=0

x[t]=1

0

1000

2000

3000

4000

5000

6000

count

Spike, t=4

inactive

active

0

1000

2000

3000

4000

5000

6000

count

activity of a neuron, t=1

inactive

active

0

1000

2000

3000

4000

5000

count

activity of a neuron, t=2

inactive

active

0

1000

2000

3000

4000

5000

count

activity of a neuron, t=4

Peer reviews:
comments: SUMMARY:
This paper proposes a Local Zeroth Order method, which can fit arbitrary surrogate functions by sampling a group of variables from a certain distribution. Experiments have verified the superiority of the proposed scheme.

STRENGTHS:
1. The authors‘ idea about fitting arbitrary surrogate functions by sampling is very novel.
2. Theoretical analysis about ZO function is persuasive and profound.
3. Relevant experiments, especially on ResNet-19, can show the obvious advantages of ZO function.

WEAKNESSES:
1. If time permits, I suggest that the authors can supplement their experiments on large-scale datasets (e.g. ImageNet).

QUESTIONS:
1. I note that the firing threshold used by the authors in the code seems to be a fixed value of 1, so has the expected threshold proposed by the authors in Section 4.5 been used in actual experiments?

LIMITATIONS:
I think that the proposed ZO function may require more extensive experiments, especially on large-scale datatsets. Overall, I think this paper have met the acceptance standard.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a new method for training spiking neural networks\n(SNNs). To estimate the gradient of the step function for spike\ngeneration, they propose to directly estimate the gradient by local\nsampling around the point of derivation and averaging the linearly\ncalculated slopes, a method that is known in other fields as\nzeroth-order estimation of the gradient of an arbitrary (potentially\nnon continuous) function. They show that, if the distribution used for\nsampling is chosen appropriately, the approximated gradient approaches\nin expectation the common surrogate gradient approach, where an\narbitrary surrogate function is used for smoothening the gradient of\nthe step function.  They further show how to choose the sampling\ndistribution for a couple of given surrogate functions. Finally, they\nshow that their approach outperforms previous methods on common\nbenchmark tasks in terms of accuracy. In particular, gradient\ncalculation is sparser (since gradients are deemed zero for larger\nsampled values) in particular if only few values are sampled, and thus\nmore efficiently to compute (unless one truncates the surrogate, which\nperforms similarly in this regard).\n\nSTRENGTHS:\nThe paper makes a very interesting connection of the surrogates used\nto train SNNs to a sampling based zeroth order method, and shows that\nand how (in theoretical depth) samples can be generated to approximate\nany surrogate used in literature.\n\nWhile the method does not significantly changes the state-of-the-art,\nsince the SparseGrad method (that truncates the surrogates to have a\nsparser backward pass) seem to perform very similar in accuracy\nas well as compute efficiency, it nevertheless provides a very\ninteresting theoretical underpinning of how sparseness can be\nintroduced into the backward pass.\n\nThe paper is also very well written, clearly motivated and\nprovides clear mathematical proofs of their claims.\n\nWEAKNESSES:\nWhile the theoretical connection is interesting it could be argued\nthat the existing SparseGrad method performs similarly in practice, so that there is not much of an improvement of the state-of-the-art. However, still, the accuracy seems to be slightly improved due to the\nintroduced randomness that apparently leads to slightly better\ngeneralization.\n\nQUESTIONS:\nThe general approach is motivated by dropout, where activity is\nrandomly changed in the forward pass. However, the authors actually\n\"apply\" this principle only for the backward pass (as demanded by\nthe zeroth-order approach). Since $m=1$ for most cases in the\nexperiments, I wonder how results would change if spikes in the\nforward pass would be also randomly generated (and not only the\n(pseudo) backward spikes), in other words, when the pseudo-backward\nspikes where actually also the forward spikes. This would be similar\nto neuron with random spike generation, which actually is biologically\nvery realistic. The randomness in the forward pass would effectively\nsmooth over the Heaviside function as well. However, maybe in practice\nthe activity profile of the SNN would be impacted too much. That might\nbe an interesting discussion point.\n\nLIMITATIONS:\nIt would have been interesting to discuss / compare the methods with SNNs that use some form of random spike generation.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

comments: SUMMARY:
This paper proposes a new direct training algorithm for SNN, combining the standard surrogate methods and zeroth order method together. The algorithm applies the 2-point zeroth order method on the Heaviside function to generate a surrogate gradient, which is more efficient. The author applied his method to various dataset such as CIFAR-10 and CIFAR-100 and outperforms the SOTA methods.

STRENGTHS:
1.	Due to the efficiency of zeroth order method, this algorithm is computational friendly, reducing the computational burden significantly. The gradients are only back propagated through active neurons.
2.	According to theorem 3, the algorithm is able to simulate arbitrary surrogate, which means this algorithm has great expression capability.

WEAKNESSES:
1.This paper has limited novelty. It seems simply to be a combination the forward gradient method [1] and sparse gradient together [2]. 

2.Worry about the scalability of the proposed method, as forward gradient method is not widely applicable to large networks [1].

3.The experiments only show marginal improvements.




[1] SCALING FORWARD GRADIENT WITH LOCAL LOSSES. ICLR 2023
[2] Sparse Spiking Gradient Descent. NeurIPS 2021.

QUESTIONS:
How to choose the distribution function of z for different tasks?

LIMITATIONS:
No. The author should discuss the limitation of the scalability of the forward gradient method in large network.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

comments: SUMMARY:
This paper presented a direct SNN training algorithm that alleviates the loss of the gradient information and improves the performance of the SNN on multiple datasets, including both static image datasets and dynamic vision datasets.

STRENGTHS:
- Rigorous theoretical and empirical analysis to justify the necessity of the Zeroth Order technique. 

- Comprehensive experimental results with improved performance against the previous SoTA method.

WEAKNESSES:
- The proposed method is only verified on small-scale datasets such as CIFAR-10/100 and DVS-CIFAR-10. Since the proposed method shows improved performance on simple vision tasks, it is necessary to further verify the performance on large-scale datasets such as ImageNet-1K or ImageNet-100. 

- I understand that static CIFAR datasets are standard vision tasks in most of the recent direct SNN training methods. However, one of the major advantages of SNN is the ability to process the spatial temporal visual information which widely exists in the captures of the event-based sensor/camera. Solely verifying the proposed method with the DVS-CIFAR10 dataset is insufficient, it is very useful for the research community if the author can provide the performance with more DVS datasets (e.g., IBM gesture, NCARS, N-CalTech101), which was reported by [1]. 

[1] AEGNN: Asynchronous Event-based Graph Neural Networks, CVPR'22. 

- I wonder if the proposed method is applicable to the SpikeFormer [2]? 

[2] Spikformer: When Spiking Neural Network Meets Transformer, ICLR'23

QUESTIONS:
Please refer to the Weaknesses section.

LIMITATIONS:
The major limitation of the paper is the insufficient experimental results.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07