Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The authors aimed at addressing the inefficient issue of SNN in backpropagation with dense tensor calculations. They designed a rule to estimate the spikes' gradient iteratively and theoretically avoid a lot of calculation. 

main_review: Overall, this paper is written clearly and easy to follow. I have two major concerns about the current work. 
1. First, the algorithm was only tested on small datasets. It is not clear how it would scale up to larger datasets like CIFAR10/100 and DVS-CIFAR. 

2. As the author also stated, there lacked efficient sparse tensor operations in most auto-differentiation libraries. On the other hand, there will be an incompatibility between the sparse operation and the parallelism of GPU, which may intrinsically limit the potential application of the proposed methods to complex models. A related question is whether the proposed method can be supported by any existing neuromorphic hardware? 

Another question to clarify is why the authors used the BP time of the second layer rather than the BP time of the whole network? 
limitations_and_societal_impact: NA
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work introduces a new method to train spiking neural networks using far fewer gradients during the backwards back-propagation pass.  By only retaining the potential of so-called "active" neurons (those within a threshold epsilon of spiking) and learning through a surrogate function, these networks of spiking neurons can be sparsely trained (requiring only around 1% of the gradients).  The authors test these claims on three datasets (a time-encoded version of Fashion-MNIST, N-MNIST, and Spiking Heidelberg Digits), and achieve speedups of up to 70x and memory saving of 40% while maintaining approximately the same accuracy.  Further contributions interpret the learning rule from a Hebbian perspective and establish the savings in computational complexity.
main_review: Originality: this work combines two disparate threads in neural network improvements: the literature on sparse learning (adaptive dropout and adaptive sparsity), and methods for surrogate learning for more effective spiking neural networks.  This is a combination I have not seen combined before, offering an unexpected and valuable insight to achieve efficient training.

Quality: the authors have provided a number of valuable contributions.  They provide a clear and detailed description of their neuron model, perform a computational complexity analysis, interpret and discuss their learning rule from a Hebbian perspective, and perform a number of high-quality experiments.  Multiple datasets are used (Fashion MNIST, Neuromorphic-NMIST, and the Spiking Heidelberg Dataset), and standard errors are evaluated.  Real-world computational efficiency and sparsity is evaluated, and GPU-based speedup and memory improvements are reported.

Limitations to this work were not sufficiently discussed here, however.  For example, one dataset used is Fashion-MNIST.  Typical performance scores are around ~90%+ for this dataset, while the methodology here achieves around 80%, though it is difficult to discern from the plot and no performance is stated.  This can be problematic if sparse spiking gradient descent is proposed as an alternative to standard training: the difference in performance must be small in order for this to be a significant work.  A variety of modified training rules or even only-approximately-correlated feedback [1] and surrogate learning rules can be effective, or successful on simpler datasets, but struggle to scale to reach state-of-the-art accuracy [2][3].  

This is also true of more complex neuron architectures, whereas the architecture in this work is a three-layer fully-connected network.  As these are time-varying neurons, they are capable of quite complex computation, but an explicit evaluation against alternative architectures (LSTM, frame-based convnets, etc.) would allow a more realistic comparison point for the viability of sparse spiking gradients as a learning method.

Similarly, further baseline models or ablation studies would permit a better understanding of the sparse spiking gradient methodology.  For example, what if an equivalent random connected percentage (say 1-3%) of gradients were sampled and used from the full set of gradients?  Presumably, the most important gradients would be the ones that this work retains - those most relevant for around the spiking threshold - but this should be established in the work. How much worse would the method perform if a different set were chosen?  Additionally, as an ablation study, progressively widening the backpropagation threshold $Bth$ would provide valuable insight into the sparsity / accuracy tradeoff.

Finally, I would like to see a more detailed description of the time-stepped / recurrent implications of this model.  Fashion-MNIST is not a time-varying input set, so it is artificially coded into the time domain via spike latencies; N-MNIST has built-in movement of the MNIST digits; and the SHD codes represents its output in the time domain.  Presumably, a natively recurrent model like the one presented here would perform best compared to standard architectures that would not appropriately process the time-varying inputs, and allow training tasks which would be difficult with a standard static neuron model.  Depending on the time-binning, it could be cost-prohibitive to train a model in any way other than sparsely, and such an analysis could reinforce those advantages.

Clarity: The clarity in this manuscript is to be lauded.  The authors take care to draft meaningful, clear, and consistent figures.  Their derivations include color-coded term groupings to clearly discuss them in text that persist whenever term groups are discussed.  Sufficient details are reported for a reader to reproduce their work, and the real-world systems which they use are well described.

Significance: the primary motivation of this training method are to achieve improvements in efficiency and a reduction in computation, a topic of general interest.  Specifically, though, this method delivers substantial advantages for spiking applications, a more niche research area, with particularly strong applicability to the neuromorphic community.  In real-world speedups of training on spiking data, GPU training on the backward pass can be a substantial 70x faster, and be 40% more memory efficient.  These are valuable improvements, but these datasets are not yet representative of real-world usage or state-of-the-art computer vision, so the work is of more specific interest specifically to these niche communities.

Overall, this work contains multiple strong contributions, but the performance gap between state-of-the-art methods on the datasets and the performance measured here, the simpler neural network architectures, and the lack of a rigorous baseline to examine the impact of their simplification makes it challenging to assess whether future research can rely on the method introduced here to achieve sparse computation without sacrificing performance.  

1.  Lillicrap, Timothy P., et al. "Random synaptic feedback weights support error backpropagation for deep learning." Nature communications 7.1 (2016): 1-10.
2. Lee, Jun Haeng, Tobi Delbruck, and Michael Pfeiffer. "Training deep spiking neural networks using backpropagation." Frontiers in neuroscience 10 (2016): 508.
3. Wu, Yujie, et al. "Spatio-temporal backpropagation for training high-performance spiking neural networks." Frontiers in neuroscience 12 (2018): 331.


---- 
After rebuttal:  I thank the authors for the interesting additional experimentation, and have raised my score in accordance with the more complex architectures and continued sustained performance.
limitations_and_societal_impact: This reviewer identifies no specific negative societal impacts from this work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a novel sparse gradient update rule to train spiking neural networks faster and more energy-efficient on GPUs. A mathematical derivation of backpropagation through time (BPTT) is given for a network of leaky integrate-and-fire (LIF), feed-forward, fully-connected layers without skip connections. A threshold is defined that marks neurons as active when they are close to the membrane threshold and the spike gradient is only propagated back if a neuron was active at that time during the forward pass. As a result, only a fraction of the actual update steps have to be computed, because all others would be zero anyway. The authors implement a version of their algorithm in CUDA to compare their sparse gradient update rule to the dense update rule from surrogate gradients. It is found, that on the three evaluated datasets, Fashion-MNIST, N-MNIST and Spiking Heidelberg Digits, a consistent improvement in memory efficiency and training speed can be measured while achieving similar accuracies.
main_review: The results of this paper are an important contribution to the deep learning and neuromorphic community, as it tackles an important issue in the current state-of-the-art in the training of spiking neural networks. That is, the training speed is very slow, because potentially hundreds of time steps have to be simulated. As the common deep learning frameworks like PyTorch and TensorFlow offer no functionality to date, to compute sparse gradients efficiently, it is not possible to implement efficient update mechanisms for sparsely active networks like most spiking neural networks. The authors both, derived the theoretical foundation of sparse updates based on BPTT and the fact that the gradient is only non-zero close to the threshold and gave an actual implementation to validate their theory.

Still, the paper can be improved in a few aspects.
1.  Fig. 1 is not necessary in my opinion, as it does not provide any insights in the actual gradient updates, nor does it give an overview of the proposed algorithm. All relevant information can be found in Fig. 2. 
2. In Tab. 2, the authors report theoretical savings in energy. As there is an actual implementation for GPUs, I think measuring the actual power consumption would be very interesting, to see if there is an actual gain in energy-efficiency.
3. In Fig. 3B, there is no colorbar and therefore it is not really possible to compare the results in a fair way. It could for example be, that all results are very close to zero and therefore the difference would be less impressive between the algorithms. Additionally, plotting the difference of Original GradW and Sparse GradW would make it easier to compare the two. 
4. When reporting results in Fig. 3D, please also report some concrete results from the literature instead of just stating that "these accuracies are on par" (line 209). 

Generally, it would have been interesting to see more results, especially on deeper neural networks. I see no imminent reason why the algorithm should not scale to deeper networks, on the opposite it should be even more efficient than surrogate gradients, that have already proven to be working with at least 10+ layers.

Overall, this paper shows solid results of a novel sparse gradient update rule with a small spiking neural network. I think this paper should be accepted, if at least some of the concerns in this review are addressed.


Small things that did not influence my score:
- ln 22: I would remove the "simultaneously", as it sounds like humans can solve auditory and visual recognition problems while also playing games, all at the same time
- Eq. 14 could be typeset prettier (without underscores and suitable variable names)
- Fig. 3D could be zoomed in more (e.g., start the bars at 25 %)
- A relevant paper to cite in the final version (was not published at submission): Event-based backpropagation can compute exact gradients for spiking neural networks, T. Wunderlich, C. Pehle, Sci Rep 11, 12829 (2021). 
limitations_and_societal_impact: The limitations have not been discussed in detail. It is not clear why no deeper networks were used for the experiments. It is also not clear to me, why the speedup does not increase when increasing the layer size in Fig. 4.
Societal impact has not been discussed.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Surrogate gradient learning (SGL) is a recent approach to reconcile backpropagation with spiking neural networks. The approach works in discrete time. At each time step and for each neuron, a spike is generated if the potential is greater or equal than the threshold, and this is done by applying the Heaviside step function. This raises a problem when computing the gradients, as the derivative of Heaviside function is zero almost everywhere. The main idea of SGL is to replace this derivative by a "surrogate" one, typically the derivative of a sigmoid.

Here, the main idea of the authors is to compute the gradients only for the neurons whose potential is close to threshold, because other neurons have negligible gradients. Although very simple, this idea allows to reduce computation time by up to 70x, and memory by 40 on simple networks (MLP with 2 hidden layers), with no accuracy loss on simple datasets (Fashion-MNIST, Neuromophic-MNIST and Spiking Heidelberg Digits)
main_review: The idea is thus interesting, but it remains to be proven if it also works with deeper networks, convolutional layers, and with more complex datasets (eg ImageNet for static images or IBM DVS Gesture for neuromorphic datasets). For example I wonder whether the agreement between the approximated gradients and the true ones would still hold in deeper networks, or if errors would accumulate through the layers.

On the same ground, the accuracies that the authors obtained with the simple datasets are much below the SOTA (see for example Table 2 on Fang et al 2020 Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks). Of course this is partly due to the fact that the authors did not use convolutional layers. But it raises the question of whether their approach would still cause no accuracy loss on a network that really solves the tasks.

limitations_and_societal_impact: N/A
review_title: null

Meta-review: Dear authors, 

congratulations on your submission being accepted at Neurips.  The reviewers appreciated the sparse BP algorithm for speeding up training of spiking neural networks, and agree that it would be of interest to the Neurips audience. However, they also included feedback and constructive criticisms which we would ask you to include in your final version, in particular regarding limitations of the approach on large data sets due to memory limitations (linear scaling with number of steps). It would be ideal if you are able to include stronger empirical results on more complex data-sets in the final version. 

Best, your AC 
Decision: Accept (Poster)