Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper focuses on the temporal, event-driven manners of training a spiking neural network from scratch. The authors first revisit the learning dynamics of event-driven learning and discover several invariance properties. Then, a problem called reverse gradient is raised and addressed. Extensive experiments are conducted to verify the effectiveness of this method. 
strengths_and_weaknesses: 1. This paper is well-presented. The writing and visualization are neat and easy to follow.

2. The experimental results are sufficient for comparison with existing event-driven learning works.
questions: TBH, I am not an expert in event-based training of SNN, therefore I cannot give useful feedback with respect to that. I have a few questions about the difference between event-based training and the "RNN-like" training. 

1. When implementing these event-based learning algorithms on the neuromorphic hardware, how to accelerate the training and why is it significantly faster than the RNN-based learning algorithms?

2. Can this method be trained with residual connection?

limitations: Overall, I found this paper interesting and could be valuable for publication at the NeurIPS conference. As I am familiar with this type of training method for SNNs, I could not give conceptual limitations for this paper. I am giving borderline acceptance of this paper due to its good presentation. Meanwhile, I will set my confidence score to 2 and will look into the comments from other reviewers to finalize my rating. 


-----

POST-REBUTTAL REVIEW:

I'd like to thank the authors for their detailed response. My questions are addressed, thus I increase my rating to 7.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors propose a temporal Backprop approach for training SNNs with some interesting backward kernel fucntion.
strengths_and_weaknesses: + The authors showcase that their temporal BP methodology yields high accuracy as compared to similar related works.
-The paper presents a direct training method using BP for SNNs. This work is very derivative and incremental. There is a lot of work from Priya Panda's group at Yale, Emre Neftci's group, and many others with regard to SNN training. The authors have failed to acknowledge most recent works and the method they are proposing is very incremental in the context of those works. Further, many recent works on SNNs have targeted larger datatsets including video segmenattion with direct training. I wonder if the author's method can even scale up, since their results are limited to CIFAR10, CIFAR100.

-The authors did not comment on how many time steps their method requires to train. In the recent work [5], the authors show that they can use backward connections to train SNNs better, is there a similarity between the backward kernel and backward conenction?


Below is a list of publications (not exhaustive) that the author should check:
[1] 
[2] Enabling spike-based backpropagation for training deep neural network architectures C Lee, SS Sarwar, P Panda, G Srinivasan, K Roy Frontiers in neuroscience, 119
[3] Rate Coding Or Direct Coding: Which One Is Better For Accurate, Robust, And Energy-Efficient Spiking Neural Networks? Y Kim, H Park, A Moitra, A Bhattacharjee, Y Venkatesha, P Panda ICASSP 2022-2022
[4] Neuromorphic Data Augmentation for Training Spiking Neural Networks Y Li, Y Kim, H Park, T Geller, P Panda arXiv preprint arXiv:2203.06145
[5] Neural architecture search for spiking neural networks Y Kim, Y Li, H Park, Y Venkatesha, P Panda arXiv preprint arXiv:2201.10355
[6] Optimizing deeper spiking neural networks for dynamic vision sensing Y Kim, P Panda Neural Networks 144, 686-698
[7] Federated Learning with Spiking Neural Networks Y Venkatesha, Y Kim, L Tassiulas, P Pand IEEE Transactions on Signal Processing 2021
[8] Beyond classification: directly training spiking neural networks for semantic segmentation Y Kim, J Chough, P Panda arXiv preprint arXiv:2110.07742
[9] Visual explanations from spiking neural networks using interspike intervals Y Kim, P Panda Scientific Reports 11, Article number: 19037 (2021)
[10] Revisiting batch normalization for training low-latency deep spiking neural networks from scratch Y Kim, P Panda Frontiers in neuroscience, 1638
questions: If the authors can highlight their technical novelty a compared to previous works, it will help me re-assess the paper's contributions. See above comments for reference.
limitations: See weakness section.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a modified event-driven backpropagation and investigate its performance on benchmarks. The authors also investigate if the backpropagation followed a gradient assignment rule, finding that max-pooling obeyed this rule. This is one of the most important conclusions of the paper. The algorithm achieved SOTA on CIFAR-10, and was the first to be trained on CIFAR-100.
strengths_and_weaknesses: Spiking neural networks represent a new paradigm of neural networks that, among other advantages, incorporates time into the building blocks of its own functioning. Thus, in addition to having greater biological plausibility, it is also believed to be more coherent with learning in the real-world, which contains the time dimension.
Event-base learning paradigm preserves de biological plausibility and the event-based features advantages of spiking neural networks when compared with surogate backpropagation.
However, this new paradigm has not yet reached SOTA in any benchmark, not even in those that demand incorporation of the time dimension.
questions: In line 233 you stated that more details about the configuration of training were available on appendix, I was not able to fin the appendix section, nor a appendix file on the supplement material, even though the code contains the configurations of the experiment. It is not clear for me if you want to provide more information in an appendix section or you were referring to the code provided.
limitations: Spiking Neural Networks has yet achieved SOTA in no benchmark.
Although spiking neural networks are believed to have greater biological plausibility, it is not clear whether biological neural networks learn through backpropagation, which was the method tested in this study. Despite this, nowadays there is no alternative that works better than backpropagation.
The authors also stated that another gap in biological plausibility is the reverse time processing feature.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The focus of this paper is training of spiking neural networks. Specifically, the paper proposes back-propagation with respect to spike timing. They analyze previous methods and propose a small increment over the current state-of-the-art. The results are not clearly discussed. 
strengths_and_weaknesses: Strengths:
- The work focuses on an aspect of the learning algorithms that requires optimization and innovation. 

Weakness
1.	It is hard to understand what the axes are for Figure 1. 
2.	It is unclear what the major contributions of the paper are. Analyzing previous work does not constitute as a contribution. 
3.	It is unclear how the proposed method enables better results. For instance, Table 1 reports similar accuracies for this work compared to the previous ones.
4.	The authors talk about advantages over the previous work in terms of efficiency however the paper does not report any metric that shows it is more efficient to train with this proposed method. 
5.	Does the proposed method converge faster compared to previous algorithms?
6.	How does the proposed methods compare against surrogate gradient techniques?
7.	The paper does not discuss how the datasets are converted to spike domain. 

questions: Please refer to Strengths and Weakness for the points. 
limitations: There are no potential negative societal impacts. One major limitation of this work is applicability to neuromorphic hardware and how will the work shown on GPU translate to neuromorphic cores.
review_title: null

Meta-review: metareview: The authors propose a novel training algorithm to train spiking neural networks (SNNs) in an event-driven manner with backpropagation. They perform experiments on standard benchmarks such as CIFAR-10 and CIFAR-100 to verify the effectiveness of the method. The algorithm achieves SOTA performance on these data sets. Event-driven methods are interesting from a hardware perspective as gradient have to propagated only at spike times.

The manuscript received mixed ratings, a clear agreement could not be found. 

Pros: 
- The authors provide an analysis of event-driven backprop in SNNs which helps to adjust the usual learning procedure.
- The authors performed experiments on several data sets and achieved SOA performance w.r.t. other event-driven methods.
- They also tackled CIFAR100 for which no results were previously shown with event-driven algorithms
- The paper is well-written, although language could be improved at places.

Cons:
- Improvements over competing event-driven techniques are rather small 
- Performance is still clearly below non-event based surrogate gradient methods (but this is not surprising)
- Not clear how the method scales up beyond CIFAR100

Since the ratings were mixed, I read the paper and believe it is publishable in NeurIPS although it is somewhat borderline.
confidence: Less certain
Decision: Accept