Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper considers that spiking neural networks are not suitable for traditional adversarial robustness analysis methods, and proposes linear relaxations for the membrane potential and spikes of SNNs. These relaxations can be used to provide robust training for SNNs.
strengths_and_weaknesses: Pros:

This paper is well written.

This paper gives a linear relaxation scheme for SNNs.

This solution takes into account not only temporal updates but also spatial updates.



Cons:

I'm not sure if the linear relaxation is too loose. I think an example of MNIST can be used to illustrate the gap.

The upper and lower bounding strategies used for spike inputs in Sec 3.2 are probabilistic, not linear. Can the introduction of probability be consistent with linear relaxation? If not, i.e. x^u=x^i=x, how will the model work?

Previous work has shown the robustness of SNN is affected by coding (Poisson coding and directly coding) [1]. What's the way of coding in your paper? 

There are some typos that need to be fixed. And the citation is not standardized. For example: "leaky integrated-and-fire (LIF) Gerstner et al. (2014)”（Line 56）should be “leaky integrated-and-fire (LIF) (Gerstner et al. 2014)”.

[1] HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training With Crafted Input Noise

questions: Is this the first analysis of the robustness of spiking neural networks? [1] is also a paper that copes with the robustness of SNNs. 

[1] HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training With Crafted Input Noise


limitations: yes
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposes a methodology to tackle adversarial robustness in SNNs.
strengths_and_weaknesses: + The authors show that their method is able to resist attacks of different types on small scale datasets.
- The authors evaluation is pretty limited. In [1, 3], the authors show that BNTT trained SNN models are inherently more robust. Can the authors comment on how their methodology is different from [1,3]. Further, the authors started their paper discussion with SNNs being advatangeous on hardware, so it makes more sense to develop hardware aware robustness. But, the author's method is algorithm-based. Can the authors comment if their robustness will transfer to hadware as is or any modification will be required? In [4], the authors show that adversarial robustness on hardware become pretty low, so they come up with a normalization technique to resist attacks. In [2], the authors show that the type of coding technique plays a role in determining adversarial robustness. I am not sure if teh author's methodology can tarnsfer across differnet coding techniques.

[1] Revisiting batch normalization for training low-latency deep spiking neural networks from scratch Y Kim, P Panda Frontiers in neuroscience, 1638

[2] Rate Coding Or Direct Coding: Which One Is Better For Accurate, Robust, And Energy-Efficient Spiking Neural Networks? Y Kim, H Park, A Moitra, A Bhattacharjee, Y Venkatesha, P Panda ICASSP 2022-2022

[3] Visual explanations from spiking neural networks using interspike intervals Y Kim, P Panda Scientific Reports 11, Article number: 19037 (2021)

[4] Bhattacharjee, Abhiroop, et al. "Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars." arXiv preprint arXiv:2206.09599 (2022).
questions: Please see above comments.
limitations: Please see weakness section.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In this paper, a robust training method for SNNs is proposed. It is based on S-IBP and S-CROWN algorithms. The results on 3 different datasets show attack error reduction with some original accuracy loss.
strengths_and_weaknesses: Strengths:

1.	The contributions of this paper are clear and original.
2.	The achieved results are significant and relevant to advancing the state-of-the-art.

Weaknesses:

1.	There are several typos and semantically incorrect sentences throughout the text. It is recommended to conduct thorough proofreading.
2.	The clarity of some key sections can be improved. See comments below.

questions: 1.	In Section 1, typo, rewrite the sentence: “Except for the biological character of SNN models, the event-driven based information propagation mechanism makes SNN can be deployed with limited computation resources.
2.	Is the methodology for generating adversarial attacks in SNN described in Section 2.2 novel, or is it inspired by related work? If the latter, the references should be included in the paragraph.
3.	Please describe in more detail which are the assumptions made to derive the equations in Section 3.1.
4.	In Section 3.2, typo, correct the sentence: “Also, the l0-norm boundary can be interpreted as the probability of an element is unstable”
5.	Section 3.3 only describes the S-IBP and S-CROWN algorithms independently. More details are required to describe the overall structure of the proposed robust training method.
6.	What is the tool flow used for conducting the experiments? Which packages/libraries are used?
7.	In Section 4.2: “By considering the original accuracy and S-CROWN result after the robust training, we select the ϵ as shown in Table 2.” It is not clear which criteria are used to select ϵ.
8.	How does this work compare with the following article?
Kundu, Souvik et al. “HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noise.” 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021): 5189-5198.

limitations: The limitations and societal impact of this work have not been discussed. However, there is no reason to penalize the submission for this, since this work provides only positive impacts.
review_title: null

Meta-review: metareview: This paper applies existing certification-based adversarial robustness techniques to spiking neural networks. They achieve this through upper and lower relaxations of the spiking equations.

Review scores were high variance, ranging from 4 through 8. Reviews were generally of high quality. The largest concern was that the use of rate coding for the network's output limited the applicability of the technique. I found the authors' response to this concern satisfying.

I appreciate that this paper is the first to apply certification-based techniques to spiking neural networks. I believe it has the potential to produce significant impact for that reason.

Based upon the reviews, and my judgement of the potential impact, I recommend the paper be accepted.
confidence: Certain
Decision: Accept