Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: null
review_content: I have updated my score to borderline accept (conditional), primarily due to the authors' apparently thorough rebuttal. However, I believe this paper requires further input from authors and necessary discussion among reviewers for consensus. 


review_title: Post rebuttal acknowledgement 

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes robust regularization during SNN training to motivate and inspire improved robustness for the trained models.
strengths_and_weaknesses: ## Weakness

### Technical

1. The authors missed a very important point, as earlier literature has already showed the effectiveness of robustness when the input is rate coded [1] and significantly less robustness when input is direct coded [2]. This key aspect is completely overlooked in the manuscript.

2. The authors made a strong claim in L208, without any empirical validation of the fact whether it happens or not. Particularly, when the model is further fine tuned in SNN domain. Hence, the motivation of the work is weak.

3. The paper does not clearly mention what might be their inspiration to use BN in their SNN models as the model definitions are never explicitly mentioned.

4. During training the authors used FGSM attack variants, and during testing they got robustness against PGD variants. This raises significant question about the experimental set up.

5. The authors should proof the robustness is real as this might easily due to gradient obfuscation, and to me it already fails the gradient obfuscation.

6. Authors mentioned they mixed and matched different FGSM attacked images for training to make the attack during training more diverse, no proof (theoretical or empirical) is provided.

7. Analysis of additional training cost is missing. [2] provided improvement in robustness without any additional training cost.

Overall the paper lacks motivation and enough contribution. Comparison with SOTA is also missing. I would encourage the authors to work on the said issues to make the paper better.

[1] Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activations, ECCV 2020.

[2] HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training With Crafted Input Noise, ICCV 2021.
questions: Please see weakness.
limitations: N/A.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes Regularized Adversarial Training (RAT), an adversarial training framework to improve the robustness of the Spike Neural Network (SNN). This paper adopts three different gradient approximations (i.e., CBA, BPTR, BPTT) to mitigate the non-differentiable of SNN. At first, to achieve stronger adversaries, this paper augments FGSM, RFGSM, and PGD with the above approximations. Then this paper proposes spiking the Lipschitz constant, a variant of normal Lipschitz constraint that regularizes the model to be Lipschitz smooth. The proposed RAT is evaluated over several popular architectures.
strengths_and_weaknesses: **Strengths:**

1. This paper is well-written and easy to follow.
2. The proposed SAT seems very effective over several strong adversaries.

**Weaknesses:**

1. This paper aims to improve the robustness of SNN but didn’t summarize previous efforts on this topic. Besides, this paper didn’t provide comparisons with other related state-of-the-art algorithms, which makes it hard to recognize the contribution and value to this field.
2. Training network with Lipschitz constraints have been welly exploited in ANN. Some key references are missing in this paper. The authors should illustrate the differences between their work and those Lipschitz constraints proposed on ANN.
3. The setting of the ablation study is unclear. In Table 3, the definitions of MIX and REG are not provided. Besides, what is the baseline of this algorithm? Since this work is named Regularized Adversarial Training, I expect the baseline would be normal adversarial training without Lipschitz constraint. However, I couldn’t find it.
questions: 1. Some key references are missing in this paper, including the robustness of SNN and the Lipschitz constraint in ANN. The authors should make a comprehensive summary of these fields and highlight their contributions.
2. The authors should provide comparisons with other state-of-the-art defense methods on SNN and also the baseline of this work.
limitations: The authors have addressed the limitations and potential negative societal impact of this work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper provides a theoretical analysis of the SNN robustness against adversarial perturbation. In this regard, a regularized adversarial training method for SNN has been proposed. The regularization is based on the spiking Lipschitz constant. The results conducted on various benchmarks demonstrate that the proposed training scheme achieves better robustness compared to the vanilla adversarial training.
strengths_and_weaknesses: Strengths:
1.	The proposed idea is original and relevant to the NeurIPS community.
2.	The results show an advancement in the state-of-the-art.

Weaknesses:
1.	Some key concepts need better clarification. See the specific questions below.
2.	The experiments should be extended using event-based datasets.
questions: 1.	In Section 1: “it is necessary to improve the adversariality of SNNs”. Do you mean to improve the adversarial robustness of SNNs?
2.	In Section 1: “The update amount of weights is the key to constructing adversarial attacks.” Please rewrite this sentence. The adversarial attacks should update the input intensities, rather than the weights.
3.	What is the noise budget used in the attacks for the results reported in Table 1?
4.	The proposed regularization method described in Section 4 is strongly based on previous works. It is recommended to explain more clearly the novel features.
5.	In Section 5, the results of the proposed method have been compared only to the vanilla adversarial training. If possible, the comparison with other methods among the related works should be included.
6.	The experiments are conducted only on static data, while SNNs are commonly used also on event-based data. Therefore, it is recommended to extend the experiment set including results on event-based datasets.
limitations: The limitations and societal impact have been discussed by the authors in the supplementary material.
review_title: null

Meta-review: metareview: This paper proposes an adversarial training method for Spike neural networks. One challenge is that spike networks are non-differentiable and the paper develops various gradient approximation methods and builds on previous attack methods like FGSM and PGD with approximate gradients. An additional innovation is the development of a regularization method that estimates Lipschitz constants. Estimating Lipschitz constants of spike neural networks is another technical challenge and the paper develops a rigorous bound using a concept they call spike distance. which is an upper bound to the normal Lipschitz contanst. 

Several concerns were raised by the reviewers including incomplete discussion of prior work, clarifications on the performed ablation study and comparison to prior SOTA. Overall the authors did a good job in their rebuttal and discussion to convince the revewers and this meta-reviewer that the paper merits publication. This is a somewhat niche problem setting but the paper has several theoretical and practical innovations that are interesting and suitable for publication. 

confidence: Certain
Decision: Accept