Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the adversarial attack algorithms have been adapted to be applied to event-based data for spiking neural networks implemented on neuromorphic hardware.
main_review: 1.	It is not clear how the adversarial attack algorithms that are commonly used for discrete data are applied to event-based data in this work. Which are the design choices that are done to develop/modify these methods? Since these modifications constitute the main contributions of this work, they should be comprehensively discussed in the main manuscript, rather than in the supplementary material. The description in Sections 2.1 and 2.2 does not contain sufficient details.
2.	In this work, the novelty is low because the authors use already existing methods applied to an unexplored system.
3.	The experimental setup is not clear. Details like training hyperparameters, ANN-SNN conversion parameters, and HW-level constraints of the neuromorphic chip that has been considered in the implementation.
4.	It is ok to keep anonymous the name and brand of the in-house neuromorphic chip on which the experiments have been conducted, but the experiments can also be conducted on publicly available neuromorphic chips, to ease the comparison with related works.
5.	For all the attacks, in particular for the universal attack, the stealthiness, i.e, how much the adversarial examples are noticed by the human eye, should be discussed.
6.	The results can be extended with experiments in which adversarial defenses, such as adversarial training or noise filter, are applied to improve the robustness against the attacks.

summary_of_the_review: This paper tackles a very important problem with rigor and soundness. However, some key points need to be addressed (see the main review section).
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents proof of the viability of adversarial attacks on CNN processing event-based vision data. The paper investigates adaptations of well-known white box attacks to the event-based and spiking domain, validates the claims on three public benchmarks, and investigates the effect of the adversarial attacks directly on neuromorphic hardware. The main result is that, just like for conventional computer vision, adversarial attacks can have a big effect on the prediction of CNNs for event-based vision inputs. The attacks also translate to networks run on neuromorphic hardware.
main_review: Strengths:
+ overall this is a solid paper, without unjustified claims, and with a straightforward message: adversarial attacks on event-based vision works
+ the paper is well written, the figures are clearly showing the effects of the adversarial attacks
+ a variety of different attacks is tried out, including adversarial patches
+ the paper is to the best of my knowledge the first to try out the effect of adversarial attacks on neuromorphic hardware

Weaknesses:
- as the authors note, there is concurrent work by Marchisio et al. 2021, which investigates a very similar question, and even goes beyond that by investigating potential defenses against adversarial attacks. The present paper only shows the possibility of attacks, but does not talk about defenses. Therefore the novelty is severely limited.
- the description of the adversarial attacks and their adaptation to the event-based domain is very short and could only be understood with the help of the supplementary material. I would propose expanding this description to make the main paper more self-contained.
- the results of the paper are not really surprising, and I would have hoped for more specific comparisons how adversarial attacks differ between conventional and event-based vision, and also some outlook into defenses
summary_of_the_review: While the paper is formally correct and well written, there are other published papers that have shown the main claims of the paper, and have even gone some steps further by analyzing possible defenses against adversarial attacks. Therefore the novelty is too limited, and the key findings are not surprising. There are some new elements, e.g. the analysis of attacks on real neuromorphic hardware, but overall this is too little for acceptance at ICLR.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper investigated how to adapt existing attack strategies in order to work with the discrete nature of event-based data. Also, the authors proved that Sparsefool can efficiently and reliably generate sparse perturbations on discrete data. Besides,the authors validate the adversarial examples on neuromorphic chip and the perturbation is effective after modified the networks. In a more realistic setting, the authors use adversarial patches to trigger a targeted misclassification. MNIST and IBM Gestures datasets are used to test the attack algorithms as a sequence of binary frames.
main_review: Strengths:
This paper draws a conclusion through rigorous and sufficient experiments, which provide quantitative results to prove the effectiveness of the attack algorithms on dvs data. How to deal with the discrete nature of event-based data and their dependence on time is explained clearly. Also, the authors verified the effectiveness of adversarial examples on neuromorphic chip for the first time.
Weaknesses:
1)More details about the existing attack algorithms after adapted and how the algorithms work on the discrete-time raster could be explained.
2)Some explanations about the meaning of the variable symbols such as ς ,r_i ,α_i and so on in the appended algorithms could be added.
3)More attack algorithms such as PGD and probabilistic PGD could be used on neuromorphic chip and calculate success rate in order to compare with the algorithm Sparsefool.

summary_of_the_review: This paper verified the attack algorithms can be adapted to the dvs data. The Sparsefool algorithm achieves high success rates through adding and removing a few events. The authors proposed a next step, which building real-world patches that can fool networks.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Authors present white-box adversarial attack algorithms for spiking neural networks (SNNs) with event-based visual data inputs, and experimentally demonstrate vulnerability of SNNs in various settings. Extensions of existing adversarial attacks on convolutional networks (i.e., PGD and SparseFool) are presented and utilized in a setting for SNNs with event-based inputs. Experimental evaluations are performed on the Neuromorphic-MNIST and IBM Gestures dynamic vision sensor datasets, and validations of these attacks on neuromorphic hardware are performed.

main_review: The paper is nicely written, and presented experiments and visualizations are well organized. Empirical validations of the proposed methods on neuromorphic hardware add a significant value to the paper with respect to the state-of-the-art.

On the other hand, methodological contribution of the paper is rather shallow, and Section 2.1 is not describing the used methods in detail. It should be clearer in mathematically describing the attack optimization objective within the SNN setting, e.g., on page 4 while describing SparseFool, some variables are mentioned out of context. Most of the important content on what is actually done for the attacks is described shortly in text and in the Supplementary algorithms. Overall, the authors implemented SparseFool attacks of [Modas et al, 2018] for SNNs, and demonstrated that SNNs with event-based inputs can be adversarially attacked. However authors also do not evaluate their attacks in comparison to the state-of-the-art PGD-like attacks with spike-compatible gradient proposed for SNNs by [Liang et al, 2020]. In fact this study is also not correctly referenced in the paper, mentioned as a work that utilizes: “… brute-force approaches that rely on heuristics to reduce the search space [Liang et al, 2020, …]”, which is not the case.

I think evaluating a binary image recognition CNN on the BMNIST dataset with SCAR attacks is not a good choice of a baseline for this work, and seems distant from the main purpose of the paper. It would be more suitable to compare the same SNNs and proposed attacks with previously presented PGD-variant attacks for event-based data [Liang et al., 2020].

One of the earliest studies in this domain by Sharmin et al, "Inherent adversarial robustness of deep spiking neural networks: Effects of discrete input encoding and non-linear activations.”, ECCV 2020, is also missing and should be discussed in the paper. In some ways, the PGD attacks presented by [Sharmin et al., 2020] for SNNs resembles to the standard PGD approach demonstrated in the paper (Section 1.2 of the Supplementary Materials). This could be highlighted and discussed in detail with the revisions. For instance, N-MNIST experiments are performed with models trained using ANN to SNN weight transfer, which [Sharmin et al., 2020] similarly demonstrated with their PGD-like attacks.

Results presented in Table 2 (and the supplementary video) are very interesting. It seems like a random patch perturbation can be quite effective for a RH Wave label, but not at all for RH Clockwise or RH Counter Clockwise labels. Could this be a result of the relevant temporal information constructing a significant aspect of the attacking patches? At this point, what I think is another missing “random patch baseline” to compare against, is one that can exploit temporal information. For example, if one would invert the signs of the events of learned adversarial patches before addition (i.e., using un-meaningful inverses of the learned adversarial patches), would it already be a better attack than a naive random patch (which only exploits locations but not temporal information)?

Looking at Fig 2 IBM Gestures examples, it looks like perturbation events that are much outside the region of movement are even sufficient to lead to misclassification (i.e., models are very vulnerable to arbitrary events occurring outside the relevant region). To compare this vulnerability against some potential robustness gains, I would be curious if the authors implemented any simple robust training approach that resembles to adversarial training? i.e., would the models be still very robust to additive events occurring at irrelevant locations? These observations can be discussed in the revisions.

Minor comments:
- Authors report that the SNN used for IBM Gestures has a test accuracy of 84.2%. For completeness, can the authors also report what is the on-chip test accuracy of the model following the modifications (e.g., removing batch-norm) and weight precision quantizations to make the model compatible with the hardware?
- Please write in the caption of Fig 2 that those examples of adversarial inputs are crafted via SparseFool attacks (the reader discovers it much later in the text).
- Caption of Algorithm 2 in the Supplementary Materials includes some interesting information that could be extended. Authors state that 50 iterations of PGD was sufficient and results did not improve afterwards. It would be interesting to plot this observation in the S.M. as it seems to demonstrate some inherent robustness.
summary_of_the_review: The paper is nicely written and organized from an empirical contribution perspective. However the paper is rather weak in depicting its methodological contributions in my opinion, and lacks comparisons/references to relevant SOTA works. I listed my major concerns in the main review, and would be willing to re-address my rating based on the authors’ responses and revisions.
review_title: null

Meta-review: The manuscript investigates common adversarial attacks on event-based data for spiking neural networks.
They conclude that also in this setup adversarial attacks can strongly harm SNN performance.

Although the reviewers agree that the paper presents some solid results and is well written, there was also substantial criticism.

The main points were:
- It is not very clear how the usual attacks are applied to event-based data, and in general experimental setups are unclear.
- The methodological contribution of the paper seems limited.
- The novelty is limited, in particular Marchisio et al. 2021 investigates a very similar question and goes somewhat further. The author noted that their attacks are not deployed on neuromorphic hardware. A number of other important prior work is not discussed.
- The impact of adversarial defences was not considered.
- A more detailed comparison of event-based attacks to standard ANN attacks would be desired.

After the reviews, the authors have invested substantial efforts to improve the paper. These efforts were appreciated by the reviewers. In particular, the authors ran additional experiments using the defence method TRADES. The results showed that TRADES is effective, but the attack has still a large success.

In summary, the reviewers agree that this is a solid manuscript and an interesting direction, however, they see it finally slightly below acceptance threshold for ICLR.
Decision: Reject