Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work analyzes the adversarial attacks on BCI systems. It induces DoS attack and incorporates domain-specific insights to achieve the adversarial attack. The designed attack is evaluated on a public dataset EEGMMIDB and shows that a popular deep learning model, EEGNet, is vulnerable to adversarial attacks.
main_review: Strengths:
1. the motivation and contributions are well written. There are indeed few studies on the adversarial attacks on BCI.

Weaknesses:
1. This work is NOT actually implementing attack (as implied in the Introduction) with a physical device, but simulating attack! It's a huge difference. Simulation is much more easier and costs much less effort. See more details in the following review summary.
summary_of_the_review: The whole paper (Introduction, Fig 1, etc.) implies that this study has a smart device that can be attached to the left ear that will inject well-designed perturbation to collected EEG data. I was excited when reading the paper and I know how hard it is. It requires a multidisciplinary team to design the wireless smart device, manufacture the device and make sure it really works, break the EEG collection equipment (like Emotive headset) which is difficult as the signal transmission is generally encrypted, and so on. I was intended to give an accept (or even a strong accept) as long as they can implement the whole system in the real world. 

But it's not the case! The dataset is not locally acquired but is the most well-known public EEGMMIDB dataset, which takes little effort to get. The attack is not really deployed with a smart device at the left ear which can emit perturbations to the source signal, but is simulated. There is no device, no signal transmission, no poisoning attack on source signals. 

Without the effort on physical systems and real-world attacks, this paper is still good but not good enough for ICLR. It's a good story, but still a story. So I decided to give a rejection.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Authors propose a method to perform adversarial attacks on BCI deployed on wearable devices. Using an electrode enclosed in a head mounted device, they show how to propagate a perturbation through the subject’s scalp capable of significantly affecting a EGG based classification model. Authors test the approach on the movements detection domain using a public dataset, showing how the proposed attack can lead to a significant decrease of  the system’s performance.
main_review: I believe this paper addresses a quite relevant topic, of interest for the community: the impact of adversarial perturbations in sensitive wearable systems. I think both motivation and contribution of the work are correct, with a proper statement of the problem and a comprehensive SoA analysis. The paper is well written, with no detecting spelling issues, and overall easy to read.  The overall contribution is adequate, being my only concern related to the completeness of the work and the overall reproducibility of results.

I believe the general structure of the manuscript  is correct. It includes a proper explanation of BCI, SoA, background of the attack, and a dataset that, although limited, given the scarce public dataset for this domain I consider adequate. The part which I consider could be improved, and hence my recommendation, is the description of the method and experimental setup. 

Since this approach relies on affecting a ML system to misclasify its input, I’d expect to see a more detailed version of adversarial architecture and it’s training process. Training params are indeed included but after reading the paper several times I’m not sure about the adversarial approach to get access to the gradient of the attacked system. 

Honestly I’d rather see less details about the propagation and plausibility of the attack, and even remove some figures (as the Figure1 which do not offer substantial information) and include the end-to-end training algorithm  used to generate the adversarial model (or substitute) in a figure. As I mentioned before I’ve serious doubts that this system could be reproduced  with the details provided in the paper. By the way, I did not see any supplementary material which I understood the authors attached. I’d suggest adding the end-to-end architecture plus training algorithm, with some details and metrics about the training and inference process. 

The paper may raise some ethical concerns since the topic is quite sensitive, but I think these matters have been explained adequately by the authors in the manuscript.

summary_of_the_review: I consider this paper as an interesting work with a proper motivation and novelty component. The topic addressed (adversarial perturbations on wearable BCI ) can be definitively of interest for the conference audience. The main improvement point in my opinion is the description of method and experimental setup. Further details on the adversarial approach I think would improve the paper. I consider the overall contribution of the work as adequate, and hence my recommendation
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper examines vulnerability of the brain computer interfaces to adversarial attacks. A new method is proposed to generate more realistic, smoother representations of the adversarial attack. A case where an attack happens at a specific location (close to an ear) is studied as a representative case of the perturbation added to the signal acquisition source. 
main_review: This is an interesting study which addresses AI-based medical applications targeted for edge devices. Related work is well presented and contributions are explained, however the novelty of the work is not clear, as some methods for generating smooth adversarial examples for different electrical body signals already exist. I believe similar concepts as used for example in ECG might be applicable to EEG as well. It would be beneficial to a reader to compare those. 

The flow of the work should be also improved for better readability. The related work section could be included in the introduction, as the introduction already contains some comparison with previous studies. Figures placement, ordering, descriptions and references in the next definitely need to be improved.  Below Eq. 9 the loss term is defined in words, while it would be better to use the equation. I believe it will be the same as the loss in Eq. 15(?), so maybe consider restructuring the work to improve its flow. Please also explain how attenuation configurations were selected, specifically explain the reason for choosing specific magnitude and delay values. 

The presented experiments were performed using one EEG dataset for a specific task of the motor imagery. It would be interesting to verify if the method can scale to other tasks as well and generalize to other datasets. Please elaborate on that. 

Also, some other more recent studies on EEG classification using e.g., transformers already exist. What is the reason for examining CNN models? Would transformers be more robust to adversarial attacks in this application, given the fact that they were proved more robust in other studies?


summary_of_the_review:  This is an interesting study, but in my opinion requires some additional work to make the contribution more solid, specifically comparison with other smooth adversarial attacks on electrical signals should be provided, and selection of CNNs vs transformers should be explained. It would be also beneficial to verify the proposed methods with other datasets/other EEG-based tasks if possible.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: Authors study the vulnerability of brain-computer interface (BCI) systems from a security perspective, and present a physiologically plausible adversarial attack approach on BCI systems that can induce denial-of-service after being transmitted by external devices. Empirical assessments are performed on an offline three-class motor imagery (MI) BCI dataset (right hand, left hand, and rest classes), where denial-of-service indicates a "rest" classification regardless of the subjects' motor imagery intent. Proposed methods are motivated to be physiologically plausible and effective to an extent that can cause failure modes for DNN-calibrated BCI systems.
main_review: MI-BCI systems are ultimately motivated to provide alternative communication and control means for people with severe neuromuscular disabilities. In this context, it is not really easy for me to understand the motivation of investigating adversarial attacks to induce DoS for a MI-BCI system in experiments, or at least it is not clearly put in the paper. I believe this motivation can only be applicable to a reasonable extent with certain scenarios like EEG-based biometric identification. Going further, how to possibly overcome this vulnerability that is proposed in the paper is also not discussed or studied, which would be in my opinion a more thorough study if preliminary analyses with standard adversarial training like approaches are at least briefly assessed?

Authors motivate their approach in comparison to existing attacks based on arguments that I am not fully convinced about. For instance, authors argue that the perturbations should be imperceptible and smooth. However, the follow-up argument on realizing artifacts at the source-level relies on e.g., tDCS delivered directly to the scalp (which is generally not imperceptible within the first few milliseconds, when EEG time-series is simply monitored). In fact, every noise-like artifact (sometimes even additive square-waves) can be rather considered imperceptible with EEG, as opposed to the large artifacts caused by tDCS or TMS at the instant. What would the authors comment on that?

Any BCI system that utilizes a discriminative DNN (e.g., EEGNet) is likely to be vulnerable to gradient-based adversarial perturbations as the authors well demonstrate. However most BCI systems (particularly MI-BCIs) that use conventional signal processing methods (i.e., filter-bank common spatial patterns or feature extraction methods based on Riemannian geometry) in fact still yields state-of-the-art (or comparable) performances with respect to deep feature learning models. It would be interesting for the current study to discuss or empirically investigate if such a vulnerability would be present (with similar impact?) in a BCI system running on such conventional feature decoding methods?

Some methodological comparisons are necessary, and missing in the paper. A baseline could simply be to add white noise or square waves to the signals (i.e., random pseudo-perturbations that have a maximum amplitude bound), and obtain a baseline attack success rate. Similarly, comparisons to state-of-the-art ideas on adversarial attacks to BCIs are not included (e.g., [Zhang & Wu, 2019] or [Liu et al, 2021] where the signal attacks/jamming occur after the EEG acquisition phase). For instance, how do the authors' UAP attacks relate to the [Liu et al, 2021] study, when one does not consider the spatial propagation constraints?

Online decoding for MI-BCIs is a very challenging task by itself, especially if one considers inter-subject decoding (as in the paper). Can the authors also present some subject-specific pre-attack and post-attack accuracies to give a better overview of how much the BCI system truthfully suffers from this vulnerability, since pre-attack MI decoding performances can be already lower in the current setting?

Regarding the visualizations in Figure 5, additional power spectral density plots would be helpful to even better demonstrate the similarities between original and attacked EEG signals, as they are generally monitored to check EEG for irregularities.

It would be also interesting to see if the results would look exactly similar, when one imposes the same adversarial attacks with similar parameters, starting from the right ear location electrode T10 which spatially propagates towards T9 the other way around (i.e., impacts C3 less than C4). Accordingly with this question, I would be also curious how are the attack success confusion matrices look like?, i.e., are the attacks having more impact on right hand motor imagery detection due to targeting T9?

Some minor comments:
- Typo in Eq 14.
- End of page 6: "log-loss" -> "negative log-likelihood loss"
- I would label the y-axis of Figure 3 as "attack success accuracy" rather than "accuracy".
summary_of_the_review: The paper is well written with a clear narrative, and used methods are well described. Main limitations of this work is on the presentation of the general motivating perspective, as well as a general lack of methodological comparisons/evaluations and discussions on possible countermeasures that one can consider for DoS attacks on BCI systems. Some experiments/comparisons are highlighted and suggested in my main review for the authors.
review_title: null

Meta-review: This work has triggered engaged discussions between authors and reviewers and also among reviewers.
These conversations have highlighted the potential weaknesses of the contribution, namely that the work
is a proof-of-concept experimentally (although arguably for ethical reasons) and that the
overall theoretical contribution is not strong.

Despite the merit of this work, and given the strong expectations of ICLR, I don't feel that this work
can be endorsed for publication at ICLR 2022.
Decision: Reject