Practical Adversarial Attacks on Brain--Computer Interfaces
Authors: Rodolfo Octavio Siller Quintanilla
Xiaying Wang
Michael Hersche
Luca Benini
Gagandeep Singh
Venue: ICLR 2022 Submitted (2022)
Keywords: neuroscience
brain-computer interfaces
practical attacks
adversarial attacks
EEGNet
edge computing
embedded systems

Abstract: Deep learning has been widely employed in brain--computer interfaces (BCIs) to decode a subject's intentions based on recorded brain activities enabling direct interaction with computers and machines. BCI systems play a crucial role in motor rehabilitation and have recently experienced a significant market boost as consumer-grade products. Recent studies have shown that deep learning-based BCIs are vulnerable to adversarial attacks. Failures in such systems might cause medical misdiagnoses, physical harm, and financial damages, hence it is of utmost importance to analyze and understand in-depth, potential malicious attacks to develop countermeasures. In this work, we present the first study that analyzes and models adversarial attacks based on physical domain constraints in EEG-based BCIs. Specifically, we assess the robustness of EEGNet which is the current state-of-the-art network for embedded BCIs. We propose new methods to induce denial-of-service attacks and incorporate domain-specific insights and constraints to accomplish two key goals: (i) create smooth adversarial attacks that are physiologically plausible; (ii) consider the realistic case where the attack happens at the origin of the signal acquisition and it propagates on the human head. Our results show that EEGNet is significantly vulnerable to adversarial attacks with an attack success rate of more than 50\%. With our work, we want to raise awareness and incentivize future developments of proper countermeasures.

Under review as a conference paper at ICLR 2022
PRACTICAL ADVERSARIAL ATTACKS ON BRAIN- COMPUTER INTERFACES
Anonymous authors Paper under double-blind review
ABSTRACT
Deep learning has been widely employed in brain-computer interfaces (BCIs) decode a subject's intentions based on recorded brain activities enabling direct interaction with computers and machines. BCI systems play a crucial role in motor rehabilitation and have recently experienced a significant market boost as consumer-grade products. Recent studies have shown that deep learning-bast -based BCIs are vulnerable to adversarial attacks. Failures in such systems might cause medical misdiagnoses, physical harm, and financial damages, hence it is of utmost importance to analyze and understand in-depth, potential malicious attacks to develop countermeasures. In this work, we present the first study that analyzes and models adversarial attacks based physical domain constraints in EEG- based BCIs. Specifically, we assess the robustness of EEGNet which is the current state-of-the-art network for embedded BCIs. We propose new methods to induce denial-of-servie attacks and incorporate domain-specific insights and constraints accomplish two key goals: (i) create smooth adversarial attacks that are physiologically plausible; (ii) consider the realistic case where the attack happens at the origin of the signal acquisition and propagates on the human head. Our results show that EEGNet is significantly vulnerable to adversarial attacks with an attack success rate of more than 50%. With our work, we want raise awareness and incentivize future developments of proper countermeasures.
INTRODUCTION
Recent work has shown that adversarial perturbations can cause soute-of-thh-aee (SoA) deep learning
models to misbehave in various domains including vision (Szegedy et al. 2014 Goodfellow et
2015), NLP (Li et al.] [2019]] Zhang et al. (2020). speech (Qin et al.) 2019: Li et al. 2019b), and
biomedicine (Finlayson et al. 2019:] Han al. 2020). Neural networks have been applied in brain-computer interfaces (BCis) achieving impressive results (Lawhern et al. 2018 Dose et al. 2018). A BCI enables direct interactions with external devices based on brain activities, typically recorded using clectroencephallgrrapiic (EEG) systems. It can provide communication pathway for severely paralyzed patients or assist rehabilitation (Chaudhary et al.. 2016). Besides medical applications, recent developments in wearable devices have pushed BCIs towards consumer-grade products to improve life quality (Aricò et al. 2020). e.g.. the Interaxon Muse headband for stress relief (Arsalan al. 2019) the Emotiv headset for controlling drones (Marin et al.. (0200) and ground vehicles (Zhuang et al. 2021). Safety in BCI systems is paramount (Dutta) [020 Bernal et al. 202T, because a Tailure would cause misdiagnoses, user frustration, even danger while driving wheelchair or controlling drone, causing physical and financial damages.
Zhang Wu (2019) were the first show that EEG- based BCIs are vulnerable to adversarial attacks by proposing an unsupervised fast gradient sign method (FGSM) (Goodfellow et al. 2015). More recent work has proposed more practical attack where universal adversarial perturbation (UAP) is computed once and can be applied to all EEG trials without learning it for every new input (Liu ct al. 202I). Both works assume that the acquired signals are sent to a remote compute engine, e.g., computer, and the attacker can alter the signals during the transmission by attaching a "jamming" module between the signal preprocessing step and the classifier. Recent developments in smart edge computing (Akmandor & Jha 2018) Beach et al.) (202)) climinate the need for data transmission. making this attack scenario inapplicable Novel BCI solutions (Kartsch et al. 2019: Wang et al.) 2020) embed the signal processing and classification directly the sensor edge. A more practical

---

Under review as a conference paper at ICLR 2022
adversarial example has been identified by Meng al. 22021) It consists of a square-shaped signal that can be added to EEG trials before the preprocessing step. However, the attack is proposed as backdoor key, which means that the attacker has direct access to the training dataset and pollutes with adversarial examples, which is improbable if the attacker is not directly involved in the data acquisition or in the training of the classifier. [i et al. (20T9b) have shown attack scenario in the audio domain by considering the on- boordd edge processing of a wake-word detection system, where an adversarial audio trace delivered to the environment causing denial-of-service (DoS). No similar studies can be currently found in the BCI domain.
Designing natural attacks and modeling its propagation. Unlike in audio applications where the signal can simply propagate over-the-air and is sensed by a microphone, extra modeling is required to evaluate the signal propagation in BCIs based on the physical properties of the biological tissues. In this work, rather than assuming "jamming" module between the preprocessing and the classification steps as in related works, we consider more realistic and practically applicable attack scenario where the adversarial perturbations are introduced at the source of the data acquisition, as showcased in Figure 6 in Appendix A This can be achieved, for example, via electromagnetic waves delivered to the environment (Dutta 2020) via transcranial current stimulation with electrical current delivered directly to the scalp (Bodranghien et al. [017 Fertonani et al. (2015). by exploiting wearable devices, such as smart glasses over-ear headsets (Flowneuroscience [202]] Marin et al.
Challenges:
2020). The adversarial perturbations translate into electrical signals propagating over the scalp and are sensed by the electrodes in addition to the EEG signals.
To guarantee the imperceptibility of the attacks, previous works in BCIs create perturbations that are small in amplitude (Zhang& & Wu 2019 Jiang et al., 2019 Liu et al.. (02I). limiting the attack success rate (ASR). Increased perturbation's s amplitude yields higher ASR (Meng et al. [02TT) but makes the attack more easily detectable. Moreover, the generated perturbations are square-shaped which is implausible for biosignals. Han et al. (2020) are the first to observe square-wave artifacts in biosignals' attacks and propose smooth perturbations for electrocardiograms (ECGs). No similar works have been found for EEGs.
This work: Practical attacks on BCI models. To address the above technical challenges, and for analyzing the vulnerability of embedded BCI models in practical scenarios, we design new attack algorithm that generates smooth adversarial examples based on the signals' first derivative and model its propagation over the scalp based on a realistic head model by taking into consideration the attack source and the electrical and physical properties of the conducting tissues. This enables the creation of practically effective perturbations, that can be delivered by external device attack ERG-baaed BCIs at the source of signal acquisition. We attack the most energy-efficiend network that has been embedded on microcontrollers for smart wearable BCIs called EEGNet (Lawhern al.) 2018 Schneider et al.] [2020). It is a resource-friende friendly convolutional neural network (CNN) and is the SoA terms of accuracy and energy-efficiency trade-off (Belwali al. 2018 Malekmohammadi et al.
2019 Wang et al., 2020; Schneider et al. 2020).
We evaluate our methods and show experimental results on BCIs based on the motor imagery (MI) paradigm, which is of special interest among others because it can be asynchronously self-paced without external stimuli (Freer & Yang, 2020). By imagining the movement of different body parts, the decoded intention is translated into control signals. It is widely applied in several BCI applications, such as the control of wheelchairss (Yu et al. 2018), prosthetic armss (Elstob & Secco 2016), ground vehicles ZZhuang et al., 2021). and in communication (Brumberg et al. 2016). It has been proven to be the most difficult task to be attacked among the most common BCI paradigms (Zhang & Wu 2019; Meng et al. 2021). We evaluate
methods by "fooling" the victim model to always predict "rest" class. This essentially yields a DoS attack, because resting-statt EEG signals are generally interpreted as no subject's intention decoded, i.e., no control action needs to be taken by the BCI system (Yu et al.] |l.] (0188). While for healthy subjects it might solely cause user frustration and financial losses, for severely paralyzed patients can lead to loss of communication and independence, We generalize our methodology to an other MI task of BCI Competition IV-2a dataset and believe that it can be easily adapted to other BCI paradigms.
Main contributions. Our main contributions are:
We design a new method to generate smooth adversarial perturbations that are physiologically plausible and imperceptible to the human eye.
2

---

Under review as a conference paper at ICLR 2022
We consider practical scenario where the perturbation is added the signal acquisition source and model its propagation constrained by the physical properties of the human scalp.
The first study of adversarial perturbations in BCI consider the practical scenario of smart edge computing and physical signal propagation. We create both local and global perturbations and show that our attacks consistently achieve success rate of 50% in different settings pointing to the significant vulnerability of the SoA embedded EFGNet.
We hope that work raises awareness for potential risks and motivates the future development of appropriate countermeasures.
BACKGROUND
2.1 CLASSIFICATION BCIS
We first describe the commonly used approach in BCIs for classification. consisting of preprocessing step and classifier. The brain activity recorded with an EEG device which samples Noh channels rate F.. We define one trial j (X6), y()), where y E {0, 1,,,, Na - 13 is the true label of Ne MI tasks, and ( NRN, the multi-channel recording defined
x
(x
(
(
Non-1)
(1)
with xx E R.N. corresponding to the recording of the j-th trial and the i-th channel containing N, temporal samples. For simplicity, we denote X x and y == y(6)
The EEG recordings are often preprocessed with a band- paas filter, e.g., using Fast Fourier Transform (FFT) filter h-p before being fed a classifier, yielding
Xip Hap (X) = (Xup(Xu), hip((1),.... (xp(....-
Finally, the preprocessed signal Xbp is classified with trainable model f and is mapped to p f (Xtp). where E Na contains the output probabilities, e.g., originating from a softmax activation as final operation f. The model's final prediction ý is the index with the maximum score in p:
i f (Xop)
argmax f (Xtp) [y]. vE (O....... Na-1)
(3)
2.2 INSTANCE-BASER BASED ATTACKS
Instance-based attacks try to fool an EEG classifier / to misclassify an EEG signal X to targeted class y/- In this section, we describe the attack directly on the classifier J without considering the preprocessing Hop; the inclusion the preprocessing is described in Section 3.3 We define an
adversarial example as any X X + E N.N.NNN.. such that
X f (X')
= yt-
FGSM. The FGSM (Goodfellow et al.
2015) generates an adversarial perturbation V E N.x.N
of magnitude c which points in the negative direction of a loss function's 's gradient:
V
sign (Vx- (X,y:)),
(5)
where the loss function contains the negative log likelihood
1(X, y+) loo (p(y) (p(ye)) = log (f (X) [yr])
(6)
As p is the output of the softmax activation function, equation 6 becomes cross-entropy loss which maximizes the output ply.] while minimizing the remaining outputs.
PGD. The projected gradient descent (PGD) (adry et al. 2018) is variant of the basic iterative method (Kurakin al.), generally considered be more effective than FGSM. PGD aims to find perturbation by iteratively taking small steps of size in the gradient's 's direction and projecting the resulting perturbation back to the sample's neighborhood after each iteration. randomly initialize the attack inside the Lox ball of radius and update the attack V++1 for any iteration / with
V41 clip, (V, a sign (Vvl (X Vt, yt))),
(7)
where a is step size smaller than e which decays linearly with each iteration and the function clip, (+) clips the signal at the maximum desired amplitude
3

---

Under review as a conference paper at ICLR 2022
2.3 UNIVERSAL ATTACKS
UAPs have been introduced by Moosavi-Dezfooll et al. (2017) in the context of natural images, seeking to find image-agnostic perturbation that fools the classifier any input image. In the BCI domain (Liu et al. 2021). we seek to find a perturbation V E IRN,XNa, such that
f(x V) * f (X) for "most" X~ D,
where D is the distribution of the EEG data. The UAP can be determined by optimizing the negative log-likelihood loss with respect to V using batch gradient descent the trials in the training set.
3 MODELING PRACTICAL ATTACKS IN BCI
This section is the main contribution of the paper: we present a design of practical DoS attacks on MI- BCIs that operates at the source of the signal acquisition. We propose new method to eliminate the square wave artifacts to generate adversarial examples that are natural and physiologically plausible. The perturbation is emitted by a smart, adversarial device placed close to the ear, e-g., smart glass or in-ear headphones, and is propagated to the individual EEG electrodes over the scalp's s skin. As can be experimentally observed on measured EEG traces (Merlet et al. 2013 Sazgar & Young 2019), the same electrical source, e.g., electrocardiographc activities, is sensed by each EEG electrode with different degrees of attenuation and delay. We present a practical propagation model that determines the magnitude and delay for every individual electrode based on the distance along the scalp the adversarial device. The perturbation is trained end-to-end to fool the classifier to always output "rest," hence DoS, while respecting the spatial model and the amplitude constraints to remain imperceptible.
3.1 DESIGN AND ASSESSMENT OF PHYSIOLOGICALLY PLAUSIBLE ATTACKS
PGD-designed attacks on EEG tend to form perturbation signals which resemble a square-wave artifact (see Figure 2). an effect that has been observed on ECG data, too (Han et al., 2020 However, EEG signals are of random nature and can be modeled as frequency dependent stationary or non- stationary random processes (Karlekar & Gupta 2014). To this end, we introduce a new loss term in the PGD optimization such that the perturbation resembles the random nature of EEG signals, which we achieve by promoting signal changes represented in the first order derivative. We estimate the per-channel derivative V (V'+ v',, ... VN..-1 E N. 1xNNo using the sample-wisc difference:
v14 :- voll] vo - velt - 1] E {1, 2.., N₂ - 1}, CE I..,, No. -13
The additive loss term is determined by 4 (V) = - ENr, | where || 111 is the fn-rrm, e the maximum perturbation amplitude, and / 2 0 weighting factor. When designing a oinedimensional perturbation, the derivative loss becomes I (v) --
Measuring the Plausibility of Attacks None of the previous works have given quantitative measures to assess the physiological plausibility of an EEG adversarial attack. In this work, we propose data-driven measures for quantifying the naturalism of an attack. We compute either the cross correlation, the Euclidian distance, or the cosine similarity between the attacked signal and the original EEG, and average the values over the No. channels and over the samples in the dataset.
3.2 SPATIAL PROPAGATION MODEL
So far, a perturbation signal was designed for every individual channel. It unrealistic for an attacker to perturb the signal for all individual channels simultancously; hence, we consider more practical use case where the perturbation signal v E NR,,, is emitted from one location, e.g., from an adversarial device placed on the left side of the subject or close the left ear. More specifically, in this study, we assume that the EEG electrode at the position T9 according to the international 10-10 system (Sch). which is the closest to the left ear, senses the largest perturbation. The signal subsequently propagates over the skin to each electrode, which results in an individual magnitude and delay depending on the distance between the adversarial device and the electrode. More formally, we model the sensed perturbation at channel and time instant t
h5(v, Am. (t) = m(hi Am) v(t Aa)),
(10)
4

---

Under review as a conference paper at ICLR 2022
where m(4,, Am) and d(4,, Aa) are the magnitude and the delay respectively, both of which depend on the distance 11 and on characteristic parameters Am and Ag. We define the resulting multi-channel perturbation V E IR NNN,,, which is added to the multi-channel EEG signal, as
V(Ama Aa) = H Am: Ad) == (ho Am, Aa), hu (v, Am, Ad), hN - (v, Am Xa)). (11)
We estimate the distance I, between the electrode at position T9 and the remaining, attacked positions using the 10-10 system and a head model with radius of 8.7 em (Algazi et al.) 2001). We decouple the distance-dependent modeling of the magnitude and delay, explained in the following paragraphs.
Magnitude. For modeling the magnitude, we assume that the adversarial device injects or induces current I, yielding potential V measured near T9. The current propagates over the head surface through the skin to each of the remaining attacked electrodes, which can be modeled as a cylindrical resistor with resistance
4 GA
R
(12)
where is the conductivity of the skin which be in the range of [0.28, 0.87] Siemens/m (Vorwerk) eet al. 2019), and A is the area of the skin conductor. The potential at electrode is V, = =V R. and hence the magnitude can be described as
mlli, Al)
=
V - Vi
VoA = Am
14
(13)
where we further constrain 0 S "m(li, )m) S 1. The characteristic magnitude parameter Am represents the complex interplay between input current, voltage, conductivity, and area, covering various attack scenarios. We consider different characteristic magnitude parameters Am E [1, 15]. A large Am represents cases with large attenuation and limited propagation, i.c., a limited set of neighboring electrodes sense the perturbation. Conversely, a small Xm covers cases with lower attenuation where the perturbation can propagate further and infects all electrodes. We consider also an intermediate case where around half of the electrodes are affected by the attack with Am = 5. Appendix B provides examples of the magnitude of the spatial propagation on the head model.
Delay. The propagation of a signal on the head surface yields a position-dependeen phase angle or delay, as shown by experimental measurements of related studies (Plutchik & Hirsch 1963] Qiao et al. 1994). The delay stems from a combination of resistive and capacitive components that are encountered during the propagation of the signal, which can be modeled as an RC-cireuit with resistance R, capacity C, and time constant т = R C' that relates to the group delay. Specifically, the contacts between the electrodes and the skin are predominantly capacitive whereas the skin itself is both resistive and capacitive (Kim et al. 2010). As explained in the previous part, an increasing distance between the attacker and the target electrode yields a larger resistance R. As a result, the time constant and the delay increase too.
Here, we model a linear distance-delay relation. We rely on a study by Plutchik Hirsch (963)). which conducted human skin impedance and phase angle measurements by placing electrodes at an approximate distance of 10 em and applying voltages with frequencies in the range 2-1000 Hz. When assuming linear frequency-pha -haoee relation in low-frequency region (Qiao et al. 1994), one can derive the group delay to be 2.8 ms when considering measured angle of 10 at 10 As those measurements were conducted for only one distance, we extrapolate the delay for the remaining distances using a rectified linear model:
Aa . (4 - Lo) > ? d(1,, = >a 40) + do :d(1, da), =0
(14)
where dg 2.8 ms is the delay at distance to em. The delay depends not only on the distance, but also on other parameters such as the electrode-to-ski contact, the humidity of the skin, etc. To this end, we evaluate the propagation of the attack with different characteristic delay parameters Xa E 10.1, 0.563] s/m. With Xa- 0.1 we cover the cases where very little delay happens, while the largest considered Aa = 0.563 s/m yields a maximum delay of 0.1 s at the farthest electrode T10, which is in alignment with the observed EEG measurements (Merlet al. 2013) Sazgar & Young 2019). Similarly to Am we showcase also for an intermediate value of Xa 0.3 which corresponds to delay of 0.053 ms at T10.
5

---

Under review as a conference paper at ICLR 2022
Algorithm 1: Generation of physiologically plausible UAP.
input : Xtrains EEG training samples; Am; Ad, spatial propagation parameters; 8, weight of derivative loss term; 6, maximum perturbation amplitude; G, number of PGD iterations; F, number of epochs
output: :v, adversarial perturbation
<- (-c, R:::;
// Initialisation
for E E 1 Eo
Shuffle Xtrain;
for each batch B E Xtrain do 4 $ for g 1 to G do
+- H(v, Am, Ad):
// Spatial propagation
p (Hbp [B V));
// Model pass with perturbation
a sign
Vv ((p, Mrest)
- - =111111)): // Update w/derivative
clip, (v):
// PGD projection // Learning rate update
+ 0-1Ž 9
12
end
13
end
14 end
3.3 ATTACK DESIGN
We present practical DoS attacks in BCIs that respect domain constraints such as maximum amplitude, spectral distribution, physiological plausibility, and the spatial propagation of the perturbation. To this end, we formulate general objective function that contains the spatial propagation, the preprocessing step, and the first order derivative loss term:
Liot (X, V, Amz Aa) (Hop (X H(,, Am Xa)) (reet) -
111
(15)
where I(:,.) -) is the negative log-likelihood loss defined in equation 6 and f=1e-6 is a scalar that weights the contribution of the derivative loss term. We compare different attack scenarios:
Instance-based attacks. A perturbation computed using either FGSM or PGD based on the knowledge of the currently attacked EEG signal X. FGSM computes the perturbation stated in equation 5] where the e defines the perturbation amplitude which is varied between 1-50 mV. Alternatively, compute the perturbation using PGD with G=10 iterations, where each iteration consists of gradient-bbal (see equation 7.. The update rate is initialized with c/2 and linearly decreased with each iteration, reaching final value of 0.1 mV at iteration 10. The PGD computation restarted times with different initial perturbations, which are drawn from a uniform distribution within the range [-e, +e].
update of the perturbation and a projection to the L ball with radius e
Universal attacks. A universal perturbation is computed for all the samples in the training data. We optimize the UAP objective function
min Ex- (X, v, Am, Aa) s.t./lvlloc Se
(16)
using batched PGD. We pass batch of 16 samples together with the current perturbation through the preprocessing and classifier, compute the loss function, and update the perturbation based on the negative gradient with consecutive projection the Loo ball with radius e. This step repeated G=10 times before processing the next batch. Overall, the UAP is learned for F==10 epochs.
Propagation model. We distinguish between three use cases of spatial propagation model, where in all cases either instance specific attack or universal attack be computed: Case 1) Ignore the propagation model: a multi-channed perturbation V computed, which attacks each channel individually, replacing the terms H(v, Am, Aa) by V and v' by V in equation 15 Case 2) Consider

---

Under review as a conference paper at ICLR 2022
100
90
80
Ea-norm [mV]
[18]
70
PGD eobe:
clmV]
(a)
(b)
(c)
(a)
(b)
(c)
(a)
(b)
(c)
ASR [%c]
3.31
1.98
3.42
20.8
15.2
21.2
99.89
99.93
99,89
30
16.6
7.76
17.3
99.1
61.2
102
97.99
99.22
97.87
40
32.5
12.6
34.1
191
112
198
93.82
97.47
93.49
UAP daace Cacaeeeeeeao
79.61
90.05
78.92
30
125
39.0
135
823
462
855
64.17
79.70
63.06
20
10
Table Plausibility metrics for PGD attack (a) without derivative term, (b) with the derivative loss term, and (c)
25
50
Max. amplitude [mV]
with a Gaussian kernel (Han et al. (020). The smaller
the cross correlation and the Fuchidian distance f2-
Figure 1: Performance of random noise, FGSM, PGD, and UAP with and without derivative loss term.
norm, and the higher the cosine similarity y, the more natural the generated attack.
[]]
Original EEG
Adversarial perturbation
Attacked ERG
(a)
0
25
25
(b)
4
2
Time [s]
Figure 2: A successful attack (a) without and (b) with derivative loss term (PGD, c=10mV). The background traces show the original signal before the preprocessing filter.
the propagation model: a single-channel perturbation v computed and tested with specific propagation configuration Am and Åa- Case 3) Consider a use-case where the attacker does not know the spatial propagation model and computes the same perturbation v for channels. The actual propagation model is applied during testing model the real-world signal propagation.
End-to-end algorithm. We illustrate the algorithmic procedure for designing a physiologically plausible UAP in Algorithm I Analogously, the proposed methods of derivative loss and model propagation are applied with PGD. The hyperparameters a, 3, the number of PGD iterations and the restarts, the batch size and the number of epochs in UAP are determined based on cross-validated grid search on the training set.
4 EXPERIMENTS AND RESULTS
We evaluate our methods on the Physionet EEG Motor Movement/Imagery Dataset (Goldberger et al. 2000 (ch)) tackling inter-subject challenges, and generalize subject-specific inter-session dataset IV-2a of BCI Competition (Brunner et al. 2008) (See Appendix
Dataset. The Physionet dataset contains valid EEG recordings of 105 subjects (Dose et al. 2018) and is publicly available under Open Data Commons Attribution License v1.0. We use the Mi recordings that contain tasks of the imagination of left against right fist for 3 s. The EEG trials were recorded with -64 channels sampled at F.=160 Hz, yielding N,=3-160==80 samples per trial. Additional baseline runs provide resting-state data, where the subjects did not perform any tasks while having eyes open. Overall, we get total of 6615 trials with Na=3 balanced classes "left", "right", and "rest."
Training and validation. We train and validate both the classification models and the generated adversarial examples with 5-fold cross-validation, splitting the dataset into 84 subjects used for training and 21 subjects used for validation to effectively test the model on inter-subject variability. Similar to Wang et al. (020), which achieved SoA performance
this dataset, the baseline model is trained for 100 epochs using Adam with 3,=0.9, 3290.99 and batch size of 16. The learning rate is 0.01 and decreased by factor of 10 at epochs 20 and 50, achieving average accuracy of 74.78%.
7

---

Under review as a conference paper at ICLR 2022
Original EEG
Adversarial perturbation
Attacked EEG
100
100
100
-100
T10
100
100
2
Time [s]
Figure A successful attack with derivative loss and spatial constraints Am and Aa 0.563 (PGD, <=50mm. The background traces show original signal before preprocessing filter.
.dd PGD-w/HM
PGD-woHHM
UAP-WHHM
UAP-wOHHM
Am
Am
Am =1
Am
Ad
0.1
Au 0.3
0.563 Ad
50
ASR [%]
Am
75
5
Aa 0.1
Aa 0.3
Ad 0.563
ASA [36)
75
Am
Am
Am
Am
Aa 0.1
0.3
0.563
30
25
1.5 10
25
50 5 10
25
50
25
50
Max. amplitude [mVV]
Figure ASR with
PGD attack propagating from different EEG channels with fixed X2=0.3 and variable An.
Figure 4: ASR of PGD and UAP in Case 2). i.c., computed with head model (w/HM), and in Case 3), i.e., computed without head model (w/oHM).
An FFT band-pass filter hip with customary passband of 1-1440 Hz (Lawhern et al. 2018 used preprocessing step in both baseline and attack experiments. To determine the ASR, we compute the ratio between the successfully fooled trials, i.e., trials now classified as "rest", and the total number of attacked trials, where we only consider the ones initially correctly classified "rigt"/"rght"".
Physiologically plausible attacks. We first analyze the instance-based attacks without considering the propagation model (Case 1), depicted Figure I We compare our methods against random noise with amplitude as in (Zhang & Wu 2019). FGSM that is the same as in (Zhang & Wu 2019) with targeted scenario, and UAP designed specifically for EEG (Liu et al. 2021). For both FGSM and PGD, the ASR increases together with the maximum amplitude of the perturbation. They always outperform
random noise, with PGD performing slightly better than FGSM. They reach the maximum ASR of 99.97% with 10mV. The post-attack classification accuracy drops from 74.78% to 48% for perturbation amplitude of 2 m and to 33% for mV and higher amplitudes. Figure 2 shows the signals of successful attack using PGD. The adversarial perturbation has square- wave form which negatively affects the natural shape of the EEG signal. By adding the proposed derivative term, the square-wave artifacts significantly reduced (2p), making the perturbation more
8

---

Under review as a conference paper at ICLR 2022
physiologically plausible. When comparing the power spectral density of the original and attacked signals, the attacked signal designed without derivative presents large components in low frequencies, making it more easily detectable. Whereas the attack with derivative loss better resembles the power spectral density of the original signal (see Appendix D). Moreover, the introduction of the derivative term does not degrade the ASR (Figure I. The quantitative measures between the original and the adversarial samples in Table [ demonstrate that our proposed method with derivative term generates adversarial samples that are more similar to the original EEG, allowing them remain imperceptible even with high с (Appendix D). We reproduce the attacks using Gaussian kernel as in (Han et al. 2020). After tuning the kernel size and variance of the Gaussian kemel,, the method could not improve the plausibility metrics. The inferior performance of the Gaussian kernel could stem from the different nature of the signal: it was originally designed for ECGs which have a pseudo-periodic structure.
We extend the application of the derivative term to the UAP attack, while still not considering the propagation model (Case 1). F shows a comparison in performance for different values of c. The saturation in ASR is reached with higher e, i.e., 99. 99.94% with 50mV. This is expected since the UAP is a more difficult attack where a single set of perturbations per EEG channel is generated for all the test samples. Likewise in PGD, the ASR does not drop with the addition of the derivative term. We reproduce the UAP proposed by (Liu et al. 2021). Our UAP consistently reaches higher ASR.
Spatial Propagation. Finally, we introduce the spatial constraints in the signal propagation over the scalp (Case 2). We consider 9 different scenarios by combining 3 realistic attenuation configurations Am (1,5, 5, 15} with 3 delay configurations Xa E {0.1, 0.3, 0.563), which capture the range described in Section 3.2] For evaluating the highest achievable attack efficiency, we test a scenario where the attacker is assumed to know the propagation model: the adversarial perturbation is generated and evaluated on fixed spatial parameters Am, and Ad. shown in Figure 4 where the ASR reaches up 69.2% with PGD and 45.6% with UAP at 50mV. FigureBJdepice an example of a successful attack with the highest perturbation amplitude. The introduction of the spatial constraints makes the attack problem harder yielding seldom square distortions. However, the resulting EEG signals still resemble physiological random processes typical of EEGs. Next, we ablate the spatial constraints during generation and test the resulting perturbations on the 9 above-mentioned scenarios (Case 3). The ASR drops significantly, especially for Am=5 and Am=15 where the attenuation of the perturbation over the scalp is greater (see Figure [7.. and with the global UAP attack, where the attacker does not have access the attacked EEG signals.
Our spatial propagation models allows us to identify the vulnerability of the individual EEG channels. Figure 5 shows the ASR when initiating propagating it to the rest of the head. In the case with the greatest attenuation (Am=15) we find the maximum ASR at the electrode Cz between the regions of the electrodes C3 and C4, which are the most relevant ones for MI of the left and right hand tasks (Pfurtscheller & Lopes da Silva 1999). We (see When the attack propagates from the left side (T9), more samples with ground-truth label "right" can be fooled "rest", while the attacks from the right side (T10) are more effective "left" labels.
attack from specific channel (T9, T10, etc.) and
compute the pre- and post-attack confusion matrices for attacks from T9 and TIO (see Appendix EE)
Overall, our methods successfully generates perturbations resembling natural noise in EEGs, that can be added at the source of the signal acquisition and are propagated over the scalp, creating attacked signals that are physiologically plausible. Similar results have been observed on the BCI Competition IV-2a dataset, shown in Appendix C
CONCLUSION
With the incentive of improving security in BCIs, in this work, we demonstrated that DoS attacks are feasible and effective despite physical domain constraints. Experimental results reveal potential risks of realistic attacks on smart wearable BCIs and incentivize the need for future development of defense mechanisms while designing deep learning models to be embedded in smart wearable BCIs. Our detailed analysis on each EEG channel shows that special attention has to be paid, combined with the findings in neuroscience, to the brain regions that are found responsible for a specific task. In future work, the proposed attacks can cover uncertainty in the propagation model and the timing of the MI activity. Moreover, hardware implementations of such attacks can be created to evaluate the proposed methods in
with the ultimate goal of developing effective countermeasures.
9

---

Under review as a conference paper at ICLR 2022
ETHICS STATEMENT
The sore goal of this work is to raise awareness of potential adversarial attacks in BCIs and incentivize the development of coutermeasures, especially in the current moment when the BCIs are facing an increasing growth in applications of everyday life. The active development of smart wearable BCIs introducing paradigm shift where the processing algorithms are embedded near the data acquisition. While this improves the system security to a certain extend, with this work we have shown that not the only and ultimate way to safe and reliable BCI, since we have shown that BCI systems vulnerable also to attacks at the signals' source. We hope that our work sheds light on the fact that practical BCI systems vulnerable, despite the physical constraints, and motivates the design and development of more reliable and robust BCI systems.
REPRODUCIBLITY
A link to anonymous downloadable source code of this work is submitted as supplementary materials.
REFERENCES
BC12000: A general-purpose brain-computer interface (BCI) system.
Ayten Ozge Akmandor and Niraj Jha. Smart health care: An edge-side computing perspective. IEEE Consumer Electronics Magazine, 7(1):29-37, 2018.
Ralph Algazi, Carlos Avendano, and Richard O Duda. Estimation of spherical-head model from anthropometry. J. Aud. Eng. Soc., 49, 2001.
Pietro Aricò, Nicolina Sciaraffa, and Fabio Babiloni. Brain-computer interfaces: Toward daily life employment. Brain Sciences, 10(3), 2020.
Aamir Arsalan, Muhammad Majid, Amna Rauf Butt, and Syed Muhammad Anwar. Classification of perceived mental stress using a commercially available EEG headband. IEEE Journal of Biomedical and Health Informatics, 23(6):2257-2244 2019.
Christopher Beach, Ertan Balaban, and Alexander J. Casson. Chapter 14 edge algorithms for wearables: overview of truly multi-disciplinary problem. In Edward Sazonov (ed.), Wearable Sensors (Second Edition), pp. 379-414 Academic Press, Oxford, second edition edition, 2021.
Kais Belwafi, Olivier Romain, Sofien Gannouni, Fakhreddine Ghaffari, Ridha Djemal, and Bouraoui Ouni. An embedded implementation based on adaptive filter bank for brain crainccompueer interface systems. Journal of Neuroscience Methods, 2018
Sergio López Bernal, Alberto Huertas Celdrán, Gregorio Martínez Pérez, Michael Taynnan Barros, and Sasitharan Balasubramaniam. Security in brain-computer interfaces: State-of-the-art, opportunities, and future challenges. ACM Comput. Surv., 54(1), January 2021.
Florian C. Bodranghien, Margot Langlois Mahe, Serge Clément, and Mario U. Manto. pilot study on the effects of transcranial direct current stimulation on brain rhythms and entropy during self-paced finger movement using the Epoc helmet. Frontiers in Human Neuroscience, 11:201, 2017.
Jonathan S. Brumberg, Jeremy D. Burnison, and Kevin M. Pitt. Using motor imagery to control brain-computer interfaces for communication. In Dylan D. Schmorrow and Cali M. Fidopiastis (eds.), Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, pp.
14 15, Cham, 2016. Springer International Publishing.
Brunner, Leeb, R Müller-Putz, A Schlögl, and G Pfurtscheller. BCI competition 2008 Graz data set A, 2008. URL http::/bbciinee -ori http:/000.ee/aaaaaaaeee//aaaeeee
Ujwal Chaudhary, Niels Birbaumer, and Ander Murguialay.. Brain Brain-comee computer interfaces for communication and rehabilitation. Nature Reviews Neurology, 12, 08 2016.
10

---

Under review as a conference paper at ICLR 2022
Hauke Dose, Jakob S. Moller, Sadasivan Puthusserypady, and Helle K. Iversen. A deep learning
MI - EEG classification model for BCIs. In 2018 26th European Signal Processing Conference (EUSIPCO), pp. 1676-1679. IEEE, 2018.
Anirban Dutta. Brain-computer interface spellers for communication: Why we need to address their security and authenticity. Brain Sciences, 10:139, 03 2020.
Daniel Elstob and Emanuele Secco. low cost EEG based BCI prosthetic using motor imagery. International Journal of Information Technology Convergence and Services, 6:23-36, 02 2016.
Anna Fertonami, Clarissa Ferrari, and Carlo Miniussi. What do you feel if I apply transcranial electric stimulation? Safety, sensations and secondary induced effects. Clinical Neurophysiology, 126(11): 2181-2188, 2015.
Samuel G. Finlayson, John D. Bowers, Joichi Ito, Jonathan L. Zittrain, Andrew L. Beam, and Isaac S. Kohane. Adversarial attacks on medical machine learning. Science, 363(64433,,2877-1233 2019.
Flowneunoscience. Flow tDCS device. 2021-05-27.
2021. Accessed:
Daniel Freer and Guang-Zhong Yang. Data augmentation for self- motor imagery classification with C-LSTM. Journal of Neural Engineering, 17, 01 2020.
Ary L Goldberger, Luis A N Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph Mietus, George B Moody, Chung- Chung-Kang Peng, and Eugene Stanley. PhysioBank, Pookio,o and PhysioNet: components of research resource for complex physiologic signals. circulation, 101(23):e21-=2220, 2000.
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Proc. International Conference on Learning Representations, ICLR, 2015.
Xintian Han, Yuxuan Hu, Luca Foschini, Larry Chinitz, Lior Jankelson, and Rajesh Ranganath. Deep learning models for clectrocardiograms are susceptible to adversarial attack. Nature medicine, 26 (3):360-363, 2020.
Xue Jiang, Xiao Zhang, and Dongrui Wu. Active learning for black-box adversarial attacks in BEG- based brain computer interfaces. In 2019 IEEE Symposium Series on Computational Intelligence (SSC7), pp. 361-368, 2019.
Mandar Karlekar and Anubha Gupta. Stochastic modeling of EEG rhythms with fractional gaussian noise. In 2014 22nd European Signal Processing Conference (EUSIPCO), pp. 2520-2524, 2014.
Victor Kartsch, Giuseppe Tagliavini, Marco Guermandi. Simone Benatti. Davide Rossi, and Luca Benini. Biowolf: A sub-10-mb 8-channel advanced train-computed interface platform with nine-core processor and BLE connectivity. IEEE Transactions on Biomedical Circuits and Systems, 13(5):893-906, 2019.
Min Kim, Youngchang Cho. Suk-Tae Seo, Chang- -ik Son, Hee-Joon Park, and Yoon-Nyun Kim. A new method for non-invasive measurement of skin in the low frequency range. Healthcare informatics research, 16:143-8. 09 2010.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. CoRR, 002013607.0233.
Vernon J Lawhern, Amelia Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung, and Brent J Lance. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. Journal of Neural Engineering, 15(5):056013, 7 2018.
Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Texthugger: Generating adversarial text against real-world applications. In Proc. Network and Distributed System Security Symposium, NDSS. The Internet Society, 2019a.
Juncheng Li, Shuhui Qu, Xinjian Li, Joseph Szurley, J. Zico Kolter, and Florian Metze. Adversarial music: Real world audio adversary against wake-word detection system. In Proc. Neural Information Processing Systems (NeurIPS), pp. 11908-11918, 2019b.
11

---

Under review as a conference paper at ICLR 2022
Zihan Liu, Lubin Meng, Xiao Zhang. Weili Fang, and Dongrui Wu. Universal adversarial perturbations for CNN classifiers in EEG-based BCIs, 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In Proc. International Conference on Learning Representations (ICLR), 2018.
Alireza Malekmmohammaii, Hoda Mohammadzade, Alireza Chamanzar, Mahdi Shabany, and Benyamin Ghojogh. An efficient hardware implementation for motor imagery brain computer interface system. Scientia Iranica, 26:72-4 94, 2019.
Iuliana Marin, Myssar Jabbar Hammood Al-Battbootti, and Nicolae Goga. Drone control based on mental commands and facial expressions. In 2020 12th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), pp. 1-4, 2020.
Lubin Meng. Jian Huang, Zhigang Zeng, Xue Jiang, Shan Yu, Tzyy-Ping Jung, Chin- Chin-Teng Lin, Ricardo Chavarriaga, and Dongrui Wu. EEG-based brain-computer interfaces vulnerable to backdoor attacks, 2021.
Isabelle Merlet, Gwénaël Birot, Ricardo Salvador, Behnam Molaeo-Ardekani, Abeye Mckonnen, Aureli Soria-Frish, Giulio Ruffini, Pedro C. Miranda, and Fabrice Wendling. From oscillatory transcranial current stimulation scalp EEG changes: A biophysical and physiological modeling study. PLOS ONE, 8:1-12, 02 2013.
Seyed- Scyed-Mohsenn Moosavi-Dezfooli. Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
G. Pfurtscheller and F.H. Lopes Silva. Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical Neurophysiology, (00(1)):882-BBB 11 1999.
Robert Plutchik and Henry R Hirsch. Skin impedance and phase angle as a function frequency and current. Science, 141(354)) 1963.
ZG Qiao, Mørkrid, and NK Vøllestad. Characteristics of phase angle, extrapolated conductance and ion relaxation time in the low- foequeeyyy admittance locus plot of human palmar skin. Medical and Biological Engineering and Computing, 1112):161-167, 1994.
Yao Qin, Nicholas Carlini, Ian J. Goodfellow, Garrison W. Cottrell, and Colin Raffel. Imperceptible, robust, and targeted adversarial examples for automatic speech recognition. CoRR, atbs//993.10000 10346, 2019.
Mona Sazgar and Michael G. Young. EEG Artifacts, pp. 149-162. Springer International Publishing. Cham, 2019.
Tibor Schneider, Xiaying Wang, Michael Hersche, Lukas Cavigelli, and Luca Benini. Q-EEGNet: an energy-efficient 8-bit quantized parallel EEGNet implementation for edge motor-imagery brain- machine interfaces. In 2020 IEEE International Conference on Smart Computing (SMARTCOMP), pp. 214-289999 289, 2020.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Proc. International Conference Learning Representations, ICLR, 2014.
Johannes Vorwerk, Ümit Aydin, Carsten H Wolters, and Christopher R Butson. Influence of head tissue conductivity uncertainties on EEG dipole reconstruction. Frontiers in neuroscience, 13:531, 2019.
Xiaying Wang, Michael Hersche, Batuhan Tömekce, Burak Kaya, Michele Magno, and Luca Benini. An accurate EEGNet-based imtor-imagery brain computer interface for low- tow-power edge computing. In 2020 IEEE International Symposium on Medical Measurements and Applications (MeMeA), pp. 1-6, 2020.
12

---

Under review as a conference paper at ICLR 2022
Yang Yu, Yadong Liu, Jun Jiang, Erwei Yin, Zongtan Zhou, and Dewen Hu. An asynchronous control paradigm based on sequential motor imagery and its application in wheelchair navigation. JEEE Transactions Neural Systems and Rehabilitation Engineering, 26(12) 2018.
Wei Emma Zhang, Quan Z. Sheng, Ahoud Abdulrahmn F. Alhazmi, and Chenliang Li. Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Trans. Intell. Syst. Technol,, 11(3):24:1-2444 2020.
Xiao Zhang and Dongrui Wu. On the vulnerability of CNN classifiers EEG-based BCIs. IEEE Transactions Neural Systems and Rehabilitation Engineering, 27(5):814-825, 2019.
Jiayu Zhuang, Keke Geng, and Guodong Yin. Ensemble learning based brain-compute, interface system for ground vehicle control. IEEE Transactions on Systems, Man, and Cybernetics: Systems, (I())5332--5404 2021.
13

---

Under review as a conference paper at ICLR 2022
Signal acquisition
X + HM
Classification model
Control action
Preprocessing
Figure Practical adversarial attack scenario BCIs: a smart device close to the car emits perturbation signal which propagates over the head surface to the EEG electrodes.
T9
T10
TR
T9
(a) Am=
(b) Am 5
(c) Am=15
Figure Magnitude the spatial propagation for different Am. The perturbation emitted from left side of the head and propagates over the head surface. The leftmost electrode senses the highest magnitude (red), which linearly decreases towards zero (white) with growing propagation distance and Amn The electrodes which sense the perturbations, i.e., magnitude >0, are marked with dots. The electrodes T9, C3, C4, and T10 are labeled for reference.
A ATTACK AT THE SOURCE OF SIGNAL ACQUISITION
Fig. 6 iillustrates the attack scenario where the perturbation is delivered to the human scalp and propagates to the sensing electrodes at the source of the signal acquisition.
B SPATIAL PROPAGATION MODELS
Fig. 7 illustrates the magnitude of signal propagation using different propagation parameters Am = {1,5, 15). A large An represents cases with large attenuation and limited propagation (e.g., attack over the air) and small Am covers cases with lower attenuation where the perturbation can propagate farther (e.g., smart glass).
C EXPERIMENTS ON BCI COMPETITION IV-2A
Dataset. The IV-2a dataset of the BCI Competition contains recordings from nine different subjects and distinguishes between four classes of imagined movements: left and right hand, both feet, and the tongue. 22 different EEG channels with bandpass filter between 0.1 and 40 Hz. Each subject completed two recording session on two different days, where the first session is used for training and the second for testing as per the rules of the competition. Each session contains 288 trials.
recorded, sampled at 250 Hz. The data was pre-pn pre-processed
Training and validation. We train separate baseline model per subject using Adam optimizer with $1=0.9 and 3g=0.999, a batch size of 32, and 500 epochs. The learning rate is 0.001 achieving an average accuracy 71.79%. This dataset does not contain the rest class. We choose to design an attack that aims to fool the classifier to always predict "tongue." Moreover, we apply maximum perturbation amplitude of e E [0.01, 10) mV due the lower signal amplitude encountered this dataset.
Results. Fig. g compares the ASR of different attacks without considering the propagation model (Case 1). Generally, minimal perturbation amplitude of 1 mV and 2 mV suffices to achieve 100% ASR with PGD and UAP. respectively. The addition of the derivative loss term does not give any performance degradation in terms of the ASR. The average post-attack classification accuracy drops
14

---

Under review as a conference paper at ICLR 2022
100
80
PGD-w/Der.
70
PGD-w w/oDer.
ASR [%)
60
FGSM (Zhang & Wu 2019)
60
UAP-w/Der
UAP-w/Deer.
Random noise
30
UAP (Liu al. 2021
20
10
Max. amplitude e [mV]
Figure 8: ASR on BCI Competition IV-2a with random noise, FGSM, PGD, and UAP with and without derivative loss term.
from 71.79% to 50% for a perturbation amplitude of 0.15 mV and 24.7% for mV6 and higher amplitudes, when PGD with derivative is used.
Fig. shows the ASR for different propagation parameters (Am and Aạ) and maximum perturbation amplitudes e. When considering the head model during the design of the attack (Case 2, w/HM), both PGD and UAP reach significantly higher ASR compared to attacks designed without the consideration of the head model (Case 3, w/oHM).
D PLAUSIBILITY OF ATTACKS
This section provides power spectral density plots of original signals and attacked signals with and without the derivative loss term, shown in Figure 10. The power spectral density determined by computing the magnitude squared Fast Fourier Transform of the signals that were illustrated in Figure The attack designed with the derivative loss term has a similar distribution as the original signal, where the attack without derivative shows large contributions in the low frequency domain (<5 Hz), which were not present in the original signal. These low-frequency components stem from the square-wave shaped attack and can be used way detect the attack; hence, this attack cannot be considered imperceptible.
shows the attacks with and without derivative loss term with increasing maximum amplitude We can see that for low amplitudes (I mV and 5 mV) the generated attacks with and without derivative still look like EEGs. At 10 mV, the attack generated without derivative presents minor square square-wave wave artifacts, which could be still imperceptible to a non-expert. With 25 mV and 50 V. V. the ones generated without derivative have strong and perceptible square-ward wave displacements, while the ones generated with our proposed method can still be mistaken as real EEG signals. While with the instance-based attacks, (see Figure D with the universal attacks and physical constraints, the ASR increases with increasing perturbation amplitude (see Figure 4).
Moreover, Figure shoows
not necessary to have more than 10 m to get very high ASR
The same observations can be drawn from the plausibility metrics, which have been proposed for the first time in this paper to assess quantitatively the EEG attacks. For example, looking at the cosine similarity (7) in Table [[ without the derivative loss term, drops to 97.99% with = =5mV, whereas, with the derivative, y drops to about the same value of 97.47% with e = 10 mV, yielding an increase in ASR from 85% (5 V) to 99% m10 shown in Figure T with PGD.
15

---

Under review as a conference paper at ICLR 2022
.d.. PGD-w/HM 1
PGD-w/HH .dr..
UAP-W/HM - UAP-w/OHM
Am
Am =1
Am =1
Ad
0.1
Ad = 0.3
A 0.563
75
50
100
5
Am
5 Ad 0.563
Ad 0.1
Aa = 0.3
ASR [%
25
Am
15
Am 15
Am 15
Ad
0.1
Ad = 0.3
Aa 0.563
75
50
0.1 12
0.1
5
10
Max. amplitude [mV]
Figure Results on BCI Competition IV-2a. ASR of PGD and UAP in Case 2), i.e., computed with head model (w/HM); and Case 3), i.c., computed without head model (woHM).
4.0
Atlack without derivative
Attack with derivative
Power Spectral Density
Original EEG sample
3.0
2.5
2.0
1.5
1.0
0.5
du AAn
25
Frequency [Hz]
Figure 10: Power spectral density comparison of the attack with and without derivative loss term, as well as the original signal shown in Figure 2
CLASSIFICATION CONFUSION MATRICES
We analyze the confusion matrices before and after the proposed attack. Fig. [1 shows the confusion matrix of EEGNet on the Physionet dataset before the attack, where all classes can be classified with similar accuracy (72.8%-73.5
Fig. shows the confusion matrices for three different propagation parameters (Am {1, 5, 15}) and two attack positions (T9 and T10) which correspond to the left and right side the head. When considering the attacks from the left side, shown Fig. 12a [2c more samples with ground-truth label "right" can be fooled to "rest". This is particularly articulated in largely attenuated propagation model (Am=15). a similar vein, attacks coming from the right side of the head (T10) more effective data with ground-truth label "left" (Fig. 12d 120
16

---

Under review as a conference paper at ICLR 2022
Leht
72.8%
9.5%
17.7%
True label
10.2%
73.5 %
16.3%
11.3%
15.9%
72.8%
Resl
Loft
Night Predicted labe!
Rest
Figure 11: Confusion matrix original EEG predictions on Physionet dataset.
Lett
28.8%
1.1%
70.1%
Let:
37.9%
3.4%
58.7%
Len
30.8%
1.1%
68.0%
Thue labe!
True label
2.7%
25.2%
72.1%
Right
19.0%
17.9%
63.0%
Right
3.9%
21.3%
74.8%
1.6%
0.2%
98.2%
2.5%
0.2%
97.3%
1.1%
0.5%
98.4%
Lehi
Leh
Right
Rist
Letk
Right
Rest
Predicted label
Predicted label
Predicted label
(a) T9, Am. 1
(b) T9, Am
(c) T9, Am
15
Leb
74.7%
2.9%
72.3% %
Lot
28.6%
5.7%
65 8%
Lelk
22.2%
2.9%
74.8%
Truo label
True
1.6%
32.2%
66.2% %
Right
5.7%
30.2%
64.2%
True
1.4%
40.1%
58.5%
1.4%
0.5%
98.2% %
2.0%
98.0%
1.4%
0.7%
98.0%
Loft
Right
Rest
Lott
Right
Rest
Loft
Right
Rest
Predicted
Predicted label
Predicted label
(d) T1O, Am =1
(e) T10, Am
( T10, Am 15
Figure 12: Confusion matrices for the Physionet dataset after attacking EEGNet with the proposed PGD attack with derivative and considering the spatial propagation. The attack is either performed from the left electrode (T9) from the right electrode (T1O). We consider different magnitude propagation parameters Am and constant delay parameter 0.=0.3.
17

---

Under review as a conference paper at ICLR 2022
[]
Original EEG
Adversarial perturbation
Attacked EEG
50
(i)
50
(ii)
0
50
2
2
Time [s]
(a) =lmV
[m]]
Original EEG
Adversarial perturbation
Attacked EEG
50
0
he/ Ww
50
(ii)
0
50
Time [s]
(b) =5mV
Im
Original EEG
Adversarial perturbation
Attacked EEG
W/ www hwppl
50
(ii)
0 km Wwy
2
Time [s]
(c) =10mV
[m]]
Original EEG
Adversarial perturbation
Attacked EEG
(i) 0
ww M
50
(ii)
hm
Ww hww mm
2
2
Time [s]
(d) e =5mV
[m]]
Original EEG
Adversarial perturbation
Attacked EEG
50
(i)
0
hA Wv www
50
50
(ii)
0
50
2
2
Time [s]
(c) 550m
Figure 13: successful PGD attack Physionet dataset (i) without and (ii) with derivative with different values of maximum amplitude e.
18


Reviews:
[{"review_id": "vcoffVGAchc", "reviewer": "Reviewer_D9vr", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I believe this paper addresses a quite relevant topic, of interest for the community: the impact of adversarial perturbations in sensitive wearable systems. I think both motivation and contribution of the work are correct, with a proper statement of the problem and a comprehensive SoA analysis. The paper is well written, with no detecting spelling issues, and overall easy to read.  The overall contribution is adequate, being my only concern related to the completeness of the work and the overall reproducibility of results.\n\nI believe the general structure of the manuscript  is correct. It includes a proper explanation of BCI, SoA, background of the attack, and a dataset that, although limited, given the scarce public dataset for this domain I consider adequate. The part which I consider could be improved, and hence my recommendation, is the description of the method and experimental setup. \n\nSince this approach relies on affecting a ML system to misclasify its input, I’d expect to see a more detailed version of adversarial architecture and it’s training process. Training params are indeed included but after reading the paper several times I’m not sure about the adversarial approach to get access to the gradient of the attacked system. \n\nHonestly I’d rather see less details about the propagation and plausibility of the attack, and even remove some figures (as the Figure1 which do not offer substantial information) and include the end-to-end training algorithm  used to generate the adversarial model (or substitute) in a figure. As I mentioned before I’ve serious doubts that this system could be reproduced  with the details provided in the paper. By the way, I did not see any supplementary material which I understood the authors attached. I’d suggest adding the end-to-end architecture plus training algorithm, with some details and metrics about the training and inference process. \n\nThe paper may raise some ethical concerns since the topic is quite sensitive, but I think these matters have been explained adequately by the authors in the manuscript.\n", "summary_of_the_paper": "Authors propose a method to perform adversarial attacks on BCI deployed on wearable devices. Using an electrode enclosed in a head mounted device, they show how to propagate a perturbation through the subject’s scalp capable of significantly affecting a EGG based classification model. Authors test the approach on the movements detection domain using a public dataset, showing how the proposed attack can lead to a significant decrease of  the system’s performance.", "summary_of_the_review": "I consider this paper as an interesting work with a proper motivation and novelty component. The topic addressed (adversarial perturbations on wearable BCI ) can be definitively of interest for the conference audience. The main improvement point in my opinion is the description of method and experimental setup. Further details on the adversarial approach I think would improve the paper. I consider the overall contribution of the work as adequate, and hence my recommendation\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "cMev5IHKJWw", "reviewer": "Reviewer_2xxQ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is an interesting study which addresses AI-based medical applications targeted for edge devices. Related work is well presented and contributions are explained, however the novelty of the work is not clear, as some methods for generating smooth adversarial examples for different electrical body signals already exist. I believe similar concepts as used for example in ECG might be applicable to EEG as well. It would be beneficial to a reader to compare those. \n\nThe flow of the work should be also improved for better readability. The related work section could be included in the introduction, as the introduction already contains some comparison with previous studies. Figures placement, ordering, descriptions and references in the next definitely need to be improved.  Below Eq. 9 the loss term is defined in words, while it would be better to use the equation. I believe it will be the same as the loss in Eq. 15(?), so maybe consider restructuring the work to improve its flow. Please also explain how attenuation configurations were selected, specifically explain the reason for choosing specific magnitude and delay values. \n\nThe presented experiments were performed using one EEG dataset for a specific task of the motor imagery. It would be interesting to verify if the method can scale to other tasks as well and generalize to other datasets. Please elaborate on that. \n\nAlso, some other more recent studies on EEG classification using e.g., transformers already exist. What is the reason for examining CNN models? Would transformers be more robust to adversarial attacks in this application, given the fact that they were proved more robust in other studies?\n\n", "summary_of_the_paper": "The paper examines vulnerability of the brain computer interfaces to adversarial attacks. A new method is proposed to generate more realistic, smoother representations of the adversarial attack. A case where an attack happens at a specific location (close to an ear) is studied as a representative case of the perturbation added to the signal acquisition source. ", "summary_of_the_review": " This is an interesting study, but in my opinion requires some additional work to make the contribution more solid, specifically comparison with other smooth adversarial attacks on electrical signals should be provided, and selection of CNNs vs transformers should be explained. It would be also beneficial to verify the proposed methods with other datasets/other EEG-based tasks if possible. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "5yZFAQ6m9oo", "reviewer": "Reviewer_n45G", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "MI-BCI systems are ultimately motivated to provide alternative communication and control means for people with severe neuromuscular disabilities. In this context, it is not really easy for me to understand the motivation of investigating adversarial attacks to induce DoS for a MI-BCI system in experiments, or at least it is not clearly put in the paper. I believe this motivation can only be applicable to a reasonable extent with certain scenarios like EEG-based biometric identification. Going further, how to possibly overcome this vulnerability that is proposed in the paper is also not discussed or studied, which would be in my opinion a more thorough study if preliminary analyses with standard adversarial training like approaches are at least briefly assessed?\n\nAuthors motivate their approach in comparison to existing attacks based on arguments that I am not fully convinced about. For instance, authors argue that the perturbations should be imperceptible and smooth. However, the follow-up argument on realizing artifacts at the source-level relies on e.g., tDCS delivered directly to the scalp (which is generally not imperceptible within the first few milliseconds, when EEG time-series is simply monitored). In fact, every noise-like artifact (sometimes even additive square-waves) can be rather considered imperceptible with EEG, as opposed to the large artifacts caused by tDCS or TMS at the instant. What would the authors comment on that?\n\nAny BCI system that utilizes a discriminative DNN (e.g., EEGNet) is likely to be vulnerable to gradient-based adversarial perturbations as the authors well demonstrate. However most BCI systems (particularly MI-BCIs) that use conventional signal processing methods (i.e., filter-bank common spatial patterns or feature extraction methods based on Riemannian geometry) in fact still yields state-of-the-art (or comparable) performances with respect to deep feature learning models. It would be interesting for the current study to discuss or empirically investigate if such a vulnerability would be present (with similar impact?) in a BCI system running on such conventional feature decoding methods?\n\nSome methodological comparisons are necessary, and missing in the paper. A baseline could simply be to add white noise or square waves to the signals (i.e., random pseudo-perturbations that have a maximum amplitude bound), and obtain a baseline attack success rate. Similarly, comparisons to state-of-the-art ideas on adversarial attacks to BCIs are not included (e.g., [Zhang & Wu, 2019] or [Liu et al, 2021] where the signal attacks/jamming occur after the EEG acquisition phase). For instance, how do the authors' UAP attacks relate to the [Liu et al, 2021] study, when one does not consider the spatial propagation constraints?\n\nOnline decoding for MI-BCIs is a very challenging task by itself, especially if one considers inter-subject decoding (as in the paper). Can the authors also present some subject-specific pre-attack and post-attack accuracies to give a better overview of how much the BCI system truthfully suffers from this vulnerability, since pre-attack MI decoding performances can be already lower in the current setting?\n\nRegarding the visualizations in Figure 5, additional power spectral density plots would be helpful to even better demonstrate the similarities between original and attacked EEG signals, as they are generally monitored to check EEG for irregularities.\n\nIt would be also interesting to see if the results would look exactly similar, when one imposes the same adversarial attacks with similar parameters, starting from the right ear location electrode T10 which spatially propagates towards T9 the other way around (i.e., impacts C3 less than C4). Accordingly with this question, I would be also curious how are the attack success confusion matrices look like?, i.e., are the attacks having more impact on right hand motor imagery detection due to targeting T9?\n\nSome minor comments:\n- Typo in Eq 14.\n- End of page 6: \"log-loss\" -> \"negative log-likelihood loss\"\n- I would label the y-axis of Figure 3 as \"attack success accuracy\" rather than \"accuracy\".", "summary_of_the_paper": "Authors study the vulnerability of brain-computer interface (BCI) systems from a security perspective, and present a physiologically plausible adversarial attack approach on BCI systems that can induce denial-of-service after being transmitted by external devices. Empirical assessments are performed on an offline three-class motor imagery (MI) BCI dataset (right hand, left hand, and rest classes), where denial-of-service indicates a \"rest\" classification regardless of the subjects' motor imagery intent. Proposed methods are motivated to be physiologically plausible and effective to an extent that can cause failure modes for DNN-calibrated BCI systems.", "summary_of_the_review": "The paper is well written with a clear narrative, and used methods are well described. Main limitations of this work is on the presentation of the general motivating perspective, as well as a general lack of methodological comparisons/evaluations and discussions on possible countermeasures that one can consider for DoS attacks on BCI systems. Some experiments/comparisons are highlighted and suggested in my main review for the authors.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "53QuJIzkpoi", "reviewer": "Reviewer_VvTE", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. the motivation and contributions are well written. There are indeed few studies on the adversarial attacks on BCI.\n\nWeaknesses:\n1. This work is NOT actually implementing attack (as implied in the Introduction) with a physical device, but simulating attack! It's a huge difference. Simulation is much more easier and costs much less effort. See more details in the following review summary.", "summary_of_the_paper": "This work analyzes the adversarial attacks on BCI systems. It induces DoS attack and incorporates domain-specific insights to achieve the adversarial attack. The designed attack is evaluated on a public dataset EEGMMIDB and shows that a popular deep learning model, EEGNet, is vulnerable to adversarial attacks.", "summary_of_the_review": "The whole paper (Introduction, Fig 1, etc.) implies that this study has a smart device that can be attached to the left ear that will inject well-designed perturbation to collected EEG data. I was excited when reading the paper and I know how hard it is. It requires a multidisciplinary team to design the wireless smart device, manufacture the device and make sure it really works, break the EEG collection equipment (like Emotive headset) which is difficult as the signal transmission is generally encrypted, and so on. I was intended to give an accept (or even a strong accept) as long as they can implement the whole system in the real world. \n\nBut it's not the case! The dataset is not locally acquired but is the most well-known public EEGMMIDB dataset, which takes little effort to get. The attack is not really deployed with a smart device at the left ear which can emit perturbations to the source signal, but is simulated. There is no device, no signal transmission, no poisoning attack on source signals. \n\nWithout the effort on physical systems and real-world attacks, this paper is still good but not good enough for ICLR. It's a good story, but still a story. So I decided to give a rejection.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work has triggered engaged discussions between authors and reviewers and also among reviewers.
These conversations have highlighted the potential weaknesses of the contribution, namely that the work
is a proof-of-concept experimentally (although arguably for ethical reasons) and that the
overall theoretical contribution is not strong.

Despite the merit of this work, and given the strong expectations of ICLR, I don't feel that this work
can be endorsed for publication at ICLR 2022.
Decision: Reject