P RACTICAL A DVERSARIAL A TTACKS ON B RAIN – C OMPUTER I NTERFACES
Anonymous authors
Paper under double-blind review
A BSTRACT
Deep learning has been widely employed in brain–computer interfaces (BCIs) to decode a subject’s intentions based on recorded brain activities enabling direct interaction with computers and machines.
BCI systems play a crucial role in motor rehabilitation and have recently experienced a signiﬁcant market boost as consumer-grade products.
Recent studies have shown that deep learning-based
BCIs are vulnerable to adversarial attacks.
Failures in such systems might cause medical misdiagnoses, physical harm, and ﬁnancial damages, hence it is of utmost importance to analyze and understand in-depth, potential malicious attacks to develop countermeasures.
In this work, we present the ﬁrst study that analyzes and models adversarial attacks based on physical domain constraints in EEGbased BCIs.
Speciﬁcally, we assess the robustness of EEGNet which is the current state-of-the-art network for embedded BCIs.
We propose new methods to induce denial-of-service attacks and incorporate domain-speciﬁc insights and constraints to accomplish two key goals: (i) create smooth adversarial attacks that are physiologically plausible; (ii) consider the realistic case where the attack happens at the origin of the signal acquisition and it propagates on the human head.
Our results show that EEGNet is signiﬁcantly vulnerable to adversarial attacks with an attack success rate of more than 50%.
With our work, we want to raise awareness and incentivize future developments of proper countermeasures.
I NTRODUCTION
Recent work has shown that adversarial perturbations can cause state-of-the-art (SoA) deep learning models to misbehave in various domains including vision (Szegedy et al., 2014; Goodfellow et al., 2015), NLP (Li et al., 2019a; Zhang et al., 2020), speech (Qin et al., 2019; Li et al., 2019b) and biomedicine (Finlayson et al., 2019; Han et al., 2020).
Neural networks have been applied in brain–computer interfaces (BCIs) achieving impressive results (Lawhern et al., 2018; Dose et al., 2018).
A BCI enables direct interactions with external devices based on brain activities, typically recorded using electroencephalographic (EEG) systems.
It can provide a communication pathway for severely paralyzed patients or assist in rehabilitation (Chaudhary et al., 2016).
Besides medical applications, recent developments in wearable devices have pushed BCIs towards consumer-grade products to improve life quality (Aricò et al., 2020), e.g., the Interaxon Muse headband for stress relief (Arsalan et al., 2019) or the Emotiv headset for controlling drones (Marin et al., 2020) and ground vehicles (Zhuang et al., 2021).
Safety in BCI systems is paramount (Dutta, 2020; Bernal et al., 2021), because a failure would cause misdiagnoses, user frustration, or even danger while driving a wheelchair or controlling a drone, causing physical and ﬁnancial damages.
Zhang & Wu (2019) were the ﬁrst to show that EEG-based BCIs are vulnerable to adversarial attacks by proposing an unsupervised fast gradient sign method (FGSM) (Goodfellow et al., 2015). More recent work has proposed a more practical attack where a universal adversarial perturbation (UAP) is computed once and can be applied to all EEG trials without learning it for every new input (Liu et al., 2021).
Both works assume that the acquired signals are sent to a remote compute engine, e.g., a computer, and the attacker can alter the signals during the transmission by attaching a “jamming” module between the signal preprocessing step and the classiﬁer.
Recent developments in smart edge computing (Akmandor & Jha, 2018; Beach et al., 2021) eliminate the need for data transmission, making this attack scenario inapplicable.
Novel BCI solutions (Kartsch et al., 2019; Wang et al., 2020) embed the signal processing and classiﬁcation directly at the sensor edge. A more practical adversarial example has been identiﬁed by Meng et al.
It consists of a square-shaped signal that can be added to EEG trials before the preprocessing step. However, the attack is proposed as a backdoor key, which means that the attacker has direct access to the training dataset and pollutes it with adversarial examples, which is improbable if the attacker is not directly involved in the data acquisition or in the training of the classiﬁer.
Li et al.
(2019b) have shown an attack scenario in the audio domain by considering the on-board edge processing of a wake-word detection system, where an adversarial audio trace is delivered to the environment causing denial-of-service (DoS).
No similar studies can be currently found in the BCI domain.
Challenges: Designing natural attacks and modeling its propagation.
Unlike in audio applications where the signal can simply propagate over-the-air and is sensed by a microphone, extra modeling is required to evaluate the signal propagation in BCIs based on the physical properties of the biological tissues.
In this work, rather than assuming a “jamming” module between the preprocessing and the classiﬁcation steps as in related works, we consider a more realistic and practically applicable attack scenario where the adversarial perturbations are introduced at the source of the data acquisition, as showcased in Figure 6 in Appendix A. This can be achieved, for example, via electromagnetic waves delivered to the environment (Dutta, 2020) or via transcranial current stimulation with electrical current delivered directly to the scalp (Bodranghien et al., 2017; Fertonani et al., 2015), by exploiting wearable devices, such as smart glasses or over-ear headsets (Flowneuroscience, 2021; Marin et al., 2020).
The adversarial perturbations translate into electrical signals propagating over the scalp and are sensed by the electrodes in addition to the EEG signals.
To guarantee the imperceptibility of the attacks, previous works in BCIs create perturbations that are small in amplitude (Zhang & Wu, 2019; Jiang et al., 2019; Liu et al., 2021), limiting the attack success rate (ASR). Increased perturbation’s amplitude yields higher ASR (Meng et al., 2021), but makes the attack more easily detectable. Moreover, the generated perturbations are square-shaped, which is implausible for biosignals. Han et al.
(2020) are the ﬁrst to observe square-wave artifacts in biosignals’ attacks and propose smooth perturbations for electrocardiograms (ECGs).
No similar works have been found for EEGs.
This work: Practical attacks on BCI models. To address the above technical challenges, and for analyzing the vulnerability of embedded BCI models in practical scenarios, we design a new attack algorithm that generates smooth adversarial examples based on the signals’ ﬁrst derivative and model its propagation over the scalp based on a realistic head model by taking into consideration the attack source and the electrical and physical properties of the conducting tissues. This enables the creation of practically effective perturbations, that can be delivered by an external device to attack
EEG-based BCIs at the source of signal acquisition.
We attack the most energy-efﬁcient network that has been embedded on microcontrollers for smart wearable BCIs called EEGNet (Lawhern et al., 2018; Schneider et al., 2020). It is a resource-friendly convolutional neural network (CNN) and is the
SoA in terms of accuracy and energy-efﬁciency trade-off (Belwaﬁ et al., 2018; Malekmohammadi et al., 2019;
Wang et al., 2020; Schneider et al., 2020). We evaluate our methods and show experimental results on BCIs based on the motor imagery (MI) paradigm, which is of special interest among others because it can be asynchronously self-paced without external stimuli (Freer & Yang, 2020).
By imagining the movement of different body parts, the decoded intention is translated into control signals.
It is widely applied in several BCI applications, such as the control of wheelchairss (Yu et al., 2018), prosthetic armss (Elstob & Secco, 2016), ground vehicles (Zhuang et al., 2021), and in communication (Brumberg et al., 2016).
It has been proven to be the most difﬁcult task to be attacked among the most common BCI paradigms (Zhang & Wu, 2019; Meng et al., 2021).
We evaluate our methods by “fooling” the victim model to always predict “rest” class. This essentially yields a DoS attack, because resting-state EEG signals are generally interpreted as no subject’s intention decoded, i.e., no control action needs to be taken by the BCI system (Yu et al., 2018).
While for healthy subjects it might solely cause user frustration and ﬁnancial losses, for severely paralyzed patients it can lead to loss of communication and independence.
We generalize our methodology to an other MI task of BCI Competition IV-2a dataset and believe that it can be easily adapted to other BCI paradigms.
Main contributions. Our main contributions are:
• We design a new method to generate smooth adversarial perturbations that are physiologically plausible and imperceptible to the human eye.
• We consider a practical scenario where the perturbation is added at the signal acquisition source and model its propagation constrained by the physical properties of the human scalp.
• The ﬁrst study of adversarial perturbations in BCI to consider the practical scenario of smart edge computing and physical signal propagation.
We create both local and global perturbations and show that our attacks consistently achieve a success rate of > 50 % in different settings pointing to the signiﬁcant vulnerability of the SoA embedded EEGNet.
We hope that our work raises awareness for potential risks and motivates the future development of appropriate countermeasures.
B ACKGROUND 2. C LASSIFICATION IN BCI S
We ﬁrst describe the commonly used approach in BCIs for classiﬁcation, consisting of a preprocessing step and a classiﬁer.
The brain activity is recorded with an EEG device which samples N ch channels at rate F s .
We deﬁne one trial j as ( X ( j ) , y ( j ) ) , where y ( j ) ∈ { 0 , 1 , ..., N cl − 1 } is the true label of
N cl MI tasks, and
X ( j ) ∈
R
N s × N ch the multi-channel recording deﬁned as
[Equation]
[Equation] with x ( j ) i ∈ R N s corresponding to the recording of the j -th trial and the i -th channel containing N s temporal samples.
For simplicity, we denote X :=
X ( j ) and y :=
y ( j ) .
The EEG recordings are often preprocessed with a band-pass ﬁlter, e.g., using a Fast Fourier Transform (FFT) ﬁlter h bp ( · ) , before being fed to a classiﬁer, yielding
[Equation]
Finally, the preprocessed signal X bp is classiﬁed with a trainable model f and is mapped to p := f ( X bp ) , where p ∈ R N cl contains the output probabilities, e.g., originating from a softmax activation as ﬁnal operation in f .
The model’s ﬁnal prediction ˆ
y is the index with the maximum score in p :
[Equation]
2. I NSTANCE - BASED ATTACKS
Instance-based attacks try to fool an EEG classiﬁer f to misclassify an EEG signal X to a targeted class y t .
In this section, we describe the attack directly on the classiﬁer f without considering the preprocessing H bp ; the inclusion of the preprocessing is described in Section 3.3.
We deﬁne an adversarial example as any X ∗ =
X + V ∈ R N s × N ch such that
[Equation]
FGSM.
The FGSM (Goodfellow et al., 2015) generates an adversarial perturbation V ∈ R N s × N ch of magnitude (cid:15) which points in the negative direction of a loss function’s gradient:
[Equation]
where the loss function contains the negative log likelihood
[Equation]
As p is the output of the softmax activation function, equation 6 becomes a cross-entropy loss which maximizes the output p [ y t ] while minimizing the remaining outputs.
PGD.
The projected gradient descent (PGD) (Madry et al., 2018) is a variant of the basic iterative method (Kurakin et al.), generally considered to be more effective than FGSM.
PGD aims to ﬁnd a perturbation by iteratively taking small steps of size α in the gradient’s direction and projecting the resulting perturbation back to the sample’s neighborhood after each iteration.
We randomly initialize the attack inside the L ∞ ball of radius (cid:15) and update the attack V t +1 for any iteration t with
[Equation]
where α is a step size smaller than (cid:15) which decays linearly with each iteration and the function clip (cid:15) ( · ) clips the signal at the maximum desired amplitude (cid:15) .
2. U NIVERSAL ATTACKS
UAPs have been introduced by Moosavi-Dezfooli et al.
(2017) in the context of natural images, seeking to ﬁnd an image-agnostic perturbation that fools the classiﬁer on any input image.
In the BCI domain (Liu et al., 2021), we seek to ﬁnd a perturbation V ∈ R N s × N ch such that
[Equation]
where D is the distribution of the EEG data.
The UAP can be determined by optimizing the negative log-likelihood loss with respect to V using batch gradient descent on the trials in the training set.
M ODELING PRACTICAL ATTACKS IN BCI
This section is the main contribution of the paper: we present a design of practical DoS attacks on MIBCIs that operates at the source of the signal acquisition.
We propose a new method to eliminate the square wave artifacts to generate adversarial examples that are natural and physiologically plausible.
The perturbation is emitted by a smart, adversarial device placed close to the ear, e.g., a smart glass or in-ear headphones, and is propagated to the individual EEG electrodes over the scalp’s skin.
As can be experimentally observed on measured EEG traces (Merlet et al., 2013; Sazgar & Young, 2019), the same electrical source, e.g., electrocardiographic activities, is sensed by each EEG electrode with different degrees of attenuation and delay.
We present a practical propagation model that determines the magnitude and delay for every individual electrode based on the distance along the scalp to the adversarial device.
The perturbation is trained end-to-end to fool the classiﬁer to always output “rest,” hence DoS, while respecting the spatial model and the amplitude constraints to remain imperceptible.
3. D ESIGN AND A SSESSMENT OF P HYSIOLOGICALLY P LAUSIBLE A TTACKS
PGD-designed attacks on EEG tend to form perturbation signals which resemble a square-wave artifact (see Figure 2), an effect that has been observed on ECG data, too (Han et al., 2020).
However,
EEG signals are of random nature and can be modeled as frequency dependent stationary or nonstationary random processes (Karlekar & Gupta, 2014).
To this end, we introduce a new loss term in the PGD optimization such that the perturbation resembles the random nature of EEG signals, which we achieve by promoting signal changes represented in the ﬁrst order derivative.
We estimate the per-channel derivative V (cid:48) =
( v (cid:48) 0 , v (cid:48) 1 , ..., v (cid:48)
N ch − 1 ) ∈ R N s − 1 × N ch using the sample-wise difference:
[Equation]
The additive loss term is determined by l 1 ( V ) = − β(cid:15) (cid:80) N ch c
=1 || v (cid:48) c
|| 1 , where || · || 1 is the (cid:96) 1 -norm, (cid:15) the maximum perturbation amplitude, and β ≥ {+
 a weighting factor. When designing a one-dimensional perturbation,+} the derivative loss becomes l 1 ( v ) = − β(cid:15) || v (cid:48)
|| . Measuring the Plausibility of Attacks None of the previous works have given quantitative measures to assess the physiological plausibility of an EEG adversarial attack.
In this work, we propose data-driven measures for quantifying the naturalism of an attack.
We compute either the cross correlation, the Euclidian distance, or the cosine similarity between the attacked signal and the original EEG, and average the values over the N ch channels and over the samples in the dataset.
3. S PATIAL PROPAGATION MODEL
So far, a perturbation signal was designed for every individual channel.
It is unrealistic for an attacker to perturb the signal for all individual channels simultaneously; hence, we consider a more practical use case where the perturbation signal v ∈ R N s is emitted from one location, e.g., from an adversarial device placed on the left side of the subject or close to the left ear.
More speciﬁcally, in this study, we assume that the EEG electrode at the position T9 according to the international 10-10 system (Sch), which is the closest to the left ear, senses the largest perturbation.
The signal subsequently propagates over the skin to each electrode, which results in an individual magnitude and delay depending on the distance between the adversarial device and the electrode.
More formally, we model the sensed perturbation at channel i and time instant t as
[Equation]
where m ( l i , λ m ) and d ( l i , λ d ) are the magnitude and the delay respectively, both of which depend on the distance l i and on characteristic parameters λ m and λ d .
We deﬁne the resulting multi-channel perturbation V ∈ R N s × N ch , which is added to the multi-channel EEG signal, as
[Equation]
We estimate the distance l i between the electrode at position T9 and the remaining, attacked positions using the 10-10 system and a head model with a radius of 8.7 cm (Algazi et al., 2001).
We decouple the distance-dependent modeling of the magnitude and delay, explained in the following paragraphs.
Magnitude.
For modeling the magnitude, we assume that the adversarial device injects or induces a current I , yielding a potential V measured near T9.
The current propagates over the head surface through the skin to each of the remaining attacked electrodes, which can be modeled as a cylindrical resistor with resistance
[Equation]
where σ is the conductivity of the skin which can be in the range of [0.28, 0.87]
Siemens/m (Vorwerk et al., 2019), and A is the area of the skin conductor.
The potential at electrode i is V
i
=
V − I · R i , and hence the magnitude can be described as
[Equation]
(13) where we further constrain 0 ≤ m ( l i , λ m ) ≤ 1 .
The characteristic magnitude parameter λ m represents the complex interplay between input current, voltage, conductivity, and area, covering various attack scenarios. We consider different characteristic magnitude parameters λ m ∈ [1 , 15] .
A large λ m represents cases with large attenuation and limited propagation, i.e., a limited set of neighboring electrodes sense the perturbation.
Conversely, a small λ m covers cases with lower attenuation where the perturbation can propagate further and infects all electrodes.
We consider also an intermediate case where around half of the electrodes are affected by the attack with λ m = 5 . Appendix B provides examples of the magnitude of the spatial propagation on the head model.
Delay.
The propagation of a signal on the head surface yields a position-dependent phase angle or delay, as shown by experimental measurements of related studies (Plutchik & Hirsch, 1963; Qiao et al., 1994).
The delay stems from a combination of resistive and capacitive components that are encountered during the propagation of the signal, which can be modeled as an RC-circuit with resistance R , capacity C , and time constant τ = R · C that relates to the group delay.
Speciﬁcally, the contacts between the electrodes and the skin are predominantly capacitive whereas the skin itself is both resistive and capacitive (Kim et al., 2010).
As explained in the previous part, an increasing distance between the attacker and the target electrode yields a larger resistance R .
As a result, the time constant τ and the delay increase too.
Here, we model a linear distance-delay relation.
We rely on a study by Plutchik & Hirsch (1963), which conducted human skin impedance and phase angle measurements by placing electrodes at an approximate distance of 10 cm and applying voltages with frequencies in the range 2–1000 Hz.
When assuming a linear frequency-phase relation in low-frequency region (Qiao et al., 1994), one can derive the group delay to be 2.8 ms when considering a measured angle of 10 ◦ at 10 Hz.
As those measurements were conducted for only one distance, we extrapolate the delay for the remaining distances using a rectiﬁed linear model:
[Equation]
where d 0 = 2 . 8 ms is the delay at distance l 0 = 10 cm.
The delay depends not only on the distance, but also on other parameters such as the electrode-to-skin contact, the humidity of the skin, etc.
To this end, we evaluate the propagation of the attack with different characteristic delay parameters λ d ∈ [0 . 1 , 0 .
563] s/m. With λ d = 0 . 1 we cover the cases where very little delay happens, while the largest considered λ d = 0 .
563 s/m yields a maximum delay of 0.1 s at the farthest electrode T10, which is in alignment with the observed EEG measurements (Merlet et al., 2013; Sazgar & Young, 2019). Similarly to λ m , we showcase also for an intermediate value of λ d = 0 . 3 which corresponds to a delay of 0.053 ms at T10.
input : X train , EEG training samples; λ m , λ d , spatial propagation parameters; β , weight of derivative loss term; (cid:15) , maximum perturbation amplitude; G , number of PGD iterations; E , number of epochs output : v , adversarial perturbation 1 v ← U ( − (cid:15), (cid:15) ) ∈ R N s ; // Initialisation 2 for e ← 1 to E do Shufﬂe X train ; for each batch B ∈ X train do α ← (cid:15) 2 ; for g ← 1 to G do
V ← H ( v , λ m , λ d ) ; //
Spatial propagation p ← f
)) ; // Model pass with perturbation v v
|
; // Update w/derivative v ← clip (cid:15)
; // PGD projection α
(cid:15)
G · g + ; // Learning rate update end end 14 end 3. A TTACK D ESIGN We present practical DoS attacks in BCIs that respect domain constraints such as maximum amplitude, spectral distribution, physiological plausibility, and the spatial propagation of the perturbation.
To this end, we formulate a general objective function that contains the spatial propagation, the preprocessing step, and the ﬁrst order derivative loss term: [Equation]
[Equation]
where l ( · , · ) is the negative log-likelihood loss deﬁned in equation 6 and β =1e-6 is a scalar that weights the contribution of the derivative loss term.
We compare different attack scenarios:
Instance-based attacks.
A perturbation is computed using either FGSM or PGD based on the knowledge of the currently attacked EEG signal X .
FGSM computes the perturbation as stated in equation 5, where the (cid:15) deﬁnes the perturbation amplitude which is varied between 1–50 mV.
Alternatively, we compute the perturbation using PGD with G =10 iterations, where each iteration consists of a gradient-based update of the perturbation and a projection to the L ∞ ball with radius (cid:15) (see equation 7).
The update rate α is initialized with (cid:15)/ 2 and linearly decreased with each iteration, reaching a ﬁnal value of 0.1 mV at iteration 10.
The PGD computation is restarted 5 times with different initial perturbations, which are drawn from a uniform distribution within the range [ − (cid:15), + (cid:15) ] .
Universal attacks.
A universal perturbation is computed for all the samples in the training data.
We optimize the UAP objective function
[Equation] [Equation]
using batched PGD.
We pass a batch of 16 samples together with the current perturbation through the preprocessing and classiﬁer, compute the loss function, and update the perturbation based on the negative gradient with consecutive projection to the L ∞ ball with radius (cid:15) .
This step is repeated G =10 times before processing the next batch.
Overall, the UAP is learned for E =10 epochs.
Propagation model.
We distinguish between three use cases of spatial propagation model, where in all cases either an instance-speciﬁc attack or a universal attack can be computed: Case 1) Ignore the propagation model: a multi-channel perturbation V is computed, which attacks each channel individually, replacing the terms H ( v , λ m , λ d ) by V and v (cid:48) by V (cid:48) in equation 15. Case 2) Consider [Figure] [Figure]
[Figure] [Table] the propagation model: a single-channel perturbation v is computed and tested with a speciﬁc propagation conﬁguration λ m and λ d . Case 3) Consider a use-case where the attacker does not know the spatial propagation model and computes the same perturbation v for all channels.
The actual propagation model is applied during testing to model the real-world signal propagation. End-to-end algorithm.
We illustrate the algorithmic procedure for designing a physiologically plausible UAP in Algorithm 1.
Analogously, the proposed methods of derivative loss term and model propagation are applied with PGD.
The hyperparameters α , β , the number of PGD iterations and the restarts, the batch size and the number of epochs in UAP are determined based on a cross-validated grid search on the training set.
E XPERIMENTS AND RESULTS
We evaluate our methods on the Physionet EEG Motor Movement/Imagery Dataset (Goldberger et al., 2000; Sch) tackling inter-subject challenges, and generalize to subject-speciﬁc inter-session dataset
IV-2a of BCI Competition (Brunner et al., 2008) (See Appendix C).
Dataset.
The Physionet dataset contains valid EEG recordings of 105 subjects (Dose et al., 2018) and is publicly available under Open Data Commons Attribution License v1.0.
We use the MI recordings that contain tasks of the imagination of left against right ﬁst for 3 s. The EEG trials were recorded with N ch
=64 channels sampled at F s =160 Hz, yielding N s =3 · 160=480 samples per trial.
Additional baseline runs provide resting-state data, where the subjects did not perform any tasks while having eyes open.
Overall, we get a total of 6615 trials with N cl =3 balanced classes “left”, “right”, and “rest.”
Training and validation.
We train and validate both the classiﬁcation models and the generated adversarial examples with a 5-fold cross-validation, splitting the dataset into 84 subjects used for training and 21 subjects used for validation to effectively test the model on inter-subject variability.
Similar to Wang et al.
(2020), which achieved SoA performance on this dataset, the baseline model is trained for 100 epochs using Adam with β 1 =0.9, β 2 =0.999, and batch size of 16.
The learning rate is 0.01 and decreased by a factor of 10 at epochs 20 and 50, achieving an average accuracy of 74.78%. [Figure] [Figure]
[Figure] [Figure] [Figure] An FFT band-pass ﬁlter h bp with a customary passband of 0.1–40 Hz (Lawhern et al., 2018) is used as preprocessing step in both baseline and attack experiments.
To determine the ASR, we compute the ratio between the successfully fooled trials, i.e., trials now classiﬁed as “rest”, and the total number of attacked trials, where we only consider the ones initially correctly classiﬁed as “left”/“right”.
Physiologically plausible attacks.
We ﬁrst analyze the instance-based attacks without considering the propagation model (Case 1), depicted in Figure 1.
We compare our methods against random noise with amplitude (cid:15) as in (Zhang & Wu, 2019), FGSM that is the same as in (Zhang & Wu, 2019) with targeted scenario, and a UAP designed speciﬁcally for EEG (Liu et al., 2021).
For both FGSM and PGD, the ASR increases together with the maximum amplitude (cid:15) of the perturbation. They always outperform the random noise, with PGD performing slightly better than FGSM.
They reach the maximum ASR of 99.97% with 10mV. The post-attack classiﬁcation accuracy drops from 74.78% to 48% for a perturbation amplitude of 2 mV and to 33% for 10 mV and higher amplitudes. Figure 2a shows the signals of a successful attack using PGD.
The adversarial perturbation has a squarewave form which negatively affects the natural shape of the EEG signal.
By adding the proposed derivative term, the square-wave artifacts are signiﬁcantly reduced (2b), making the perturbation more physiologically plausible.
When comparing the power spectral density of the original and attacked signals, the attacked signal designed without derivative presents large components in low frequencies, making it more easily detectable.
Whereas the attack with derivative loss better resembles the power spectral density of the original signal (see Appendix D).
Moreover, the introduction of the derivative term does not degrade the ASR (Figure 1).
The quantitative measures between the original and the adversarial samples in Table 1 demonstrate that our proposed method with derivative term generates adversarial samples that are more similar to the original EEG, allowing them to remain imperceptible even with high (cid:15) (Appendix D).
We reproduce the attacks using a Gaussian kernel as in (Han et al., 2020).
After tuning the kernel size and variance of the Gaussian kernel, the method could not improve the plausibility metrics.
The inferior performance of the Gaussian kernel could stem from the different nature of the signal: it was originally designed for ECGs which have a pseudo-periodic structure.
We extend the application of the derivative term to the UAP attack, while still not considering the propagation model (Case 1).
Figure 1 shows a comparison in performance for different values of (cid:15) .
The saturation in ASR is reached with higher (cid:15) , i.e., 99.94% with 50mV. This is expected since the
UAP is a more difﬁcult attack where a single set of perturbations per EEG channel is generated for all the test samples.
Likewise in PGD, the ASR does not drop with the addition of the derivative term.
We reproduce the UAP proposed by (Liu et al., 2021).
Our UAP consistently reaches higher ASR.
Spatial Propagation. Finally, we introduce the spatial constraints in the signal propagation over the scalp (Case 2).
We consider 9 different scenarios by combining 3 realistic attenuation conﬁgurations λ m ∈ { 1 , 5 , 15 } with 3 delay conﬁgurations
λ d ∈ { 0 . 1 , 0 . 3 , 0 . 563 } , which capture the range described in Section 3.2.
For evaluating the highest achievable attack efﬁciency, we test a scenario where the attacker is assumed to know the propagation model: the adversarial perturbation is generated and evaluated on ﬁxed spatial parameters λ m and λ d , shown in Figure 4, where the ASR reaches up to 69.2% with PGD and 45.6% with UAP at 50mV. Figure 3 depicts an example of a successful attack with the highest perturbation amplitude.
The introduction of the spatial constraints makes the attack problem harder yielding seldom square distortions.
However, the resulting EEG signals still resemble physiological random processes typical of EEGs.
Next, we ablate the spatial constraints during generation and test the resulting perturbations on the 9 above-mentioned scenarios (Case 3).
The ASR drops signiﬁcantly, especially for λ m =
and λ m =15 where the attenuation of the perturbation over the scalp is greater (see Figure 7), and with the global UAP attack, where the attacker does not have access to the attacked EEG signals.
Our spatial propagation models allows us to identify the vulnerability of the individual EEG channels.
Figure 5 shows the ASR when initiating an attack from a speciﬁc channel (T9, T10, etc.) and propagating it to the rest of the head.
In the case with the greatest attenuation ( λ m =15) we ﬁnd the maximum ASR at the electrode Cz between the regions of the electrodes C3 and C4, which are the most relevant ones for MI of the left and right hand tasks (Pfurtscheller & Lopes da Silva, 1999). We compute the pre- and post-attack confusion matrices for attacks from T9 and T10 (see Appendix E).
When the attack propagates from the left side (T9), more samples with ground-truth label “right” can be fooled to “rest”, while the attacks from the right side (T10) are more effective “left” labels.
Overall, our methods successfully generates perturbations resembling natural noise in EEGs, that can be added at the source of the signal acquisition and are propagated over the scalp, creating attacked signals that are physiologically plausible. Similar results have been observed on the BCI Competition
IV-2a dataset, shown in Appendix C.
C ONCLUSION
With the incentive of improving security in BCIs, in this work, we demonstrated that DoS attacks are feasible and effective despite physical domain constraints.
Experimental results reveal potential risks of realistic attacks on smart wearable BCIs and incentivize the need for future development of defense mechanisms while designing deep learning models to be embedded in smart wearable BCIs.
Our detailed analysis on each EEG channel shows that special attention has to be paid, combined with the ﬁndings in neuroscience, to the brain regions that are found responsible for a speciﬁc task.
In future work, the proposed attacks can cover uncertainty in the propagation model and the timing of the MI activity.
Moreover, hardware implementations of such attacks can be created to evaluate the proposed methods in real-world, with the ultimate goal of developing effective countermeasures.
E THICS S TATEMENT
The sore goal of this work is to raise awareness of potential adversarial attacks in BCIs and incentivize the development of coutermeasures, especially in the current moment when the BCIs are facing an increasing growth in applications of everyday life.
The active development of smart wearable BCIs is introducing a paradigm shift where the processing algorithms are embedded near the data acquisition.
While this improves the system security to a certain extend, with this work we have shown that it is not the only and ultimate way to a safe and reliable BCI, since we have shown that BCI systems are vulnerable also to attacks at the signals’ source.
We hope that our work sheds light on the fact that practical BCI systems are vulnerable, despite the physical constraints and motivates the design and development of more reliable and robust BCI systems.
R EPRODUCIBLITY
A link to a anonymous downloadable source code of this work is submitted as supplementary materials. A A TTACK AT THE SOURCE OF SIGNAL ACQUISITION Fig.
6 illustrates the new attack scenario where the perturbation is delivered to the human scalp and propagates to the sensing electrodes at the source of the signal acquisition.
B S PATIAL PROPAGATION MODELS
Fig.
7 illustrates the magnitude of signal propagation using different propagation parameters λ m = { 1 , 5 , 15 } .
A large λ m represents cases with large attenuation and limited propagation (e.g., attack over the air) and a small λ m covers cases with lower attenuation where the perturbation can propagate farther (e.g., a smart glass).
C E XPERIMENTS ON BCI C OMPETITION IV-2 A
Dataset.
The IV-2a dataset of the BCI Competition contains recordings from nine different subjects and distinguishes between four classes of imagined movements: left and right hand, both feet, and the tongue.
22 different EEG channels were recorded, sampled at 250 Hz.
The data was pre-processed with a bandpass ﬁlter between 0.1 and 40 Hz.
Each subject completed two recording session on two different days, where the ﬁrst session is used for training and the second for testing as per the rules of the competition.
Each session contains 288 trials.
Training and validation.
We train a separate baseline model per subject using Adam optimizer with β 1 =0.9 and β 2 =0.999, a batch size of 32, and 500 epochs.
The learning rate is 0.001 achieving an average accuracy of 71.79%.
This dataset does not contain the rest class.
We choose to design an attack that aims to fool the classiﬁer to always predict “tongue.”
Moreover, we apply a maximum perturbation amplitude of (cid:15) ∈ [0 . 01 , 10] mV due to the lower signal amplitude encountered in this dataset.
Results.
Fig.
8 compares the ASR of different attacks without considering the propagation model (Case 1).
Generally, a minimal perturbation amplitude of 1 mV and 2 mV sufﬁces to achieve 100%
ASR with PGD and UAP, respectively.
The addition of the derivative loss term does not give any performance degradation in terms of the ASR.
The average post-attack classiﬁcation accuracy drops
[Figure]
(cid:15)
[Figure]
PGD-w/Der.
PGD-w/oDer.
FGSM (Zhang & Wu, 2019)
UAP-w/Der.
UAP-w/oDer.
Random noise
UAP (Liu et al., 2021) from 71.79% to 50% for a perturbation amplitude of 0.15 mV and 24.7% for 0.6 mV and higher amplitudes, when PGD with derivative is used.
Fig.
9 shows the ASR for different propagation parameters ( λ m and λ d ) and maximum perturbation amplitudes (cid:15) .
When considering the head model during the design of the attack (Case 2, w/HM), both
PGD and UAP reach signiﬁcantly higher ASR compared to attacks designed without the consideration of the head model (Case 3, w/oHM).
D P LAUSIBILITY OF ATTACKS
This section provides power spectral density plots of original signals and attacked signals with and without the derivative loss term, shown in Figure 10.
The power spectral density is determined by computing the magnitude squared Fast Fourier Transform of the signals that were illustrated in
Figure 2.
The attack designed with the derivative loss term has a similar distribution as the original signal, where as the attack without derivative shows large contributions in the low frequency domain ( < 5 Hz), which were not present in the original signal.
These low-frequency components stem from the square-wave shaped attack and can be used as a way to detect the attack; hence, this attack cannot be considered imperceptible.
Moreover, Figure 13 shows the attacks with and without derivative loss term with increasing maximum amplitude (cid:15) .
We can see that for low amplitudes (1 mV and 5 mV) the generated attacks with and without derivative still look like EEGs.
At 10 mV, the attack generated without derivative presents minor square-wave artifacts, which could be still imperceptible to a non-expert.
With 25 mV and 50 mV, the ones generated without derivative have strong and perceptible square-wave displacements, while the ones generated with our proposed method can still be mistaken as real EEG signals.
While with the instance-based attacks, it is not necessary to have more than 10 mV to get a very high ASR (see Figure 1), with the universal attacks and physical constraints, the ASR increases with increasing perturbation amplitude (see Figure 4).
The same observations can be drawn from the plausibility metrics, which have been proposed for the ﬁrst time in this paper to assess quantitatively the EEG attacks.
For example, looking at the cosine similarity ( γ ) in Table 1, without the derivative loss term, γ drops to 97.99% with (cid:15) =
5mV, whereas, with the derivative, γ drops to about the same value of 97.47% with (cid:15) =
10 mV, yielding an increase in ASR from 85% (5 mV) to 99% (10 mV) shown in Figure 1 with PGD.
[Figure]
[Figure]
[Figure]
E C LASSIFICATION C ONFUSION M ATRICES
We analyze the confusion matrices before and after the proposed attack.
Fig.
11 shows the confusion matrix of EEGNet on the Physionet dataset before the attack, where all classes can be classiﬁed with similar accuracy (72.8%–73.5%).
Fig.
12 shows the confusion matrices for three different propagation parameters ( λ m ∈ { 1 , 5 , 15 } ) and two attack positions (T9 and T10) which correspond to the left and right side of the head.
When considering the attacks from the left side, shown in Fig.
12a–12c, more samples with ground-truth label “right” can be fooled to “rest”.
This is particularly articulated in largely attenuated propagation model ( λ m =15).
In a similar vein, attacks coming from the right side of the head (T10) are more effective on data with ground-truth label “left” (Fig. 12d–12f).
[Figure]
[Figure]
l
[Figure]
b e l
[Figure]
T r u e l a b e l
[Figure]
[Figure]