<figure><img src="image_1.png" /></figure>

HAL
open science

# A Deep Learning Approach Based on CSP for EEG Analysis

Wenchao Huang, Jinchuang Zhao, Wenli Fu

► To cite this version:

Wenchao Huang, Jinchuang Zhao, Wenli Fu. A Deep Learning Approach Based on CSP for EEG Analysis. 10th International Conference on Intelligent Information Processing (IIP), Oct 2018, Nanning, China. pp.62-70, 10.1007/978-3-030-00828-4_7. hal-02197785

HAL Id: hal-02197785

https://inria.hal.science/hal-02197785

Submitted on 30 Jul 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Deep Learning Approach Based on CSP for EEG Analysis

Wenchao Huang*, Jinchuang Zhao*, Wenli Fu*

* College of Computer and Electronics Information, Guangxi University, Nanning, 530004, China

wenchao_h@outlook.com
zhaojch@gxu.edu.cn
fuwenli@gxu.edu.cn

**ABSTRACT.** Deep learning approaches have been used successfully in computer vision, natural language processing and speech processing. However, the number of studies that employ deep learning on brain-computer interface (BCI) based on electroencephalography (EEG) is very limited. In this paper, we present a deep learning approach for motor imagery (MI) EEG signal classification. We perform spatial projection using common spatial pattern (CSP) for the EEG signal and then temporal projection is applied to the spatially filtered signal. The signal is next fed to a single-layer neural network for classification. We apply backpropagation (BP) algorithm to fine-tune the parameters of the approach. The effectiveness of the proposed approach has been evaluated using datasets of BCI competition III and BCI competition IV.

**KEYWORDS:** brain-computer interface (BCI), electroencephalography (EEG), motor imagery (MI), common spatial pattern (CSP), backpropagation (BP)

## 1 Introduction

Brain-computer interface (BCI) is a communication system that is established between the human brain and computers or external devices without relying on the regular brain peripheral nerve and muscle systems [1]. BCI system acquire human brain EEG signals, extract features, classify EEG and translate EEG into machine-readable control commands. The main goal of BCI system is to strengthen the ability of disabled persons affected by a number of motor disabilities. The application of BCI in the medical field mainly includes sensory recovery, cognitive recovery, rehabilitation treatment, and brain-control wheelchairs [2]. In non-medical areas, BCI can be applied to new types of entertainment games, car driving, robot replacements, lie detectors [3], etc. In addition, in the field of aviation and military industry, BCI also has a wide range of applications.
MI-BCI is the BCI application based on MI-EEG, and it is one of the main direc-
tions of brain-computer interface research. Many successful MI-BCI relies on subjects
learning to control specific EEG rhythms that manifest as EEG potentials oscillating
at a particular frequency. The EEG rhythms related to motor imagery tasks consist of
mu (8-13 Hz) rhythm and beta (13-30 Hz) rhythm. The energy in mu band observed
in motor cortex of the brain decreases by performing an MI task [4]. This decrease is
called event related desynchronization (ERD). An MI task also causes an energy in-
crease in the beta band that is called event related synchronization (ERS) [5]. For
different MI tasks, the brain motor cortex produces discriminative ERD/ERS. Fea-
tures are extracted by analysing ERD/ERS, and then a classification algorithm is
adopted to construct a MI-BCI. Two main techniques for MI-EEG analysis are feature
extraction and classification algorithms. Several feature extraction techniques such as
power spectral density (PSD), common spatial pattern (CSP) [6-9], autoregressive
(AR) model, adoptive autoregressive (AAR) model, independent components analysis
(ICA) and wavelet transform [10,11] have been studied. Classifiers such as support
vector machine (SVM) [12], k-nearest neighbors (KNN) [13,14], random forest (RF)
[15], linear discriminant analysis (LDA) [16], etc. have been explored for classifica-
tion of MI-EEG signals.

In recent years, deep learning's revolutionary advances in audio and visual signals recognition have gained significant attentions. Some recent deep learning based EEG classification approaches have enhanced the recognition accuracy. In a study by An et al, a deep belief network (DBN) model was applied for two class MI classification and DBN was shown more successful than the SVM method [17]. Yousef et al applied convolutional neural networks (CNN) and stacked autoencoders (SAE) to classify EEG Motor Imagery signals [18,19]. Schirrmeister proposed a convolutional neural network (deep ConvNets) for end to end EEG analysis. Their study shows how to design and train ConvNets to decode task-related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG based brain mapping [20].

In this paper, we propose a framework based on CSP and backpropagation algo-
rithm for MI-EEG analysis. In order to evaluate the proposed framework, we trained
and tested with BCI competition II dataset III and BCI competition IV dataset 2a. The
remainder of this paper is organized as follows. Section 2 provides a description of
the proposed framework. Section 3 describes the experimental studies and results on
the evaluation data of the BCI competition II datasets III and BCI competition IV
datasets 2a. Finally, section 4 concludes this paper with the results.

## 2 Methods

The structure of the proposed framework is shown in Fig.1. The proposed framework consists of 4 stages. The first stage is a band-pass filter for raw EEG data. The second stage performs spatial filtering using CSP algorithm. The third stage consists of the temporal projection of the spatial filtered signal. The last stage is a single-layer neural
network that is implemented as a classification layer. The following sections explain the different stages of the proposed framework in detail.

## 2.1 Band-pass filtering

As described in section 1, there are ERS/ERD when human perform MI tasks. In order to extract the EEG signals in mu band and beta band, the raw EEG data is first filtered by a band-pass filter that covers 8-30 Hz.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Diagram of the proposed framework</figcaption></figure>

## 2.2 CSP algorithm

The CSP algorithm is highly successful in calculating spatial filters for detecting ERD/ERS. The main idea is to use a linear transform to project the multi-channel EEG data into low-dimensional spatial subspace with a projection matrix, of which each row consists of weights for channels [21]. This transformation can maximize the variance of two-class signal matrices. The CSP algorithm perform spatial filtering using

$$Z_i = W_{csp}^T E_i \quad (1)$$

where $E_i$ is an $n \times t$ matrix representing the raw EEG measurement data of the i th trial, n is the number of channels, t is the number of measurement samples per channel. $W_{csp}$ denotes the CSP projection matrix, T denotes transpose operator. Z denotes the spatially filtered signal. The CSP matrix can be computed by solving the eigenvalue decomposition problem

$$S_1 W_{csp} = (S_1 + S_2) W_{csp} D \quad (2)$$
where $S_1$ and $S_2$ are estimates of the covariance matrices of the band-pass filtered EEG measurements of the respective motor imagery action, $D$ is the diagonal matrix that contains the eigenvalues of $S_1$.

However, only a small number $m$ of the spatial filtered signal is generally used as features. We perform another transform to get the spatially filtered signal. It is given by

$$Z_i = \overline{W}_{csp}^T E_i \quad (3)$$

where $\overline{W}_{csp}$ represents the first $m$ and the last $m$ columns of $W_{csp}$, the spatial filtered signal $Z$ is a $2m \times t$ matrix.

## 2.3 Joint optimization using backpropagation

Mathematically, the 3th stage and the 4th stage can be described as follows. Given the spatial filtered signal $Z$, the temporal projection matrix $V$, the classifier weights $W_c$ and bias $b$, we have

$$S = W_c^T \log(Z^2 V) + b \quad (4)$$

where $S$ denotes the input that is a vector containing class scores and will be plugged into an activation function. The output of the framework is given by

$$y = f(S) \quad (5)$$

where $y$ is a vector of probability for the classes and $f(\cdot)$ is the activation function that is the softmax function. The softmax function (softmax regression) is a generalization of logistic regression to the case where we want to handle multiple classes. The softmax output is given by

$$y_k = \frac{e^{S_k}}{\sum_j e^{S_j}} \quad (6)$$

where $S_k$ is an element for a certain class $k$ in all $j$ classes. The cost function is the cross-entropy cost function, which is

$$E = -\log(p_{y_k}) \quad (7)$$
The free parameters of the 3th stage and the 4th stage are the temporal projection matrix $V$, the classifier weights $W_c$ and the bias $b$. The parameters are learned by using back-propagation algorithm. In this method, the labeled training set is fed to the network and the error $E$ (cost function) is computed. Then the model parameter can be updated using gradient descent method. The error can be minimized by changing network parameters as shown as follows

$$V = V - \eta \frac{\partial E}{\partial V} \qquad (8)$$

$$W_c = W_c - \eta \frac{\partial E}{\partial W_c} \qquad (9)$$

$$b = b - \eta \frac{\partial E}{\partial b} \qquad (10)$$

where $\eta$ denotes the learning rate of the algorithm. $V$ is initialized to a matrix of all ones, $W_c$ is randomly initialized from a Gaussian distribution. Finally, the trained framework is used for classification of the new samples in the test set.

# 3 Experiments with BCI competition datasets

In this section, we apply the proposed framework to the BCI competition datasets, and the results of the proposed approach on these datasets are presented.

## 3.1 BCI competition II, dataset III

The first dataset is dataset III from BCI competition II. The dataset includes MI task experiments for right hand and left hand movements. EEG signals are recorded at C3, Cz and C4 channels. During acquisition of the EEG signals, at $t = 2s$ an acoustic stimulus indicating the beginning of the trial was used and a cross '+' was displayed for 1s. Then, at $t = 3s$, the subject was asked to perform the related MI task by displaying an arrow (left or right). There were 280 trials in the dataset, 140 trials for training and another 140 trials for test.

For each EEG trial, we extracted the time interval between 0.5s to 3.5s after the cue was displayed. To evaluate our method on the dataset, we used the network shown in Fig.1 and described in section 2, which consists of a band-pass filter, CSP spatial projection, temporal projection and a single-layer neural network. The framework was trained with 140 trials in the training set and tested on 140 trials in the test set. Stochastic gradient descent (SGD) was used to update the parameters and minimize the error $E$. For each training epoch, the mini-batch was set to be 1/2 of the training data randomly.
The results of BCI competition II dataset III are shown in table 1. When learning rate η was fixed to be 0.03, we obtained the best results. The accuracy performance of our method was obtained as 90.0%. The accuracy of the winner algorithm of the competition is 89.3%. We compared our results to some study (CNN and CNN-SAE) where deep learning network is used [18,19]. The results of CNN and CNN-SAE are 90.0% and 89.3% respectively. The CSP-LR method is the normal method without using deep learning methods for MI-EEG analysis, which use CSP for feature extraction and logistic regression algorithm for classification. We also compared our results to the CSP-LR method. The CSP-LR method got an accuracy of 88.9%. The kappa values of those methods are also in the table 1. The kappa value is a measure for classification performance removing the effect of accuracy of random classification. Kappa is calculated as

$$kappa = \frac{acc - 1/N}{1 - 1/N} \qquad (11)$$

where N denotes the number of classes. In this dataset N is 2. As described in table 1, the accuracy of the proposed method is equal to CNN-SAE, and is better than the winner of competition, CNN method and CSP-LR.

<table><caption>**Table 1.** The accuracy (%) and kappa results of BCI competition II dataset III</caption><thead><tr><th>Method</th><th>Proposed method</th><th>Winner</th><th>CNN</th><th>CNN-SAE</th><th>CSP-LR</th></tr></thead><tbody><tr><td>Accuracy %</td><td>90.0</td><td>89.3</td><td>89.3</td><td>90.0</td><td>88.6</td></tr><tr><td>Kappa</td><td>0.800</td><td>0.786</td><td>0.786</td><td>0.800</td><td>0.772</td></tr></tbody></table>

## 3.2 BCI competition IV, dataset 2a

BCI competition IV dataset 2a comprised 4 classes of motor imagery EEG measurements from 9 subjects, namely, left hand, right hand, feet, and tongue. Two sessions, one for training and the other for evaluation, were recorded from each subject. Each session comprised 288 trials of data recorded with 22 EEG channels and 3 monopolar electrooculogram (EOG) channels. Each trial starts with a short acoustic stimulus and a fixation cross. Then, at t = 3s an arrow indicates the MI task. The arrow is displayed for 1.25s. Then the subjects have 4 s to imagine the task.

There are 4 classes in dataset 2a that is different from BCI competition II dataset III. When performing the spatial projection, we use OVR-CSP [22] to get the spatial filtered signals. The architecture of framework described in section 2 can be changed as Fig.2. The number of temporal projection matrices needed to be fine-tuned increase to 4. The 4 temporal projection matrices are initialized to matrices of all ones and will be updated together using back propagation algorithm.
<figure><img src="image_4.png" /><figcaption>Fig. 2. Diagram of the proposed framework based on OVR-CSP</figcaption></figure>

For each EEG trial, we extracted the time interval between 1s to 5s after the cue was displayed. The framework was trained with training data and tested on test data. SGD was used to update the parameters. The Mini-batch was set to be 1/4 of the training data randomly.

The accuracy results of the proposed method and CSP-LR are shown in table 2. Kappa values of the proposed method and CSP-LR are compared to FBCSP (winner algorithm of competition) [9] in table 3. With the deep learning method, the proposed method obtained higher accuracies and better kappa values than CSP-LR method for all subjects. For subject 1, subject 2, subject 3, subject 8 and subject 9, our approach has better kappa values than FBCSP. For subject 4, subject 5, subject 6 and subject 7, our approach has worse kappa values. The average kappa value of our approach is 0.583, which is higher than FBCSP (0.569).

<table><caption>Table 2. The accuracy (%) results for the proposed method and CSP-LR</caption><thead><tr><th rowspan="2">Subject</th><th colspan="2">Accuracy %</th></tr><tr><th>Proposed method</th><th>CSP-LR</th></tr></thead><tbody><tr><td>1</td><td>78.1</td><td>73.3</td></tr><tr><td>2</td><td>58.4</td><td>50.7</td></tr><tr><td>3</td><td>81.2</td><td>59.0</td></tr><tr><td>4</td><td>60.1</td><td>45.8</td></tr><tr><td>5</td><td>53.0</td><td>47.2</td></tr><tr><td>6</td><td>42.3</td><td>37.2</td></tr><tr><td>7</td><td>80.0</td><td>40.3</td></tr><tr><td>8</td><td>84.7</td><td>64.3</td></tr><tr><td>9</td><td>81.2</td><td>64.9</td></tr><tr><td>Average</td><td>68.7</td><td>53.6</td></tr></tbody></table>
<table><caption>**Table 3.** The kappa results for the proposed method, FBCSP and CSP-LR</caption><thead><tr><th rowspan="2">Subject</th><th colspan="3">Kappa</th></tr><tr><th>Proposed method</th><th>FBCSP</th><th>CSP-LR</th></tr></thead><tbody><tr><td>1</td><td>0.708</td><td>0.676</td><td>0.644</td></tr><tr><td>2</td><td>0.445</td><td>0.417</td><td>0.343</td></tr><tr><td>3</td><td>0.749</td><td>0.745</td><td>0.453</td></tr><tr><td>4</td><td>0.468</td><td>0.481</td><td>0.277</td></tr><tr><td>5</td><td>0.373</td><td>0.398</td><td>0.296</td></tr><tr><td>6</td><td>0.231</td><td>0.273</td><td>0.163</td></tr><tr><td>7</td><td>0.733</td><td>0.773</td><td>0.204</td></tr><tr><td>8</td><td>0.796</td><td>0.755</td><td>0.524</td></tr><tr><td>9</td><td>0.749</td><td>0.606</td><td>0.532</td></tr><tr><td>Average</td><td>0.583</td><td>0.569</td><td>0.382</td></tr></tbody></table>

# 4 Conclusion

In this study, we propose a deep learning approach for MI-EEG analysis. We designed a framework by combining backpropagation algorithm and CSP. We use a band-pass filter for processing the raw EEG data. And CSP algorithm is used for spatial filtering. Then we perform temporal projection and obtain the features which are fed to a single-layer neural network for classification. The free parameters of the framework can be fine-tuned by applying the backpropagation algorithm for the best classification accuracy.

We apply the proposed framework to the BCI competition datasets. Dataset III from BCI competition II and dataset 2a from BCI competition IV were used in this study. The accuracy result of our method on dataset III is 90.0% that is equal to CNN-SAE method. And it is higher than the winner algorithm of competition II and CNN method. On dataset 2a from BCI competition IV, our method obtained average kappa value of 0.583 which is better than FBCSP. Furthermore, on both datasets our method outperformed CSP-LR method that is not using deep learning methods.

Though deep learning methods have achieved great development in computer vision, natural language processing and speech processing, its application in EEG-based BCI is still rare. Our results show that deep learning methods have great potential to be a powerful tool for EEG analysis and EEG-BCI. We believe that the number of further BCI studies using deep learning methods will increase rapidly.

# 5 References

1. Lotte, Fabien, et al. "A review of classification algorithms for EEG-based brain-computer interfaces." *Journal of neural engineering* 4.2 (2007): R1.
2. Graimann, Bernhard, Brendan Allison, and Gert Pfurtscheller. "Brain–computer interfaces: A gentle introduction." *Brain-Computer Interfaces*. Springer, Berlin, Heidelberg, 2009. 1-27.

3. Rao, Rajesh PN. *Brain-computer interfacing: an introduction*. Cambridge University Press, 2013.

4. Pfurtscheller, Gert, and Christa Neuper. "Motor imagery and direct brain-computer communication." *Proceedings of the IEEE* 89.7 (2001): 1123-1134.

5. Pfurtscheller, Gert, and FH Lopes Da Silva. "Event-related EEG/MEG synchronization and desynchronization: basic principles." *Clinical neurophysiology* 110.11 (1999): 1842-1857.

6. Ramoser, Herbert, Johannes Muller-Gerking, and Gert Pfurtscheller. "Optimal spatial filtering of single trial EEG during imagined hand movement." *IEEE transactions on rehabilitation engineering* 8.4 (2000): 441-446.

7. Novi, Quadrianto, et al. "Sub-band common spatial pattern (SBCSP) for brain-computer interface." *Neural Engineering, 2007. CNE'07. 3rd International IEEE/EMBS Conference on*. IEEE, 2007.

8. Ang, Kai Keng, et al. "Filter bank common spatial pattern (FBCSP) in brain-computer interface." *Neural Networks*, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence). *IEEE International Joint Conference on*. IEEE, 2008.

9. Ang, Kai Keng, et al. "Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b." *Frontiers in neuroscience* 6 (2012): 39.

10. Yang, Bang-hua, et al. "Feature extraction for EEG-based brain–computer interfaces by wavelet packet best basis decomposition." *Journal of neural engineering* 3.4 (2006): 251.

11. Hsu, Wei-Yen, et al. "Wavelet-based fractal features with active segment selection: Application to single-trial EEG data." *Journal of neuroscience methods* 163.1 (2007): 145-160.

12. Li, Xiaouo, et al. "Classification of EEG signals using a multiple kernel learning support vector machine." *Sensors* 14.7 (2014): 12784-12802.

13. Brown, Lindsay, Bernard Grundlehner, and Julien Penders. "Towards wireless emotional valence detection from EEG." *Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE*. IEEE, 2011.

14. Xu, Haiyan, and Konstantinos N. Plataniotis. "Affect recognition using EEG signal." *Multimedia Signal Processing (MMSP), 2012 IEEE 14th International Workshop on*. IEEE, 2012.

15. Akram, Faraz, Hee-Sok Han, and Tae-Seong Kim. "A P300-Based Word Typing Brain Computer Interface System Using a Smart Dictionary and Random Forest Classifier." *The Eighth International Multi-Conference on Computing in the Global Information Technology*. 2013.

16. Subasi, Abdulhamit, and M. Ismail Gursoy. "EEG signal classification using PCA, ICA, LDA and support vector machines." *Expert Systems with Applications* 37.12 (2010): 8659-8666.

17. An, Xiu, et al. "A deep learning method for classification of EEG data based on motor imagery." *International Conference on Intelligent Computing*. Springer, Cham, 2014.

18. Yang, Huijuan, et al. "On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification." *Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE*. IEEE, 2015.

19. Tabar, Yousef Rezaei, and Ugur Halici. "A novel deep learning approach for classification of EEG motor imagery signals." *Journal of neural engineering* 14.1 (2016): 016003.
20. Schirrmeister, Robin Tibor, et al. "Deep learning with convolutional neural networks for EEG decoding and visualization." *Human brain mapping* 38.11 (2017): 5391-5420.

21. Jamaloo, Fatemeh, and M. Mikaeili. "Discriminative Common Spatial Pattern Sub-bands Weighting Based on Distinction Sensitive Learning Vector Quantization Method in Motor Imagery Based Brain-computer Interface." *Journal of Medical Signals & Sensors* 5.3(2015):156-161.

22. Wu, W., X. Gao, and S. Gao. "One-Versus-the-Rest(OVR) Algorithm: An Extension of Common Spatial Patterns(CSP) Algorithm to Multi-class Case." *Engineering in Medicine and Biology Society, 2005. Ieee-Embs 2005. International Conference of the* IEEE, 2006:2387-2390.