arXiv:1709.08820v1 [cs.HC] 26 Sep 2017

Converting Your Thoughts to Texts: Enabling Brain
Typing via Deep Feature Learning of EEG Signals
Xiang Zhang

Lina Yao

Quan Z. Sheng

School of Computer Science & Engineering
University of New South Wales
Sydney, Australia
xiang.zhang3@student.unsw.edu.au

School of Computer Science & Engineering
University of New South Wales
Sydney, Australia
lina.yao@unsw.edu.au

Department of Computing
Macquarie University
Sydney, Australia
michael.sheng@mq.edu.au

Salil S. Kanhere

Tao Gu

Dalin Zhang

School of Computer Science & Engineering
University of New South Wales
Sydney, Australia
salilk@cse.unsw.edu.au

School of Science
RMIT University
Melbourne, Australia
tao.gu@rmit.edu.au

School of Computer Science & Engineering
University of New South Wales
Sydney, Australia
dalin.zhang@student.unsw.edu.au

Abstract—An electroencephalography (EEG) based Brain
Computer Interface (BCI) enables people to communicate with
the outside world by interpreting the EEG signals of their brains
to interact with devices such as wheelchairs and intelligent robots.
More specifically, motor imagery EEG (MI-EEG), which reflects
a subject’s active intent, is attracting increasing attention for a
variety of BCI applications. Accurate classification of MI-EEG
signals while essential for effective operation of BCI systems, is
challenging due to the significant noise inherent in the signals
and the lack of informative correlation between the signals
and brain activities. In this paper, we propose a novel deep
neural network based learning framework that affords perceptive
insights into the relationship between the MI-EEG data and
brain activities. We design a joint convolutional recurrent neural
network that simultaneously learns robust high-level feature
presentations through low-dimensional dense embeddings from
raw MI-EEG signals. We also employ an Autoencoder layer to
eliminate various artifacts such as background activities. The
proposed approach has been evaluated extensively on a largescale public MI-EEG dataset and a limited but easy-to-deploy
dataset collected in our lab. The results show that our approach
outperforms a series of baselines and the competitive state-of-theart methods, yielding a classification accuracy of 95.53%. The
applicability of our proposed approach is further demonstrated
with a practical BCI system for typing.
Index Terms—EEG, deep learning, brain typing, BCI

I. INTRODUCTION
Brain-computer interface (BCI) systems have been widely
studied for various real-world applications from mindcontrolled service robots in the healthcare domain [1] to
enriched video gaming in the entertainment industry [2]. As
an important pathway between human brains and the outside
world [3], BCI systems allow people to communicate or
interact with external devices such as wheelchairs or service
robots, through their brain signals. Among the different types
of brain signals, motor imagery Electroencephalography (MIEEG) is especially popular and has demonstrated promising
potential in discerning different brain activities in BCI systems. Motor imagery is a mental process where a subject

imagines performing a certain action such as closing eyes or
moving feet. Basically, EEG1 is a method to analyze brain
activities by measuring the voltage fluctuations of ionic current
within the neurons of brains. In practice, electrodes are usually
placed on the scalp for the measurement in a non-invasive and
non-stationary way [4].
One of the most promising and widely discussed application
of EEG-based BCI is to enable people to type via direct brain
control [5]. In this paper, we aim at enabling a brain typing
system by enhancing the decoding accuracy of EEG signals for
wider range of brain activities (e.g., multi-class scenario). We
envision a real-world implementation of such a system which
can interpret the user’s thoughts to infer typing commands in
real-time. Motor disabled people would benefit greatly from
such a system to express their thoughts and communicate with
the outer world.
However, EEG signals fluctuate rapidly and are subject to
various sources of noise including environmental noise such
as lighting and electronic equipment. Thus, the key issue
concerning an EEG-based BCI system is to accurately interpret
EEG signals so as to accurately understand the user’s intent.
More specifically, the design of a practical and effective BCIsystem is faced with the following major challenges. First,
EEG signals usually have very low signal-to-noise ratio [6].
As a result, EEG signals inherently lack sufficient spatial
resolution and insight on activities of deep brain structures.
Second, data pre-processing, parameter selection (e.g., filter
type, pass band, segment window, and overlapping), and
feature engineering (e.g., feature selection and extraction both
in time domain and frequency domain) are all time-consuming
and highly dependent on human expertise in the domain.
Third, the state-of-the-art approaches can achieve an accuracy
of at most 70∼85%, which though impressive is not sufficient
for widespread adoption of this technology. Fourth, existing
1 In

this paper, we will use the terms EEG and MI-EEG interchangeably.

research mainly focuses on discerning EEG signals under the
binary classification situation and little work has been conducted on multi-class scenarios. Intuitively, the more scenarios
an EEG-based control system can distinguish, the wider is its
applicability in the real-world.
To tackle the aforementioned challenges, we propose a novel
hybrid deep neural network that combines the benefits of
both Convolutional Neural Networks (CNNs) [7] and recurrent
neural networks (RNNs) [8] for effective EEG signal decoding.
Our model is capable of modeling high-level, robust and
salient feature representations hidden in the raw EEG signal
streams and capturing complex relationships within data via
stacking multiple layers of information processing modules
in a hierarchical architecture. Specifically, RNN is designed
to model sequential information while CNN is well suited to
extract higher-level spatial variations. In particular, a specific
RNN architecture, named Long Short-Term Memory (LSTM),
is designed to model temporal sequences and their long-range
dependencies more accurately than conventional RNNs. In
comparison, CNN is a typical feed-forward architecture and is
able to extract higher-level features that are invariant to local
spectral and temporal variations. The main contributions of
this paper are highlighted as follows:
• We design a unified deep learning framework that leverages recurrent convolutional neural network to capture
spatial dependencies of raw EEG signals based on features extracted by convolutional operations and temporal correlations through RNN architecture, respectively.
Moreover, an Autoencoder layer is fused to cope with
the possible incomplete and corrupted EEG signals to
enhance the robustness of EEG classification.
• We extensively evaluate our model using a public dataset
and also a limited but easy-to-deploy dataset that we collected using an off-the-shelf EEG device. The experiment
results illustrate that the proposed model achieves highlevel of accuracy over both the public dataset (95.53%)
and the local dataset (94.27%). This demonstrates the
consistent applicability of our proposed model. We have
made our local dataset and the source code used in
our evaluations available to the research community to
encourage further research in this area and foster reproducibility of results.
• We also present an operational prototype of a brain typing
system based on our proposed model, which demonstrates
the efficacy and practicality of our approach. A video
demonstrating the system is made available 2 .
II. EEG CHARACTERISTICS ANALYSIS
The key point of the brain typing system is to precisely
classify the user’s intent signals. Although EEG signals have
low signal-to-noise ratio and are sensitive to background brain
activities and environmental factors, it is possible to recognize
human intent by employing appropriate feature representation
and classification.
2 http://1015xzhang.wixsite.com/mysite/demos

TABLE I
T HE CORRELATION COEFFICIENTS MATRIX . S ELF, C ROSS , AND PD
SEPARATELY DENOTE SELF - SIMILARITY, CROSS - SIMILARITY AND
PERCENTAGE DIFFERENCE .
Class
0
1
2
3
4
Range
Average
STD

0
0.4010
0.2855
0.4146
0.4787
0.3700
0.1932
0.3900
0.0631

1
0.2855
0.5100
0.0689
0.0162
0.0546
0.4938
0.1870
0.1869

2
0.4146
0.0689
0.4126
0.2632
0.3950
0.3458
0.3109
0.1334

3
0.4787
0.0162
0.2632
0.3062
0.2247
0.4625
0.2578
0.1487

4
0.3700
0.0546
0.3950
0.2247
0.3395
0.3404
0.2768
0.1255

Self
0.401
0.51
0.4126
0.3062
0.3395
0.2038
0.3939
0.0700

Cross
0.3872
0.1063
0.2854
0.2457
0.3156
0.2809
0.2680
0.0932

PD
3.44%
79.16%
30.83%
19.76%
7.04%
75.72%
28.05%
27.33%

To illustrate this point, we briefly analyze the similarities
between EEG signals corresponding to different intents and
quantify them using Pearson correlation coefficient. To be able
to effectively interpret multiple classes of human intents, we
assume that the EEG signals should meet the two hypotheses:
1) the intra-intent correlation coefficients should be consistently higher than inter-intent correlation coefficients; 2) the
greater the difference between the intra-intent and inter-class
correlation coefficients, the better classification results.
According to the two hypotheses, we introduce two similarity concepts used in our measurement: the self-similarity
and the cross-similarity. Self-similarity measures the similarity
of EEG signals within the same intent. We randomly select
several EEG data samples from the same intent and calculate
the correlation coefficient of each possible pair of samples.
The self-similarity for the specific intent is measured as the
average of all the sample pairs’ correlation coefficients. Crosssimilarity is defined to measure the similarity of two samples
belonging to different EEG categories. For each specific intent,
we measure the correlation coefficient of each possible intent
pairs. In this work, the EEG dataset (discussed in detail in
Section IV-A) contains 5 intents, hence there are a total of 20
intent pairs and 4 intent pairs of each specific intent. Finally,
for each specific intent, the cross-similarity is the average of
correlation coefficients of each intent pair. Also, we measure
the correlation coefficients matrix for each specific subject and
then calculate the average matrix (by calculating the mean
value of all the matrix). For example, if there are 5 intents
for a specific subject, we calculate a 5 ∗ 5 similarity matrix.
In the matrix, ρĭ,j̆ denotes the correlation coefficients between
the sample of the intent ĭ and the sample of the intent j̆.
Table I shows the correlation coefficients matrix and the
corresponding statistical self- and cross-similarity. The last
column (PD) denotes Percentage Difference between the selfsimilarity and cross-similarity. We can observe from the results that the self-similarity is always higher than the crosssimilarity for all intents, which means that the intra-intent
cohesion of the samples is stronger than the inter-intent
cohesion. Moreover, the percentage difference has a noticeable
fluctuation, which demonstrates that the intra-intent cohesion
varies between different intents. The above analysis results
justify the two hypotheses and lay the foundation for us to
design appropriate feature representations and the classifier.

Fig. 1. The flow chart of the proposed approach. The input raw EEG data is a single sample vector denoted by Eī ∈ RK (take K = 64 as an example). The
C 1 layer denotes the first convolutional layer, the C 2 layer denotes the second convolutional layer, and so on. The same theory, the P 1 layer denotes the
first pooling layer; the FC 1 layer denotes the first fully connected layer; the H 1 layer denotes the first hidden layer. The stacked temporal-spatial feature is
generated by the FC 2 layer in CNN and the H 5 layer in RNN.

III. THE DEEP INTENT RECOGNITION MODEL
This paper proposes a hybrid deep learning model to classify
the raw EEG signal. In this section, we first provide an
overview of the proposed approach and then present the
technical details in subsequent sub-sections.
A. Overview
Figure 1 illustrates the various steps involved. The essential
goal of our approach is to design a deep learning model that
precisely classifies the user’s intents based on EEG data. To
obtain the useful and robust EEG features, we employ a parallel feature learning method which combines RNN and CNN.
RNN is useful in extracting the internal memory to process
arbitrary sequences of EEG signal while CNN is well-suited
for the dimensional relevance representation. In summary, we
propose a hybrid approach which contains several components:
the deep feature learning (Section III-B), the feature transformation and the intent recognition (Section III-C).
B. Deep Feature Learning
We aim to learn the representations of the user’s typing
intent signal which is a 1-D vector (collected in one timepoint). Let us represent the single input EEG signal as Eī ∈
RK (K = 64) with K is the number of dimensions in the
EEG raw signal. Next, we feed Eī to the RNN structure and
the CNN structure for temporal and spatial feature learning
in parallel. At last, the learned temporal features Xt and the
spatial features Xs are combined into the stacked feature X 0
for the latter feature transformation (Section III-C).
1) RNN Feature Learning: In the temporal feature processing part, the RNN structure is employed for its powerful ability
for temporal feature extraction in time-series data. RNN,
which is one class of deep neutral network, is able to explore
the feature dependencies over time through an internal state
of the network, which allows it to exhibit dynamic temporal
behavior. In this section, we take advantages of this trait to
represent the temporal feature of the input EEG signal.
We design an RNN model consisting of three components:
one input layer, 5 hidden layers, and one output layer. There

are two layers of Long Short-Term Memory (LSTM) [9]
(shown as the rectangles in Figure 1) cells among the hidden
layers. Assume a batch of input EEG data contains nbs
(generally called batch size) EEG samples and the total input
data has the 3-D shape as [nbs , 1, 64]. Let the data in the ir
th layer (i = 1, 2, · · · , 7) be denoted by Xir = {Xijk
|j =
1, 2, · · · , nbs , k = 1, 2, · · · , Ki }, Xir ∈ R[nbs ,1,Ki ] , where j
denotes the j-th EEG sample and Ki denotes the number of
dimensions in the i-th layer.
Assume that the weights between layer i and layer i+1
r
r
can be denoted by Wi(i+1)
∈ R[Ki ,Ki+1 ] , for instance, W12
r
Ki
describes the weight between layer 1 and layer 2. bi ∈ R
denotes the biases of i-th layer. The calculation between the
i-th layer data and the i + 1-th layer data can be denoted as
r
r
Xi+1
= Xir ∗ Wi,i+1
+ bri
r
Please note the sizes of Xir , Wi,i+1
and bri must match. For
example, in Figure 1, the transformation between H1 layer and
H2 layer, the sizes of X3r , X2r , W[2,3] , and br2 are separately
[1, 1, 64], [1, 1, 64], [64, 64], and [1, 64].
The 5-th and 6-th layers in the designed structure are LSTM
layers, so the calculation in these layers are implemented as
follows:
r
r
fi = sigmoid(T (X(i−1)j
, X(i)(j−1)
))
r
r
ff = sigmoid(T (X(i−1)j
, X(i)(j−1)
))
r
r
fo = sigmoid(T (X(i−1)j
, X(i)(j−1)
))
r
r
fm = tanh(T (X(i−1)j
, X(i)(j−1)
))

cij = ff

ci(j−1) + fi

r
Xij
= fo

fm

tanh(cij )

where fi , ff , fo and fm represent the input gate, forget gate,
output gate and input modulation gate separately, and
denotes the element-wise multiplication. The cij denotes the
state (memory) in the j-th LSTM cell in the i-th layer, which
is the most significant part to explore the time-series relevance

r
r
between samples. The T (X(i−1)j
, X(i)(j−1)
) denotes the operation as follows:
r
r
∗ W + X(i)(j−1)
∗ W0 + b
X(i−1)j

where W , W 0 and b denote the corresponding weights and
biases.
At last, we obtain the RNN prediction results X7r and employ cross-entropy as the cost function. The cost is optimized
by the AdamOptimizer algorithm [10]. X6r is the data in the
second last layer, which has a directly linear relationship with
the output layer and the prediction results. If the predicted
results have high accuracy, X6r is enabled to directly map to
the sample label space and has the better representative of the
input EEG sample. Therefore, we regard X6r as the temporal
feature extracted by the RNN structure and call it Xt .
2) CNN Feature Learning: While RNN is good in exploring the temporal (inter-sample) relevance, it is unable to
appropriately decode spatial feature (intra-sample) representations. To exploit the spatial connections between different
features in each specific EEG signal, we design a CNN
structure. The CNN structure is comprised of three categories
of components: the convolutional layer, the pooling layer, and
the fully connected layer. The convolutional layer contains a
set of filters to convolve with the EEG data and then through
the feature pooling and non-linear transformation to extract
the geographical features. CNN is well-suited to extract the
spatial relevance of the 2-D input data efficiently. In this paper,
we implement the CNN on the 1-D EEG data. As shown in
Figure 1, the designed CNN is stacked in the following order:
the input layer, the first convolutional layer, the first pooling
layer, the second convolutional layer, the second pooling layer,
the first fully connected layer, the second fully connected layer,
and the output layer.
The input is the same EEG data as the RNN. The input EEG
single sample Eī has shape [1, 64]. Suppose the data in the ic
th layer (i = 1, 2, · · · , 8) is denoted by Xic , Xic ∈ R[1,Ki ,di ] ,
where Kic and di separately denote the dimension number and
the depth in the i-th layer. The data in the first layer only has
depth 1 and X1c = E. We choose the convolutional filter with
size [1, 1] and the stride size [1, 1] in the first convolution. The
stride denotes the x-movements and y-movements distance of
the filter. The padding method is selected as same shape zeropadding, which results in the sample shape keeping constant
in the convolution calculation. The depth of EEG sample
transfers to 2 through the first convolutional layer, so the shape
of X2c is [1, 64, 2].
The pooling layer is a non-linear down-sampling transformation layer. There are several pooling options, with max
pooling being the most popular [11]. The max pooling layer
scans through the inputs along with a sliding window with a
designed stride. Then it outputs the maximum value in every
sub-region that the window is scanned. The pooling layer
reduces the spatial size of the input EEG features and also
prevents overfitting. In the first pooling layer (the 3-th layer
in CNN), we choose the [1, 2] window and [1, 2] stride. The
maximum in each [1, 2] window will be output to the next

layer. The pooling does not change the depth and the shape
of X3c is [1, 32, 2]. Similarly, the second convolutional layer
chooses [1, 2] filter and [1, 1] stride and gets the shape as
[1, 32, 4]. The second pooling layer selects [1, 2] window and
[1, 2] stride and obtains the shape as [1, 16, 4].
In the full connected layer, the high-level reasoning features,
extracted through previous convolutional and pooling layers,
are unfolded to a flattened vector. For example, the data of the
second pooling layer (X5c with shape [1, 16, 4]) is flattened to
the vector with shape [1, 64] (X6c ). Then the output data can be
calculated by following the regular neural network operation:
X7c = T (X6c )
X8c = sof tmax(T (X8c ))
At last, we have the CNN results X8c and employ the
cross-entropy as the cost function. The cost is optimized
by the AdamOptimizer algorithm. X7c has a directly linear
relationship with the output layer and the predicted results.
Therefore, we regard X7c as the spatial feature extracted by
the CNN structure and call it Xs .
In summary, the temporal features Xt and the spatial
features Xs are learned through the parallel RNN and CNN
structures. Both of them have the direct linear relationship
with the EEG sample label, which means that they represent
the temporal and spatial features of the input EEG sample if
both RNN and CNN have high classification accuracy. Next,
we combine the two feature vectors into a flattened stacked
vector, X 0 = {Xt : Xs }.
C. Feature Adaptation
Next, we design a feature adaptation method to map the
stacked features to a correlative new feature space which can
fuse the temporal and spatial features together and highlight
the useful information.
To do so, we introduce the Autoencoder layer [3] to further
interpret EEG signals, which is an unsupervised approach to
learning effective features. The Autoencoder is trained to learn
a compressed and distributed representations for the stacked
EEG feature X 0 . The input of Autoencoder is the stacked
temporal and spatial feature X 0 . Assume h, X́ 0 denote the
hidden layer and output layer data, respectively.
The data transformation procedure is described as the following:
h = Wen X 0 + ben
X́ 0 = Wde h + bde
where Wen , Wde , ben , bde denote the weights and biases in
the encoder and the decoder.
The cost function measures the difference between X 0 and
0
X́ as MSE (mean squared error) which is back-propagated
to the algorithm to adjust the weights and biases. The error
is optimized by the RMSPropOptimizer [12]. The data in the
hidden layer h is the transferred feature, which is output to the
classifier. Finally, the Extreme Gradient Boosting) classifier

TABLE II
T HE MOTOR IMAGERY TASKS AND LABELS AND THE CORRESPONDING

TABLE III
T HE CONFUSION MATRIX OF 5- CLASS CLASSIFICATION

TYPING COMMAND IN THE BRAIN TYPING SYSTEM
Dataset
eegmmidb
emotiv

Item
intent
label
intent
label
command

Task 1
eye closed
0
up arrow
0
up

Task 2
left hand
1
down arrow
1
down

Task 3
right hand
2
left arrow
2
left

Task 4
both hands
3
right arrow
3
right

Task 5
both feet
4
eye closed
4
confirmation

(XGBoost) is employed [13] to classify the EEG streams. It
fuses a set of classification and regression trees (CART) and
exploits detailed information from the input data. It builds
multiple trees and each tree has its leaves and corresponding
scores.
IV. EXPERIMENTS
In this section, we evaluate the proposed deep learning
model using a public dataset and a local dataset collected by
ourselves. At first, a public EEG dataset (called eegmmidb) is
used to assess our proposed deep learning model. The experimental settings (Section IV-A), the overall comparison with the
state-of-the-art methods (Section IV-B), the parameter tuning
(Section IV-C), and the efficiency analysis (Section IV-D) are
separately reported in this section. In addition, we evaluate our
model on a local dataset for demonstrating the good adaptability of proposed method (the collected EEG dataset is called
emotiv) and present the corresponding results (Section IV-E).
A. Experiment Setting
We select the widely used EEG data from PhysioNet eegmmidb (EEG motor movement/imagery database) database3 .
This data is collected using the BCI200 EEG system4 [14]
which records the brain signals using 64 channels at a sampling rate of 160Hz. The subject is asked to wear the EEG
device and sit in front of a computer screen and perform
certain typing actions in response to hints that appear on
the screen. The researchers have carefully annotated the EEG
data to correspond to the actions undertaken by the subject,
which are available from the PhysioBank ATM5 . For our
experiments, we select a total of 280,00 labeled EEG samples
collected from 10 subjects (28,000 samples per subject). Each
sample is a vector of 64 elements, each of which corresponds
to one channel of the EEG data. The subjects performed 5
actions which are labeled as 0 to 4, as shown in Table II.
To evaluate the performance of the classified results, we use
several typical evaluation metrics such as accuracy, precision,
recall, F1 score, ROC (Receiver Operating Characteristic)
curve, and AUC (Area Under the Curve).
B. Overall Comparison
In this section, we report the performance study and then
demonstrate the efficiency of our approach by comparing
with the state-of-the-art methods and other independent deep
learning algorithms. Recall that the proposed approach is a
3 https://www.physionet.org/pn4/eegmmidb/
4 http://www.schalklab.org/research/bci2000
5 https://www.physionet.org/cgi-bin/atm/ATM

Predict
Lable

Total
Average

0
1
2
3
4

Ground
0
2062
17
13
10
18
2120

Truth
1
19
1120
13
5
21
1178

2
23
19
1146
7
15
1210

3
18
15
14
1162
23
1232

4
22
20
11
10
1197
1260

Evaluation
Precision
0.9618
0.9404
0.9574
0.9732
0.9396
4.7723
0.9545

Recall
0.9380
0.9084
0.9257
0.9028
0.9392
4.6140
0.9228

F1
0.9497
0.9241
0.9413
0.9367
0.9394
4.6911
0.9382

AUC
0.9982
0.9977
0.9990
0.9990
0.9987
4.9926
0.9985

hybrid model which uses RNN and CNN for feature learning,
the AE layer for feature transformation, and the XGBoost
classifier for intent recognition. In this experiment, the EEG
data is randomly divided into two parts: the training dataset
(21,000 samples) and the testing dataset (7,000 samples). The
accuracy of our method is calculated as the average of 5 runs
on 10 subjects.
Firstly, we report that our approach achieves the classification accuracy of 0.9553. To take a closer look at the result,
the detailed confusion matrix and classification reports are
presented in Table III. We can observe that for every class,
our approach achieves an an average precision no lower than
0.939. Figure 2 shows the ROC curves of the 5 classes.
Additionally, the accuracy comparison between our method
and other state-of-the-art and baselines are listed in Table IV.
Wavelet transform [2], [15]–[18] and independent component analysis (ICA) [19], [20] are state-of-the-art methods
to process EEG signals. The Deep Neural Network [17],
[19], [21] and Linear discriminant analysis [20] are applied
to classify the EEG data. In addition, the key parameters
of the baselines are listed here: KNN (k=3), Linear SVM
(C = 1), RF (n = 500), LDA (tol = 10−4 ), and AdaBoost
(n = 500, lr = 0.3). The results show that our method
achieves the significantly higher accuracy of 0.9553 than all
the state-of-the-art methods. Our method also performs better
than other deep learning methods such as RNN or CNN.
Moreover, compared with the most existing EEG classification
research which focuses on binary classification, our method
works in multi-class scenario but still achieves a high-level of
accuracy.
To demonstrate the advantage of our proposed hybrid model
for better learning of robust features from raw EEG data,
we also compare our method (joint RNN and CNN) with
the independent deep feature learning method (RNN, CNN).
All extracted features are classified by a XGBoost classifier.
The experimental results are listed in Table V, where we
can see that our approach outperforms RNN and CNN in
classification accuracy by 3.38% and 11.44%, respectively.
Our approach also achieves the lowest standard deviation and
range, implying that it is more stable and reliable. Note that the
RNN on its own (RNN works as both feature extract method
and classifier) without feature representation achieves a higher
accuracy of 0.9325 (in Table IV) than the RNN+AE+XGBoost
method (RNN works as feature extract method), which exhibits an accuracy of 0.9215. This shows that the RNN
represented features are unsuitable for other classifiers and
the inappropriate use of AE may decrease the signal quality.

TABLE IV
P ERFORMANCE COMPARISON WITH THE STATE OF THE ART METHODS . RF
DENOTES R ANDOM F OREST AND LDA DENOTES L INEAR D ISCRIMINANT
A NALYSIS .
State
of the art

Baselines

Index

Methods

1

Almoari [2]

2
3
4
5
6
7
8
9
10
8
11
12
13
14
15
16
17

Sun [15]
Mohammad [16]
Major [19]
Shenoy [21]
Tolic [17]
Rashid [18]
Ward [22]
Sita [20]
Pinheiro [1]
KNN
SVM
RF
LDA
AdaBoost
RNN
CNN
Ours

Binary/Multi

Acc
0.7497

Binary

Multi (3)
Multi (3)
Multi (4)

Multi (5)

0.65
0.845
0.68
0.8206
0.6821
0.92
0.8
0.8724
0.8505
0.8769
0.5082
0.7739
0.5127
0.3431
0.9325
0.8409
0.9553

Figure 3 illustrates separately the accuracy changes along with
the training iterations under three categories of feature learning
methods. Three curves (in Figure 3) show that the proposed
joint method converges to its high accuracy in fewer iterations
than independent RNN and CNN. The learned features are fed
into the AE for further processing and finally classified by the
XGBoost classifier.
C. Parameter Tuning
In this section, we conduct a series of empirical studies
for analyzing the impact of various parameters on the classification accuracy of the proposed approach. We extensively
explore the impact of the following key factors: the training
data size, the RNN learning rate, the CNN learning rate, the
AE learning rate, the XGBoost learning rate, the AE hidden
neuron size, and the classifier. We next investigate the impact
of varying the data used for training on the accuracy of our
model. The results are illustrated in Figure 4. As expected,
the accuracy increases as more data is available for training.
Our method achieves an accuracy of 95% when 55% of
the available data set is used for training. There is a only
a marginal improvement in accuracy with the inclusion of
additional training data. Also observe that we can achieve an
accuracy of 87% with only 15% of training data. This indicates
that our approach is less dependent on the training data size.
The time required for training the model is shown on the right
vertical axis in Figure 4 and as expected varies linearly with
the size of the training data.
Figure 5(a) to Figure 5(d) show that the proposed approach
performs differently under different learning rates in each
component. We choose the appropriate learning rates as 0.005,
0.004, 0.002, and 0.5 for RNN, CNN, AE, and XGBoost, respectively. Figure 5(e) illustrates that the more hidden neurons
in AE, the better classification results. Therefore, we choose
800 neurons as a trade-off between the accuracy and efficiency.
Figure 5(f) shows that the XGBoost classifier outperforms
other classifiers and achieves the highest classification accuracy over the same features refined by RNN+CNN+AE. It

should be noted that all the not mentioned hyper-parameters
are set as default value except those shown in Table VI.
D. Efficiency Analysis
Generally, deep learning algorithms require substantial time
to execute. This can limit their suitability for BCI applications
(e.g., typing) which typically require close to real-time performance. For instance, the practical deployment of a BCI system
could be limited by its recognition time-delay if it takes two
minutes to recognize the user’s intent. In this section, we will
focus on the running time of our approach and compare it to
the widely used baselines. The results are shown in Figure 6.
We first illustrate the time required to train the model in
Figure 6(a). Our model requires 2,000 seconds for training,
which is significantly longer than other baseline approaches.
A breakdown of the training time required for the 3 components, namely, RNN, CNN and XGBoost is also shown.
XGBoost requires the most training time as the result of its
gradient boosting structure. However, training is a one-time
operation. For practical considerations, the execution time of
an algorithm during testing is what matters most. Figure 6(b)
shows that the testing time of our approach is less than 1
second, which is similar with other baselines (except KNN
which requires 9 seconds). In summary, the proposed approach
takes very short testing time although it requires more time to
train the model. Reducing the training time of our approach
will be part of our future work.
E. Adaptability Evaluation on Local EEG Dataset
To examine the adaptability and consistency of our model,
we further evaluate our proposed model on a limited but
easy-to-deploy dataset. We conduct the EEG collection by
using a portable and easy-to-use commercialized EEG headset,
Emotiv Epoc+ headset. The headset contains 14 channels
and the sampling rate is 128 Hz. The local dataset can be
accessed from this link6 . Compared to the BCI 2000 system
(64 channels) used for construct the eegmmidb dataset, our
local equipment (Emotiv headset) only contains 14 channels
and is much easier to be deployed in a natural environment.
1) Experimental Setting: This experiment is carried on
by 7 subjects (4 males and 3 females) aged from 23 to
26. During the experiment, the subject wearing the Emotiv
Epoc+7 EEG collection headset, faces the computer screen
and focuses on the corresponding hint which appears on the
screen (shown in Figure 7). The brain activities and labels
used in this paper are listed in Table II. In summary, this
experiment contains 241,920 samples with 34,560 samples for
each subject. In order to distinguish with the aforementioned
eegmmidb dataset, we name this dataset as emotiv.
2) Recognition Results and Comparison: For each participant, the training set contains 25,920 samples and the testing
set contains 8,640 samples. The experiment parameters are the
same as listed in Table VI. The proposed approach achieves
6 https://drive.google.com/open?id=0B9MuJb6Xx2PIM0otakxuVHpkWkk
7 https://www.emotiv.com/product/emotiv-epoc-14-channel-mobile-eeg/

TABLE V
T HE RECOGNITION ACCURACY OF 10 SUBJECTS UNDER DIFFERENT FEATURE LEARNING METHODS . T HE IMPROVEMENT REPRESENTS THE INCREASE
AMPLITUDE OF OUR METHOD OVER THE MAXIMUM OF RNN AND CNN FEATURE LEARNING METHODS .
S1
0.9005
0.9021
0.9390
0.0369

S2
0.8928
0.5938
0.9186
0.0258

S3
0.9506
0.9395
0.9784
0.0278

S4
0.9264
0.9659
0.9736
0.0077

S6
0.9427
0.9942
0.9832
-0.0110

S7
0.9098
0.9273
0.9675
0.0402

S8
0.9293
0.6177
0.9245
-0.0048

class 0
class 1
class 2
class 3
class 4

0.9

0.9

Accuracy

0.8

Accuracy

0.7

0.6

0.75

0.5

0.7

0.4

Joint
RNN
CNN

0.3

0.65

10 -2

10 -1

10 0

0

500

1000

1500

2000

2500

log(False Positive Rate)

The number of iterations

Fig. 2. The ROC curves of the 5-class classification. Note that X-axis is the logarithmic of
the False Positive Rate.

Fig. 3. The relationship between the testing
accuracy and the number of iterations.

0.9

Accuracy

Accuracy

0.9

0.85

0.8

0.8

0.75

0.75
0

0.005

0.01

0.015

0.02

0

0.025

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

RNN learning rate

CNN learning rate

(a) RNN learning rate

(b) CNN learning rate

0.05

0.966

0.97

0.965

0.964

0.96

Accuracy

Accuracy

0.962
0.955

0.95

0.96

0.958
0.945
0.956

0.94

0.935
0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.954
0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

AE learning rate

XGBoost learning rate

(c) AE learning rate

(d) XGBoost learning rate

0.955

1

0.95

0.9

0.945

1

0.8

0.94

Accuracy

0.7
0.935
0.93

0.6
0.5

0.925
0.4

0.92

0.3

0.915

0.2

0.91
0

200

400

600

800

1000

1200

1400

average
0.9215
0.8409
0.9553
0.0228

std
0.0341
0.1580
0.0335
0.0209

0.96

1200

0.95

1100

0.94

1000

0.93

900

0.92

800

0.91

700

0.9

600

0.89

500

0.88

400

KNN

1600

SVM

RF

LDA

AdaBoost

RNN

CNN

XGBoost

AE hidden neuron size

(e) AE hidden neuron size

(f) Classifier

Fig. 5. The classification accuracy with different hyper-parameter settings

9

2500
RNN
CNN
XGBoost

8

2000

300

0.86
10

20

30

40

50

60

70

80

90

200
100

The training data proportion

Fig. 4. The relationship between the classification
accuracy and the training data proportion.

TABLE VI
H YPER -PARAMETER S ETTING . F OR INSTANCE , RNN CONTAINS ONE
INPUT LAYER (64 NEURONS ), 5 HIDDEN LAYERS (64 NEURONS EACH
LAYER ), AND ONE OUTPUT LAYER (5 NEURONS ).

0.95

0.95

0.85

Range
0.1145
0.4004
0.1013
0.0590

0.87

0.2

0.6
10 -3

S10
0.8498
0.6358
0.8954
0.0456

0.8

0.85

Accuracy

S9
0.9643
0.9310
0.9758
0.0115

1

1

0.95

True Positive Rate

S5
0.9487
0.9013
0.9967
0.0480

Training time (s)

Feature learning
RNN
CNN
RNN+CNN
Improvement

Hyper-parameter
Layer
Neuron size
Iterations
Batch size
RNN
Learning rate
Activation function
Cost function
Regularization
Layer
Input neuron size
1st convolutional
1st pooling
2nd convolutional
2nd pooling
Padding method
Pooling methods
Activation function
CNN
1st fully connected
2nd fully connected
Output neuron size
Iterations
Batch size
Learning rate
Activation function
Cost function
Regularization
Layer
Neuron size
AE
Iterations
Learning rate
Cost function
Objective
Learning rate
Classifier
max depth
Iterations

Value
7=1+5+1
64*1+64*5+5*1
2500
7000
0.005
Soft-max
Cross entropy
`2 norm (λ = 0.004)
8
64
Filter [1,1],stride [1,1], depth 2
Window [1,2], stride [1,2]
Filter [1,2],stride [1,1], depth 4
Window [1,2], stride [1,2]
Zero-padding
Max
ReLU
64
120
5
2500
7000
0.004
Softmax
Cross entropy
`2 norm (λ = 0.001)
1+1+1
184+800+184
400
0.01
MSE
Multi:softmax
0.5
6
500

7

Testing time (s)

Training time (s)

6
1500

1000

5
4
3
2

500

1
0

0
KNN

SVM

RF

LDA

AdaBoost

RNN

CNN

(a) Training time

Ours

KNN

SVM

RF

LDA

AdaBoost

RNN

CNN

(b) Testing time

Fig. 6. The training time and testing time comparison

Ours

the 5-class classification accuracy of 0.9427. The confusion
matrix and evaluation is reported in Table VII.
Subsequently, to demonstrate the efficiency of the proposed
approach, we compare our method with the state-of-the-art
methods and report the accuracy and testing time in Figure 8.
To conclude, our model still achieves good performance
with EEG signals collected from hardware with fewer channels

(a) EEG collection

(b) EEG raw data

Fig. 7. EEG collection and the raw data. The emotiv dataset only consists
of the imagination task data since the rest state data is contaminated by eye
blink and other noises.
TABLE VII
T HE CONFUSION M ATRIX AND THE EVALUATION OVER emotiv DATASET
Ground truth
0
1
0 1608 35
1 18
1602
2 29
31
3 19
25
4
34
19
1708 1712

Predicted
Label
Total
Average

2
51
21
1642
11
15
1740

3
21
29
27
1615
21
1713

4
18
15
22
36
1676
1767

Evaluation
Precision
0.9279
0.9507
0.9377
0.9467
0.9496
4.7126
0.9425

Recall
0.9415
0.9357
0.9437
0.9428
0.9485
4.7122
0.9424

F1
0.9346
0.9432
0.9407
0.9447
0.9490
4.7123
0.9425

AUC
0.9982
0.9977
0.9990
0.9990
0.9987
4.9926
0.9985

10 1

1
0.9
0.8

10 0

Testing time (s)

Accuracy

0.7
0.6
0.5
0.4
0.3

10 -1

10 -2
0.2
0.1
0
SVM

RF

KNN

LDA AdaBoost CNN

RNN

Ours

(a) Accuracy

10 -3

SVM

RF

KNN

LDA AdaBoost CNN

RNN

ours

(b) Testing time

Fig. 8. The accuracy and testing time comparison over emotiv dataset

Fig. 9. Overview of the brain typing system. The user’s typing intent is
collected by headset and sent to the server through client 1. The server uses
the pre-trained deep leaning model to recognize the intent, which is used
to control the typing interface through client 2. The server and clients are
connected using TCP connections.

and in a more natural setting.
V. APPLICATION: BRAIN TYPING SYSTEM
Based on the high EEG signals classification accuracy of the
proposed deep learning approach, in this section, we develop
an online brain typing system to convert user’s thoughts to
texts.
The brain typing system (as shown in Figure 9) consists of
two components: the pre-trained deep learning model and the
online BCI system. The pre-trained deep learning model, which

is trained offline, aims to accurately and recognize the user’s
typing intent in real time. This model (introduced in detail
in Section III) is central to the operation of the brain typing
system, and is the main contribution of this paper. The online
system contains 5 components: the EEG headset, the client
1 (data collector), the server, the client 2 (typing command
receiver), and the typing interface.
The user wears the Emotiv EPOC+ headset (introduced in
Section IV-E) which collects EEG signals and sends the data to
client 1 through a bluetooth connection. The raw EEG signals
are transported to the server through a TCP connection. The
server feeds the incoming EEG signals to the pre-trained deep
learning model. The model produces a classification decision
and converts it to the corresponding typing command which is
sent to client 2 through a TCP connection. The typing interface
receives the command and manifests the appropriate typing
action.
Specifically, the typing interface (Figure 10) can be divided
into three levels: the initial interface, the sub-interface, and the
bottom interface. All the interfaces have similar structure: three
character blocks (separately distributed in left, up, and down
directions), a display block, and a cancel button. The display
block shows the typed output and the cancel button is used
to cancel the last operation. All interfaces include the display
block and cancel button but differ in character blocks. The
typing system totally includes 27 = 3∗9 characters (26 English
alphabets and the space bar) and all of them are separated
by 3 character blocks (each block contains 9 characters) in
the initial interface. Overall, there are 3 alternative selections
and each selection will lead to a specific sub-interface which
contains 9 characters. Again, the 9 = 3 ∗ 3 characters are
divided into 3 character blocks and each of them is connected
to a bottom interface. In the bottom interface, each block
represents only one character. As an example, Figure 10 shows
the procedure to type the character ‘I’.
In the brain typing system, there are 5 commands to control
the interface: ‘left’, ‘up’, ‘right’, ‘cancel’, and ‘confirm’.
Each command corresponds to a specific motor imagery EEG
category (as shown in Table II). To type each single character,
the interface is supposed to accept 6 commands. Consider
typing the letter ‘I’ as an example (see Figure 10). The
sequence of commands to be entered is as follows: ‘left’,
‘confirm’, ‘right’, ‘confirm’, ‘right’, ‘confirm’. In our practical
deployment, the sampling rate of Emotiv EPOC+ headset is
set as 128Hz, which means the server can receive 128 EEG
recordings each second. Since the brainwave signal varies
rapidly and is very easy to be affected by noises, the EEG
data stream is sent to server each half second, which means
that the server receives 64 EEG samples each time. The 64
EEG samples are classified by the deep learning framework
and generates 64 categories of intents. we calculate the mode
of 64 intents and regard the mode as the final intent decision.
Furthermore, to achieve steadiness and reliability, the server
sends command to client 2 only if three consecutive decisions
remain consistent. After the command is sent, the command
list will be reset and the system will wait until 3 consistent

Fig. 10. The brain typing procedure to type the character ‘I’. Firstly, select
the left character block (contains ‘ABCDEFGHI’ characters) in the initial
interface and then confirm the selection to step in the corresponding subinterface; then, select the right character block (contains ‘GHI’ characters) in
the sub-interface and confirm to jump to the bottom interface; at last, select
the right character block (only contains ‘I’) and the character ‘I’ will appear
in the display block after the confirmation.

decisions are made. Therefore, client 2 must wait for at least
1.5 seconds for a command and the entire process of typing
each character takes at least 9 (6∗1.5) seconds. In other words,
theoretically, the proposed brain typing system can achieve the
highest typing speed of 6.67 = 60/9 characters per minute.
VI. DISCUSION
The proposed deep learning framework achieves the highest
accuracy compared to the state-of-the-art EEG classification
methods. The classification accuracy of the public dataset
(eegmmidb) is consistently higher than the local real-world
dataset (emotiv). The possible reason may be due to the
different channels of two datasets (eegmmidb contains 64
channels and emotiv only takes 14 channels). In general, our
framework can achieve high classification accuracy with both
datasets.
The accuracy in the online mode is however lower than
what can be achieved in an offline setting (over 95%), which
could be attributed to a number of reasons. At first, the user’s
mental state and fluctuations in emotions may affect the quality
of the EEG signals. For example, if the offline dataset used to
train the deep learning model is collected when the user is in
an excited emotion state but then applied in an online setting
when the user is upset, would lead to low accuracy. In addition,
subtle variations in the way the EEG headset is mounted on
the subject’s head may also impact online decision making.
Specifically, the position of each of the electrodes (e.g.. the
14 electrodes in the Emotiv headset) on the scalp may vary
during training and testing. Moreover, the EEG signals vary
from person to person, which makes it difficult to construct
a common model that applies to all individuals. One of our
future work is to identify the intra-class variabilities shared by
all the activities of different subjects. Last but not least, some
limitations are caused by the intrinsic attributes of the headset.
For instance, the headset used in our case study is too tight for
the user to wear longer than 30 minutes and the conductive
quality of the wet electrodes decreases after prolonged usage.
VII. RELATED WORK
In EEG decoding and interpretation area, there are mainly
two research directions: the EEG feature representation and
the EEG classifier.
Effectively representing features from EEG raw data is
critical for the classification accuracy for the complexity and
high dimensionality of EEG signals. Vzard et al. [23] employ

common spatial pattern (CSP) along with LDA to pre-process
the EEG data and obtain an accuracy of 71.59% to binary
alertness states. The autoregressive (AR) modeling approach,
a widely used algorithm for EEG feature extraction, is also
broadly combined with other feature extraction techniques to
gain a better performance [24], [25]. Duan et al. [26] introduce
the Autoencoder method for feature extraction and finally
obtain a classification accuracy of 86.69%. Wavelet analysis
[27] is employed to carry on a diagnosis of Traumatic Brain
Injury (TBI) by quantitative EEG (qEEG) data and reaches
87.85% accuracy. Power spectral density [28] is extracted as
EEG data features to input into SVM. The work achieves 76%
accuracy with the data from FC4 ∼ AF8 channels and 92%
with the data from CPz ∼ CP2 channels.
Recently, more and more studies exploit deep learning [29],
[30] to classify EEG signals. The work in [31] builds one deep
belief net (DBN) classifier and achieves the accuracy of 83%
on binary classification. The algorithm combined CNN and
stacked Autoencoder (SAE) is investigated in [32] to classify
EEG Motor Imagery signals and results in 90% accuracy.
Based on the EEG signal decoding, a few researches start to
explore the non-invasive brain typing method. The approach in
[33] enables ALS patients to type through BCI and achieves
the typing rate of 6 characters per minute. The authors in
[34] investigate three kinds of typing interfaces and illustrates
that both matrix presentation and RSVP (rapid serial visual
presentation) can work well.
VIII. CONCLUSION AND FUTUREWORK
In this paper, we present a hybrid deep learning model to
decode the raw EEG signals for the aim of converting the
user’s thoughts to texts. The model employs the RNN and
CNN to learn the temporal and spatial dependency features
from the input EEG raw data and then stack them together.
Our proposed approach adopts an Autoencoder to recognize
the stacked feature and to eliminate the artifacts and employs
the XGBoost classifier for the intent recognition. We evaluate
our approach on a public MI-EEG dataset and also a real
world dataset collected by ourselves. Both results (95.53% and
94.27%) outperform the state-of-the-art methods.
Our future work will focus on improving the accuracy
in the person-independent scenario, wherein some subjects
participate in the training and the rest of subjects involve in the
testing. Our recent study on human activity recognition atop
multi-task learning based framework [35] shows the capability
to capture certain underlying local commonalities under the
intra-class variabilities shared by all the activities of different
subjects.
R EFERENCES
[1] O. R. Pinheiro, L. R. Alves, M. Romero, and J. R. de Souza, “Wheelchair
simulator game for training people with severe disabilities,” in Technology and Innovation in Sports, Health and Wellbeing (TISHW),
International Conference on. IEEE, 2016.
[2] M. H. Alomari, A. Abubaker, A. Turani, A. M. Baniyounes, and A. Manasreh, “EEG Mouse : A Machine Learning-Based Brain Computer
Interface,” vol. 5, no. 4, pp. 193–198, 2014.

[3] T. Nguyen, S. Nahavandi, A. Khosravi, D. Creighton, and I. Hettiarachchi, “Eeg signal analysis for bci application using fuzzy system,”
in Neural Networks (IJCNN), 2015 International Joint Conference on.
IEEE, 2015, pp. 1–8.
[4] E. Niedermeyer and F. L. da Silva, Electroencephalography: basic
principles, clinical applications, and related fields, 2005.
[5] F. Akram, S. M. Han, and T.-S. Kim, “An efficient word typing p300bci system using a modified t9 interface and random forest classifier,”
Computers in biology and medicine, vol. 56, pp. 30–36, 2015.
[6] G. Repovs, “Dealing with noise in eeg recording and data analysis,” in
Informatica Medica Slovenica, vol. 15, no. 1, 2010, pp. 18–25.
[7] H. Sak, A. W. Senior, and F. Beaufays, “Long short-term memory
recurrent neural network architectures for large scale acoustic modeling.”
in Interspeech, 2014.
[8] T. Mikolov, M. Karafiát, L. Burget, J. Cernockỳ, and S. Khudanpur,
“Recurrent neural network based language model.” in Interspeech, vol. 2,
2010, p. 3.
[9] W. Zaremba, I. Sutskever, and O. Vinyals, “Recurrent neural network
regularization,” arXiv preprint arXiv:1409.2329, 2014.
[10] D. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
arXiv preprint arXiv:1412.6980, 2014.
[11] J. Nagi, F. Ducatelle, G. A. Di Caro, D. Cireşan, U. Meier, A. Giusti,
F. Nagi, J. Schmidhuber, and L. M. Gambardella, “Max-pooling convolutional neural networks for vision-based hand gesture recognition,”
in Signal and Image Processing Applications (ICSIPA), 2011 IEEE
International Conference on. IEEE, 2011, pp. 342–347.
[12] G. Hinton, N. Srivastava, and K. Swersky, “Rmsprop: Divide the gradient
by a running average of its recent magnitude,” Neural networks for
machine learning, Coursera lecture 6e, 2012.
[13] T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,”
in Proceedings of the 22nd ACM SIGKDD International Conference on
Knowledge Discovery and Data Mining. ACM, 2016.
[14] G. Schalk, D. J. McFarland, T. Hinterberger, N. Birbaumer, and J. R.
Wolpaw, “Bci2000: a general-purpose brain-computer interface (bci)
system,” IEEE Transactions on biomedical engineering, vol. 51, no. 6,
pp. 1034–1043, 2004.
[15] L. Sun et al., “Classification of imagery motor eeg data with wavelet
denoising and features selection,” in Wavelet Analysis and Pattern
Recognition (ICWAPR), 2016 International Conference on, 2016.
[16] M. H. Alomari, A. M. Baniyounes, and E. A. Awada, “Eeg-based
classification of imagined fists movements using machine learning and
wavelet transform analysis,” 2014.
[17] M. Tolić and F. Jović, “Classification of wavelet transformed eeg
signals with neural network for imagined mental and motor tasks,”
Kineziologija, vol. 45, no. 1, pp. 130–138, 2013.
[18] M. M. or Rashid and M. Ahmad, “Classification of motor imagery hands
movement using levenberg-marquardt algorithm based on statistical
features of eeg signal,” in Electrical Engineering and Information Communication Technology (ICEEICT), 2016 3rd International Conference
on. IEEE, 2016, pp. 1–6.
[19] T. C. Major and J. M. Conrad, “The effects of pre-filtering and
individualizing components for electroencephalography neural network
classification,” in SoutheastCon, 2017. IEEE, 2017.
[20] J. Sita and G. Nair, “Feature extraction and classification of eeg signals
for mapping motor area of the brain,” in Control Communication and

[21]

[22]

[23]
[24]
[25]
[26]
[27]
[28]

[29]
[30]
[31]

[32]
[33]

[34]

[35]

Computing (ICCC), 2013 International Conference on. IEEE, 2013,
pp. 463–468.
H. V. Shenoy, A. Vinod, and C. Guan, “Shrinkage estimator based
regularization for eeg motor imagery classification,” in Information,
Communications and Signal Processing (ICICS), 2015 10th International Conference on. IEEE, 2015.
C. R. Ward, J. Picone, and I. Obeid, “Applications of UBMs and IVectors in EEG Subject Verification,” 2016 IEEE 38th Annual International Conference of the Engineering in Medicine and Biology Society
(EMBC), pp. 748–751, 2016.
L. Vézard, P. Legrand, M. Chavent, F. Faı̈ta-Aı̈nseba, and L. Trujillo,
“Eeg classification for the detection of mental states,” Applied Soft
Computing, vol. 32, pp. 113–131, 2015.
M. Rahman, W. Ma, D. Tran, and J. Campbell, “A comprehensive survey
of the feature extraction methods in the eeg research,” Algorithms and
architectures for parallel processing, pp. 274–283, 2012.
Y. Zhang, B. Liu, X. Ji, and D. Huang, “Classification of eeg signals
based on autoregressive model and wavelet packet decomposition,”
Neural Processing Letters, pp. 1–14, 2016.
L. Duan, Y. Xu, S. Cui, J. Chen, and M. Bao, “Feature extraction of
motor imagery eeg based on extreme learning machine auto-encoder,”
in Proceedings of ELM-2015 volume 1, 2016, pp. 361–370.
B. Albert, J. Zhang, A. Noyvirt, R. Setchi, H. Sjaaheim, S. Velikova,
and F. Strisland, “Automatic eeg processing for the early diagnosis of
traumatic brain injury,” in World Automation Congress (WAC), 2016.
A. M. Al-Kaysi, A. Al-Ani, C. K. Loo, T. Y. Powell, D. M. Martin,
M. Breakspear, and T. W. Boonstra, “Predicting tdcs treatment outcomes
of patients with major depressive disorder using automated eeg classification,” Journal of Affective Disorders, vol. 208, pp. 597–603, 2017.
S. Jirayucharoensak, S. Pan-Ngum, and P. Israsena, “Eeg-based emotion
recognition using deep learning network with principal component based
covariate shift adaptation,” The Scientific World Journal, vol. 2014, 2014.
Y. Ren and Y. Wu, “Convolutional deep belief networks for feature extraction of eeg signal,” in Neural Networks (IJCNN), 2014 International
Joint Conference on. IEEE, 2014, pp. 2850–2853.
X. An, D. P. Kuang, X. J. Guo, Y. L. Zhao, and L. H. He, “A
Deep Learning Method for Classification of EEG Data Based on Motor
Imagery,” Intelligent Computing in Bioinformatics, vol. 8590, pp. 203–
210, 2014.
Y. R. Tabar and U. Halici, “A novel deep learning approach for classification of eeg motor imagery signals,” Journal of neural engineering,
vol. 14, no. 1, p. 016003, 2016.
W. Speier, N. Chandravadia, D. Roberts, S. Pendekanti, and N. Pouratian, “Online bci typing using language model classifiers by als patients
in their homes,” Brain-Computer Interfaces, vol. 4, no. 1-2, pp. 114–121,
2017.
M. Moghadamfalahi, U. Orhan, M. Akcakaya, H. Nezamfar, M. FriedOken, and D. Erdogmus, “Language-model assisted brain computer
interface for typing: A comparison of matrix and rapid serial visual
presentation,” IEEE Transactions on Neural Systems and Rehabilitation
Engineering, vol. 23, no. 5, pp. 910–920, 2015.
L. Yao, F. Nie, Q. Z. Sheng, T. Gu, X. Li, and S. Wang, “Learning
from less for better: semi-supervised activity recognition via shared
structure discovery,” in Proceedings of the 2016 ACM International Joint
Conference on Pervasive and Ubiquitous Computing. ACM, 2016.

