# Deep Extreme Learning Machine and Its Application in EEG Classification

Authors: Shifei Ding; Nan Zhang; Xinzheng Xu; Lili Guo; Jian Zhang
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

Recently, deep learning has aroused wide interest in machine learning fields. Deep learning is a multilayer perceptron artificial neural network algorithm. Deep learning has the advantage of approximating the complicated function and alleviating the optimization difficulty associated with deep models. Multilayer extreme learning machine (MLELM) is a learning algorithm of an artificial neural network which takes advantages of deep learning and extreme learning machine. Not only does MLELM approximate the complicated function but it also does not need to iterate during the training process. We combining with MLELM and extreme learning machine with kernel (KELM) put forward deep extreme learning machine (DELM) and apply it to EEG classification in this paper. This paper focuses on the application of DELM in the classification of the visual feedback experiment, using MATLAB and the second brain-computer interface (BCI) competition datasets. By simulating and analyzing the results of the experiments, effectiveness of the application of DELM in EEG classification is confirmed.


---

## Body

## 1. Introduction

Brain-computer interface (BCI) is a kind of technology that enables people to communicate with a computer or to control devices with EEG signals [1]. The core technologies of BCI are to extract the feature of preprocessed EEG and classify ready-processed EEG, and this paper is mainly about classification analysis. In recent years, BCI has gotten a great advance with the rapid development of computer technology. BCI has been applied to many fields, such as medicine and military [2–4]. Currently, many different methods have been proposed for EEG classification, including decision trees, local backpropagation (BP) algorithm, Bayes classifier, K-nearest neighbors (KNN), support vector machine (SVM), batch incremental support vector machine (BISVM), and ELM [5–8]. However, most of them are shallow neural network algorithms in which the capabilities achieve approximating the complex functions that are subject to certain restrictions, and there is no such restriction in deep learning.Deep learning is an artificial neural network learning algorithm which has multilayer perceptrons. Deep learning has achieved an approximation of complex functions and alleviated the optimization difficulty associated with the deep models [9–11]. In 2006, the concept of deep learning was first proposed by Hinton and Salakhutdinov who presented deep structure of multilayer autoencoder [12]. Deep belief network was proposed by Hinton [13]. LeCun et al. put forward the first real deep learning algorithm—convolutional neural networks (CNNs) [14]. More and more people put forward some new algorithms based on deep learning. Then convolutional deep belief network was put forward [15]. In 2013, the model of multilayer extreme learning machine (MLELM) was proposed by Kasun et al. [16], and DELM takes advantages of deep learning and extreme learning machine. Extreme learning machine (ELM) proposed by Huang et al. is a simple and efficient learning algorithm of single layer feed-forward neural networks (SLFNs) [17, 18]. In addition, some people put forward some deformation algorithms based on ELM, such as regularized extreme learning machine (RELM) [19], extreme learning machine with kernel (KELM) [20], optimally pruned extreme learning machine (OP-ELM) [21], and evolving fuzzy optimally pruned extreme learning machine (eF-OP-ELM) [22].We combining with multilayer extreme learning machine (MLELM) and extreme learning machine with kernel (KELM) put forward deep extreme learning machine (DELM) and apply it to EEG classification, and the paper is organized as follows: Section2 gives the model of ELM, RELM, and KELM. Section 3 describes the model structure of MLELM. Section 4 details the model structure of DELM. Section 5 first evaluates the usefulness of DELM on UCI datasets and then applies DELM to EEG classification. In Section 6, the conclusion is gotten.


## 2. Extreme Learning Machine (ELM)

### 2.1. Basic Extreme Learning Machine (Basic ELM)

ELM proposed by Huang et al. is a simple and efficient learning algorithm of SLFNs. The model of ELM constituted input layer, single-hidden layer, and output layer. The model structure of ELM is shown in Figure1, with j input layer nodes, n hidden layer nodes, m output layer nodes, and the hidden layer activation function g
(
x
).Figure 1
The model structure of ELM.ForN distinct samples x
i
∈
R
N
×
R
j, y
i
∈
R
N
×
R
m
(
i
=
1,2
,
…
,
N
), the outputs of the hidden layer can be expressed as (1), and the numerical relationship between output of the hidden layer and output of the output layer can be expressed as (2):(1)
h
=
g
a
x
+
b
,
(2)
h
x
i
V
=
y
i
,
i
=
1,2
,
…
,
N
.The above equation can be written compactly as (3)
H
V
=
Y
,where(4)
H
=
g
(
a
→
1
,
b
1
,
x
→
1
)
g
(
a
→
1
,
b
1
,
x
→
2
)
⋯
g
(
a
→
n
,
b
n
,
x
→
N
)
g
(
a
→
2
,
b
2
,
x
→
1
)
g
(
a
→
2
,
b
2
,
x
→
2
)
⋯
g
(
a
→
n
,
b
n
,
x
→
N
)
⋮
⋮
⋱
⋮
g
(
a
→
n
,
b
n
,
x
→
1
)
g
(
a
→
n
,
b
n
,
x
→
2
)
⋯
g
(
a
→
n
,
b
n
,
x
→
N
)
T
,
(5)
V
=
v
1
T
v
2
T
⋮
v
n
T
n
×
m
,
Y
=
y
1
T
y
2
T
⋮
y
N
T
N
×
m
,where a
i
=
[
a
i
1
,
a
i
2
,
…
,
a
i
n
]
T are the weights connecting the ith input nodes and hidden layer, b
j is the bias of the jth hidden node, and v
j
=
[
v
j
1
,
v
j
2
,
…
,
v
j
n
]
T are the weights connecting the jth hidden node and the output layer. H is output matrix of the neural network. We need to set input weights a
i
j and the bias of the hidden layer b
j; the output weights V can be obtained by a series of linear equations transformations.In conclusion, using ELM to obtain the output weightsV can be divided into three steps.Step 1.
Randomly select numerical values between 0 and 1 to set input weightsa
i
j and the bias of the hidden layer b
j.Step 2.
Calculate the output matrixH.Step 3.
Calculate the output weightsV:(6)
V
=
H
†
Y
,where H
† represents the generalized inverse matrix of the output matrix H.


### 2.2. Regularized Extreme Learning Machine (RELM)

ELM has the advantage of fast training speed and high generalization performance, but ELM also has the disadvantage of bad robustness. Deng et al. combining with experiential risk and structural risk put forward regularized extreme learning machine (RELM) which has better robustness, and RELM aims to solve the output weights by minimizing the regularized cost function of least squares estimate regularization, which leads to the following formulation:(7)
min
⁡
L
RELM
=
1
2
V
2
+
C
2
Y
-
H
V
2
,where C is a scale parameter which adjusts experiential risk and structural risk.By setting the gradient ofL
RELM with respect to V to zero, we have(8)
V
+
C
H
T
(
Y
-
H
V
)
=
0
.When the number of training samples is more than the number of hidden layer nodes, the output weight matrixV in RELM can be expressed as (9)
V
=
I
C
+
H
T
H
-
1
H
T
Y
.When the number of training samples is less than the number of hidden layer nodes, the output weight matrixV in RELM can be expressed as (10)
V
=
H
T
I
C
+
H
H
T
-
1
Y
.


### 2.3. Extreme Learning Machine with Kernel (KELM)

Huang et al. combining with the kernel method and extreme learning machine put forward extreme learning machine with kernel (KELM). The outputs of the hidden layer of ELM can be regarded as the nonlinear mapping of samples. When the mapping is an unknown, we can construct the kernel function instead ofH
H
T:(11)
H
H
T
=
Ω
ELM
=
K
(
x
1
,
x
1
)
⋯
K
(
x
1
,
x
N
)
⋮
⋱
⋮
K
(
x
N
,
x
1
)
⋯
K
(
x
N
,
x
N
)
,
h
x
H
T
=
K
x
,
x
1
⋮
K
x
,
x
N
T
.The most popular kernel of KELM in use is the Gaussian kernelK
(
x
i
,
x
j
)
=
exp
⁡
(
-
x
i
-
x
j
/
γ
), where γ is the kernel parameter.Thus, the output weight matrixV in KELM can be expressed as (12) and the Classification of formula of KELM can be expressed as (13):(12)
V
=
I
C
+
Ω
ELM
-
1
Y
,
(13)
f
x
=
h
x
H
T
V
=
K
x
,
x
1
⋮
K
x
,
x
N
T
I
C
+
Ω
ELM
-
1
Y
.


## 2.1. Basic Extreme Learning Machine (Basic ELM)

ELM proposed by Huang et al. is a simple and efficient learning algorithm of SLFNs. The model of ELM constituted input layer, single-hidden layer, and output layer. The model structure of ELM is shown in Figure1, with j input layer nodes, n hidden layer nodes, m output layer nodes, and the hidden layer activation function g
(
x
).Figure 1
The model structure of ELM.ForN distinct samples x
i
∈
R
N
×
R
j, y
i
∈
R
N
×
R
m
(
i
=
1,2
,
…
,
N
), the outputs of the hidden layer can be expressed as (1), and the numerical relationship between output of the hidden layer and output of the output layer can be expressed as (2):(1)
h
=
g
a
x
+
b
,
(2)
h
x
i
V
=
y
i
,
i
=
1,2
,
…
,
N
.The above equation can be written compactly as (3)
H
V
=
Y
,where(4)
H
=
g
(
a
→
1
,
b
1
,
x
→
1
)
g
(
a
→
1
,
b
1
,
x
→
2
)
⋯
g
(
a
→
n
,
b
n
,
x
→
N
)
g
(
a
→
2
,
b
2
,
x
→
1
)
g
(
a
→
2
,
b
2
,
x
→
2
)
⋯
g
(
a
→
n
,
b
n
,
x
→
N
)
⋮
⋮
⋱
⋮
g
(
a
→
n
,
b
n
,
x
→
1
)
g
(
a
→
n
,
b
n
,
x
→
2
)
⋯
g
(
a
→
n
,
b
n
,
x
→
N
)
T
,
(5)
V
=
v
1
T
v
2
T
⋮
v
n
T
n
×
m
,
Y
=
y
1
T
y
2
T
⋮
y
N
T
N
×
m
,where a
i
=
[
a
i
1
,
a
i
2
,
…
,
a
i
n
]
T are the weights connecting the ith input nodes and hidden layer, b
j is the bias of the jth hidden node, and v
j
=
[
v
j
1
,
v
j
2
,
…
,
v
j
n
]
T are the weights connecting the jth hidden node and the output layer. H is output matrix of the neural network. We need to set input weights a
i
j and the bias of the hidden layer b
j; the output weights V can be obtained by a series of linear equations transformations.In conclusion, using ELM to obtain the output weightsV can be divided into three steps.Step 1.
Randomly select numerical values between 0 and 1 to set input weightsa
i
j and the bias of the hidden layer b
j.Step 2.
Calculate the output matrixH.Step 3.
Calculate the output weightsV:(6)
V
=
H
†
Y
,where H
† represents the generalized inverse matrix of the output matrix H.


## 2.2. Regularized Extreme Learning Machine (RELM)

ELM has the advantage of fast training speed and high generalization performance, but ELM also has the disadvantage of bad robustness. Deng et al. combining with experiential risk and structural risk put forward regularized extreme learning machine (RELM) which has better robustness, and RELM aims to solve the output weights by minimizing the regularized cost function of least squares estimate regularization, which leads to the following formulation:(7)
min
⁡
L
RELM
=
1
2
V
2
+
C
2
Y
-
H
V
2
,where C is a scale parameter which adjusts experiential risk and structural risk.By setting the gradient ofL
RELM with respect to V to zero, we have(8)
V
+
C
H
T
(
Y
-
H
V
)
=
0
.When the number of training samples is more than the number of hidden layer nodes, the output weight matrixV in RELM can be expressed as (9)
V
=
I
C
+
H
T
H
-
1
H
T
Y
.When the number of training samples is less than the number of hidden layer nodes, the output weight matrixV in RELM can be expressed as (10)
V
=
H
T
I
C
+
H
H
T
-
1
Y
.


## 2.3. Extreme Learning Machine with Kernel (KELM)

Huang et al. combining with the kernel method and extreme learning machine put forward extreme learning machine with kernel (KELM). The outputs of the hidden layer of ELM can be regarded as the nonlinear mapping of samples. When the mapping is an unknown, we can construct the kernel function instead ofH
H
T:(11)
H
H
T
=
Ω
ELM
=
K
(
x
1
,
x
1
)
⋯
K
(
x
1
,
x
N
)
⋮
⋱
⋮
K
(
x
N
,
x
1
)
⋯
K
(
x
N
,
x
N
)
,
h
x
H
T
=
K
x
,
x
1
⋮
K
x
,
x
N
T
.The most popular kernel of KELM in use is the Gaussian kernelK
(
x
i
,
x
j
)
=
exp
⁡
(
-
x
i
-
x
j
/
γ
), where γ is the kernel parameter.Thus, the output weight matrixV in KELM can be expressed as (12) and the Classification of formula of KELM can be expressed as (13):(12)
V
=
I
C
+
Ω
ELM
-
1
Y
,
(13)
f
x
=
h
x
H
T
V
=
K
x
,
x
1
⋮
K
x
,
x
N
T
I
C
+
Ω
ELM
-
1
Y
.


## 3. Multilayer Extreme Learning Machine (MLELM)

### 3.1. Extreme Learning Machine-Autoencoder (ELM-AE)

Autoencoder is an artificial neural network model which is commonly used in deep learning. Autoencoder is an unsupervised neural network, the outputs of autoencoder are the same as the inputs of autoencoder, and autoencoder is a kind of neural networks which reproduces the input signal as much as possible. ELM-AE proposed by Kasun et al. is a new method of neural network which can reproduce the input signal as well as autoencoder.The model of ELM-AE constituted input layer, single-hidden layer, and output layer. The model structure of ELM-AE is shown in Figure2, with j input layer nodes, n hidden layer nodes, j output layer nodes, and the hidden layer activation function g
(
x
). According to the output of the hidden layer representing the input signal, ELM-AE can be divided into three different representations as follows.j
>
n: Compressed Representation: this represents features from a higher dimensional input signal space to a lower dimensional feature space.
j
=
n: Equal Dimension Representation: this represents features from an input signal space dimension equal to feature space dimension.
j
<
n: Sparse Representation: this represents features from a lower dimensional input signal space to a higher dimensional feature space.Figure 2
The model structure of ELM-AE.There are two differences between ELM-AE and traditional ELM. Firstly, ELM is a supervised neural network and the output of ELM is label, but ELM-AE is an unsupervised neural network and the output of ELM-AE is the same as the input of ELM-AE. Secondly, the input weights of ELM-AE are orthogonal and the bias of hidden layer of ELM-AE is also orthogonal, but ELM is not so. ForN distinct samples, x
i
∈
R
n
×
R
j
,
(
i
=
1,2
,
…
,
N
), the outputs of ELM-AE hidden layer can be expressed as (14), and the numerical relationship between the outputs of the hidden layer and the outputs of the output layer can be expressed as (15):(14)
h
=
g
(
a
x
+
b
)
,
where
a
T
a
=
I
,
b
T
b
=
1
,
(15)
h
x
i
V
=
x
i
T
,
i
=
1,2
,
…
,
N
.Using ELM-AE to obtain the output weightsV can be also divided into three steps, but the calculation method of the output weights V of ELM-AE in Step 3 is different from the calculation method of the output weights V of ELM.For sparse and compressed ELM-AE representations, output weightsV are calculated by (16) and (17).When the number of training samples is more than the number of hidden layer nodes,(16)
V
=
I
C
+
H
T
H
-
1
H
T
X
.When the number of training samples is less than the number of hidden layer nodes,(17)
V
=
H
T
I
C
+
H
H
T
-
1
X
.For equal dimension ELM-AE representation, output weightsV are calculated by (18)
V
=
H
-
1
X
.


### 3.2. Multilayer Extreme Learning Machine (MLELM)

In 2006, Hinton et al. put forward an effective method of establishing a multilayer neural network on the unsupervised data. In the new method, first the parameters in each layer are obtained by unsupervised training, and then the network is fine-tuned by supervised learning. In 2013, MLELM was proposed by Kasun et al. Like other deep learning models, MLELM makes use of unsupervised learning to train the parameters in each layer, but the difference is that MLELM does not need to fine-tune the network. Thus, compared with other deep learning algorithms, MLELM does not need to spend a long time on the network training.MLELM makes use of ELM-AE to train the parameters in each layer, and MLELM hidden layer activation functions can be either linear or nonlinear piecewise. If the activation function of the MLELMith hidden layer is g
(
x
), then the parameters between the MLELM ith hidden layer and the MLELM (i
-
1) hidden layer (if i
-
1
=
0, this layer is the input layer) are trained by ELM-AE, and the activation function should be g
(
x
), too. The numerical relationship between the outputs of MLELM ith hidden layer and the outputs of MLELM (i
-
1) hidden layer can be expressed as (19)
H
i
=
g
v
i
T
H
i
-
1
,where H
i represents the outputs of MLELM ith hidden layer (if i
-
1
=
0, this layer is the input layer, and H
i
-
1 represents the inputs of MLELM). The model of MLELM is shown in Figure 3, v
i represents the output weights of ELM-AE, the input of ELM-AE is H
i
-
1, and the number of ELM-AE hidden layer nodes is identical to the number of MLELM ith hidden nodes when the parameters between the MLELM ith hidden layer and the MLELM (i
-
1) hidden layer are trained by ELM-AE. The output of the connections between the last hidden layer and the output layer can be analytically calculated using regularized least squares.Figure 3
The model structure of MLELM.


## 3.1. Extreme Learning Machine-Autoencoder (ELM-AE)

Autoencoder is an artificial neural network model which is commonly used in deep learning. Autoencoder is an unsupervised neural network, the outputs of autoencoder are the same as the inputs of autoencoder, and autoencoder is a kind of neural networks which reproduces the input signal as much as possible. ELM-AE proposed by Kasun et al. is a new method of neural network which can reproduce the input signal as well as autoencoder.The model of ELM-AE constituted input layer, single-hidden layer, and output layer. The model structure of ELM-AE is shown in Figure2, with j input layer nodes, n hidden layer nodes, j output layer nodes, and the hidden layer activation function g
(
x
). According to the output of the hidden layer representing the input signal, ELM-AE can be divided into three different representations as follows.j
>
n: Compressed Representation: this represents features from a higher dimensional input signal space to a lower dimensional feature space.
j
=
n: Equal Dimension Representation: this represents features from an input signal space dimension equal to feature space dimension.
j
<
n: Sparse Representation: this represents features from a lower dimensional input signal space to a higher dimensional feature space.Figure 2
The model structure of ELM-AE.There are two differences between ELM-AE and traditional ELM. Firstly, ELM is a supervised neural network and the output of ELM is label, but ELM-AE is an unsupervised neural network and the output of ELM-AE is the same as the input of ELM-AE. Secondly, the input weights of ELM-AE are orthogonal and the bias of hidden layer of ELM-AE is also orthogonal, but ELM is not so. ForN distinct samples, x
i
∈
R
n
×
R
j
,
(
i
=
1,2
,
…
,
N
), the outputs of ELM-AE hidden layer can be expressed as (14), and the numerical relationship between the outputs of the hidden layer and the outputs of the output layer can be expressed as (15):(14)
h
=
g
(
a
x
+
b
)
,
where
a
T
a
=
I
,
b
T
b
=
1
,
(15)
h
x
i
V
=
x
i
T
,
i
=
1,2
,
…
,
N
.Using ELM-AE to obtain the output weightsV can be also divided into three steps, but the calculation method of the output weights V of ELM-AE in Step 3 is different from the calculation method of the output weights V of ELM.For sparse and compressed ELM-AE representations, output weightsV are calculated by (16) and (17).When the number of training samples is more than the number of hidden layer nodes,(16)
V
=
I
C
+
H
T
H
-
1
H
T
X
.When the number of training samples is less than the number of hidden layer nodes,(17)
V
=
H
T
I
C
+
H
H
T
-
1
X
.For equal dimension ELM-AE representation, output weightsV are calculated by (18)
V
=
H
-
1
X
.


## 3.2. Multilayer Extreme Learning Machine (MLELM)

In 2006, Hinton et al. put forward an effective method of establishing a multilayer neural network on the unsupervised data. In the new method, first the parameters in each layer are obtained by unsupervised training, and then the network is fine-tuned by supervised learning. In 2013, MLELM was proposed by Kasun et al. Like other deep learning models, MLELM makes use of unsupervised learning to train the parameters in each layer, but the difference is that MLELM does not need to fine-tune the network. Thus, compared with other deep learning algorithms, MLELM does not need to spend a long time on the network training.MLELM makes use of ELM-AE to train the parameters in each layer, and MLELM hidden layer activation functions can be either linear or nonlinear piecewise. If the activation function of the MLELMith hidden layer is g
(
x
), then the parameters between the MLELM ith hidden layer and the MLELM (i
-
1) hidden layer (if i
-
1
=
0, this layer is the input layer) are trained by ELM-AE, and the activation function should be g
(
x
), too. The numerical relationship between the outputs of MLELM ith hidden layer and the outputs of MLELM (i
-
1) hidden layer can be expressed as (19)
H
i
=
g
v
i
T
H
i
-
1
,where H
i represents the outputs of MLELM ith hidden layer (if i
-
1
=
0, this layer is the input layer, and H
i
-
1 represents the inputs of MLELM). The model of MLELM is shown in Figure 3, v
i represents the output weights of ELM-AE, the input of ELM-AE is H
i
-
1, and the number of ELM-AE hidden layer nodes is identical to the number of MLELM ith hidden nodes when the parameters between the MLELM ith hidden layer and the MLELM (i
-
1) hidden layer are trained by ELM-AE. The output of the connections between the last hidden layer and the output layer can be analytically calculated using regularized least squares.Figure 3
The model structure of MLELM.


## 4. Deep Extreme Learning Machine (DELM)

MLELM makes use of ELM-AE to train the parameters in each layer, and ML-ELM hidden layer activation functions can be either linear or nonlinear piecewise, and the mapping of MLELM is linear or nonlinear. When the mapping is an unknown, we can add one hidden layer and construct the kernel function. In other words, at last the outputs of MLELM hidden layerH
k (the matrix size is n
k
*
N) are the inputs of KELM, and we can construct the kernel function instead of H
k
+
1
H
k
+
1
T. This algorithm combining with MLELM and KELM is called deep extreme learning machine (DELM):(20)
H
k
+
1
H
k
+
1
T
=
Ω
DELM
=

K
(
H
k
(
:
,
1
)
,
H
k
(
:
,
1
)
)
⋯
K
(
H
k
(
:
,
1
)
,
H
k
(
:
,
N
)
)
⋮
⋱
⋮
K
(
H
k
(
:
,
N
)
,
H
k
(
:
,
1
)
)
⋯
K
(
H
k
(
:
,
N
)
,
H
k
(
:
,
N
)
)
,
h
k
+
1
h
k
x
H
k
+
1
T
=
K
h
k
x
,
H
k
:
,
1
⋮
K
h
k
x
,
H
k
:
,
N
T
.The model of DELM is shown in Figure4, v
i
(
i
∈
[
1
,
…
,
k
]
) represents the output weights of ELM-AE, the input of ELM-AE is H
i
-
1, and the number of ELM-AE hidden layer nodes is identical to the number of DELM ith hidden nodes when the parameters between the DELM ith hidden layer and the MLELM (i
-
1) hidden layer are trained by ELM-AE. And we can construct the kernel function instead of H
k
+
1
H
k
+
1
T; thus the output weight matrix V in DELM can be expressed as (21) and the classification of formula of KELM can be expressed as (22):(21)
V
=
I
C
+
Ω
DELM
-
1
Y
,
(22)
f
x
=
h
k
+
1
(
h
k
(
x
)
)
H
k
+
1
T
V
=
K
(
h
k
(
x
)
,
H
k
(
:
,
1
)
)
⋮
K
(
h
k
(
x
)
,
H
k
(
:
,
N
)
)
T
I
C
+
Ω
DELM
-
1
Y
.Figure 4
The model structure of DELM.


## 5. Experiments and Analysis

The execution environment of experiments is MATLAB 2012B. All activation functions of ELM, MLELM, and DELM select sigmoid function and the kernel functions of KELM and DELM are Gaussian kernel. ELM, MLELM, and DELM were executed100 times, and the average values and the best values are reported.


### 5.1. UCI Datasets Classification

In this part, the UCI datasets were used to test the performances of DELM, and the details of UCI dataset are presented in Table1, including ionosphere dataset and diabetes dataset.Table 1
The details of UCI datasets.
Dataset
Number of samples
Attributes information
Number of labels
Training samples
Testing samples
Number of attributes
Attribute characteristics
Ionosphere
200
151
34
Continuous attributes
2
Diabetes
576
192
8
Categorical, integer
2As shown in Figure5, we can make choices that the numbers of ELM hidden layer nodes on ionosphere dataset and diabetes dataset are 50 and 40, the regularized parameter C and the kernel parameter γ of KELM on ionosphere dataset are 103 and 102, and the regularized parameter C and the kernel parameter γ of KELM on diabetes dataset are 102 and 101. The structure of MLELM on ionosphere dataset is 34-30-30-50-2, where the parameter C for layer 34-30 is 103, the parameter C for layer 30-50 is 10−2, and the parameter C for layer 50-2 is 108. And the structure of MLELM on diabetes dataset is 8-10-10-40-2, where the parameter C for layer 8-10 is 106, the parameter C for layer 10-40 is 108, and the parameter C for layer 40-2 is 105. The structure of DELM on ionosphere dataset is 34-30-30-L-2, where the parameter C for layer 34-30 is 101, the parameter C for layer L-2 is 103, and the kernel parameter γ is 102. And the structure of DELM on diabetes dataset is 8-10-10-L-2, where the parameter C for layer 34-30 is 101, the parameter C for layer L-2 is 102, and the kernel parameter γ is 101.Figure 5
Basic ELM and KELM for UCI dataset.
(a)
Basic ELM for ionosphere dataset
(b)
KELM for ionosphere dataset
(c)
Basic ELM for diabetes dataset
(d)
KELM for diabetes datasetThe performance comparison of DELM with ELM, KELM, and MLELM on UCI datasets is shown in Table2. It is clearly observed that DELM testing accuracy is higher than MLELM, either the average or the maximum, and the best values of DELM testing accuracy are higher than ELM and KELM. And DELM training time is the longest, but there is little difference between testing times. Sigillito et al. investigated ionosphere dataset using backpropagation and the perceptron training algorithm; they found that “linear” perceptron achieved 90.7%, a “nonlinear” perceptron achieved 92%, and backprop an average of over 96% accuracy [23]. Although the average value of DELM on ionosphere dataset only achieves 94.74%, the best value has reached to 99.34%.Table 2
Performance comparison of DELM with ELM, KELM, and MLELM on UCI datasets.
Dataset
Algorithm
#
Training accuracy
Testing accuracy
Training time (s)
Testing time (s)
Ionosphere
Basic ELM
Average
0.9207 ± 0.0151
0.9342 ± 0.0222
0.0044 ± 0.0074
0.0013 ± 0.0048
Best
0.9500
0.9735
—
—
KELM
—
0.9900
0.9735
0.0064
0.0017
ML-ELM
Average
0.9112 ± 0.0159
0.9447 ± 0.0216
0.0115 ± 0.0116
0.0014 ± 0.0050
Best
0.9500
0.9801
—
—
DELM
Average
0.9503 ± 0.0111
0.9474 ± 0.0292
0.0164 ± 0.0079
0.0045 ± 0.0064
Best
0.9750
0.9934
—
—
Diabetes
Basic ELM
Average
0.7983 ± 0.0062
0.7725 ± 0.0129
0.0072 ± 0.0132
0.0034 ± 0.0098
Best
0.8125
0.8021
—
—
KELM
—
0.7899
0.7917
0.6818
0.0314
ML-ELM
Average
0.7666 ± 0.0207
0.7522 ± 0.0324
0.0091 ± 0.0143
0.0044 ± 0.0109
Best
0.7951
0.8177
—
—
DELM
Average
0.7871 ± 0.0079
0.7580 ± 0.0422
0.0641 ± 0.0143
0.0112 ± 0.0086
Best
0.8038
0.8229


### 5.2. EEG Classification

The effectiveness of DELM has been confirmed, so the effectiveness of the application of DELM in EEG classification is tested in this part.


#### 5.2.1. Visual Feedback Experiment (Healthy Subject)

The performances of DELM on the second BCI competition dataset IA are tested in this section, and this dataset comes from the visual feedback experiment (healthy subject) provided by University of Tuebingen [24].The datasets were taken from a healthy subject. The subject was asked to move a cursor up and down on a computer screen, while his cortical potentials were taken. Cortical positivity leads to a downward movement of the cursor on the screen. Cortical negativity leads to an upward movement of the cursor. Each trial lasted 6 s. The visual feedback was presented from second 2 to second 5.5. Only this 3.5-second interval of every trial is provided for training and testing. The sampling rate of 256 Hz and the recording length of 3.5 s result in 896 samples per channel for every trial, and the details are presented in Table3.Table 3
The details of the second BCI competition dataset IA.
Dataset
Number of samples
Number of attributes
Number of labels
Training samples
Testing samples
BCI competition II dataset IA
268
293
5376
2As shown in Figure6, we can make choices that the number of ELM hidden layer nodes on BCI competition II dataset IA is 3000; the regularized parameter C and the kernel parameter γ of KELM are 103 and 104. The structure of MLELM is 5376-500-500-3000-2, where the parameter C for layer 5376-500 is 21, the parameter C for layer 500-3000 is 28, and the parameter C for layer 3000-2 is 2−7. The structure of DELM is 5376-500-500-L-2, where the parameter C for layer 5376-500 is 10−1, the parameter C for layer L-2 is 10−1, and the kernel parameter γ is 102.Figure 6
Basic ELM and KELM for the BCI competition II dataset IA.
(a)
Basic ELM for the BCI competition II dataset IA
(b)
KELM for the BCI competition II dataset IAThe performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IA is shown in Table4. It is clearly observed that DELM testing accuracy is higher than MLELM, either the average or the maximum, and the best values of DELM testing accuracy are higher than ELM and KELM. MLELM training time is the longest, and the testing time of MLELM and DELM is less than ELM. The performance comparison of DELM with the results of BCI competition II dataset IA is shown in Table 5. It is clear that the average error value of DELM on BCI competition II dataset IA achieves 13.50%, but the min error value has reduced to 8.19%, which is much lower than the results of BCI competition II.Table 4
Performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IA.
Dataset
Algorithm
#
Training accuracy
Testing accuracy
Training time (s)
Testing time (s)
BCI competition II dataset IA
Basic ELM
Average
1.0000 ± 0
0.8609 ± 0.0187
3.3670 ± 0.0866
2.2361 ± 0.0498
Best
1.0000
0.9078
—
—
KELM
—
0.8582
0.9010
0.0754
0.1430
ML-ELM
Average
0.7849 ± 0.0213
0.8642 ± 0.0216
9.2012 ± 0.1444
0.4820 ± 0.0272
Best
0.8358
0.9113
—
—
DELM
Average
0.7515 ± 0.0161
0.8650 ± 0.0224
6.7438 ± 0.2099
0.2932 ± 0.0290
Best
0.7873
0.9181
—
—Table 5
Performance comparison of DELM with the results of BCI competition II on dataset IA.
#
Contributor
Error (%)
1
Brett Mensh
11.3
2
Guido Dornhege
11.6
3
Kai-Min Chung
11.9
4
Tzu-Kuo Huang
15.0
5
David Pinto
15.7
6
Juma Mbwana
17.1
7
Vladimir Bostanov
17.4
8
Ulrich Hoffmann
17.8
9
Deniz Erdogmus
19.1
10
Justin Sanchez
19.8
*
Ours (the average value of DELM)
13.50
*
Ours (the best value of DELM)
8.19


#### 5.2.2. Visual Feedback Experiment (ALS Patient)

The performances of DELM on the second BCI competition dataset IB are tested in this section, and this dataset comes from the visual feedback experiment (ALS patient) provided by University of Tuebingen.The datasets were taken from an artificially respirated ALS patient. The subject was asked to move a cursor up and down on a computer screen, while his cortical potentials were taken. Cortical positivity leads to a downward movement of the cursor on the screen. Cortical negativity leads to an upward movement of the cursor. Each trial lasted 8 s. The visual feedback was presented from second 2 to second 6.5. Only this 4.5-second interval of every trial is provided for training and testing. The sampling rate of 256 Hz and the recording length of 4.5 s result in 1152 samples per channel for every trial, and the details are presented in Table6.Table 6
The details of the second BCI competition dataset IB.
Dataset
Number of samples
Number of attributes
Number of labels
Training samples
Testing samples
BCI competition II dataset IB
200
180
8064
2As shown in Figure7, we can make choices that the number of ELM hidden layer nodes on BCI competition II dataset IA is 2000; the regularized parameter C and the kernel parameter γ of KELM are 10−1 and 103. The structure of MLELM is 8064-500-500-2000-2, where the parameter C for layer 8064-500 is 101, the parameter C for layer 500-2000 is 108, and the parameter C for layer 2000-2 is 104. The structure of DELM is 8064-500-500-L-2, where the parameter C for layer 8064-500 is 10−2, the parameter C for layer L-2 is 10−8, and the kernel parameter γ is 101.Figure 7
Basic ELM and KELM for the BCI competition II dataset IA.
(a)
Basic ELM for the BCI competition II dataset IA
(b)
KELM for the BCI competition II dataset IAThe performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IB is shown in Table7. It is clearly observed that the best of DELM testing accuracy is not lower than MLELM, ELM, and KELM. MLELM training time is the longest, and the testing time of MLELM and DELM is less than ELM. The performance comparison of DELM with the results of BCI competition II dataset IA is shown in Table 8. It is clear that the average error value of DELM on BCI competition II dataset IA achieves 47.89%, but the min error value has reduced to 39.44%, which is much lower than the results of BCI competition II.Table 7
Performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IB.
Dataset
Algorithm
#
Training accuracy
Testing accuracy
Training time (s)
Testing time (s)
BCI competition II dataset IB
Basic ELM
Average
1.0000 ± 0
0.5172 ± 0.0395
3.4636 ± 0.1255
2.0602 ± 0.0670
Best
1.0000
0.6056
—
—
KELM
—
1.0000
0.5333
0.0738
0.1285
ML-ELM
Average
0.6145 ± 0.0306
0.5219 ± 0.0284
10.4225 ± 0.1134
0.3970 ± 0.0182
Best
0.6750
0.5833
—
—
DELM
Average
0.7151 ± 0.0485
0.5211 ± 0.0266
8.9814 ± 0.2085
0.2603 ± 0.0231
Best
0.8450
0.6056
—
—Table 8
Performance comparison of DELM with the results of BCI competition II on dataset IB.
#
Contributor
error
1
Vladimir Bostanov
45.6%
2
Tzu-Kuo Huang
46.7%
2
Juma Mbwana
46.7%
4
Kai-Min Chung
47.8%
5
Xichen Sun
48.3%
6
Amir Saffari
53.3%
7
Fabien Torre
54.4%
8
Brett Mensh
56.1%
*
Ours (the average value of DELM)
47.89%
*
Ours (the best value of DELM)
39.44%


## 5.1. UCI Datasets Classification

In this part, the UCI datasets were used to test the performances of DELM, and the details of UCI dataset are presented in Table1, including ionosphere dataset and diabetes dataset.Table 1
The details of UCI datasets.
Dataset
Number of samples
Attributes information
Number of labels
Training samples
Testing samples
Number of attributes
Attribute characteristics
Ionosphere
200
151
34
Continuous attributes
2
Diabetes
576
192
8
Categorical, integer
2As shown in Figure5, we can make choices that the numbers of ELM hidden layer nodes on ionosphere dataset and diabetes dataset are 50 and 40, the regularized parameter C and the kernel parameter γ of KELM on ionosphere dataset are 103 and 102, and the regularized parameter C and the kernel parameter γ of KELM on diabetes dataset are 102 and 101. The structure of MLELM on ionosphere dataset is 34-30-30-50-2, where the parameter C for layer 34-30 is 103, the parameter C for layer 30-50 is 10−2, and the parameter C for layer 50-2 is 108. And the structure of MLELM on diabetes dataset is 8-10-10-40-2, where the parameter C for layer 8-10 is 106, the parameter C for layer 10-40 is 108, and the parameter C for layer 40-2 is 105. The structure of DELM on ionosphere dataset is 34-30-30-L-2, where the parameter C for layer 34-30 is 101, the parameter C for layer L-2 is 103, and the kernel parameter γ is 102. And the structure of DELM on diabetes dataset is 8-10-10-L-2, where the parameter C for layer 34-30 is 101, the parameter C for layer L-2 is 102, and the kernel parameter γ is 101.Figure 5
Basic ELM and KELM for UCI dataset.
(a)
Basic ELM for ionosphere dataset
(b)
KELM for ionosphere dataset
(c)
Basic ELM for diabetes dataset
(d)
KELM for diabetes datasetThe performance comparison of DELM with ELM, KELM, and MLELM on UCI datasets is shown in Table2. It is clearly observed that DELM testing accuracy is higher than MLELM, either the average or the maximum, and the best values of DELM testing accuracy are higher than ELM and KELM. And DELM training time is the longest, but there is little difference between testing times. Sigillito et al. investigated ionosphere dataset using backpropagation and the perceptron training algorithm; they found that “linear” perceptron achieved 90.7%, a “nonlinear” perceptron achieved 92%, and backprop an average of over 96% accuracy [23]. Although the average value of DELM on ionosphere dataset only achieves 94.74%, the best value has reached to 99.34%.Table 2
Performance comparison of DELM with ELM, KELM, and MLELM on UCI datasets.
Dataset
Algorithm
#
Training accuracy
Testing accuracy
Training time (s)
Testing time (s)
Ionosphere
Basic ELM
Average
0.9207 ± 0.0151
0.9342 ± 0.0222
0.0044 ± 0.0074
0.0013 ± 0.0048
Best
0.9500
0.9735
—
—
KELM
—
0.9900
0.9735
0.0064
0.0017
ML-ELM
Average
0.9112 ± 0.0159
0.9447 ± 0.0216
0.0115 ± 0.0116
0.0014 ± 0.0050
Best
0.9500
0.9801
—
—
DELM
Average
0.9503 ± 0.0111
0.9474 ± 0.0292
0.0164 ± 0.0079
0.0045 ± 0.0064
Best
0.9750
0.9934
—
—
Diabetes
Basic ELM
Average
0.7983 ± 0.0062
0.7725 ± 0.0129
0.0072 ± 0.0132
0.0034 ± 0.0098
Best
0.8125
0.8021
—
—
KELM
—
0.7899
0.7917
0.6818
0.0314
ML-ELM
Average
0.7666 ± 0.0207
0.7522 ± 0.0324
0.0091 ± 0.0143
0.0044 ± 0.0109
Best
0.7951
0.8177
—
—
DELM
Average
0.7871 ± 0.0079
0.7580 ± 0.0422
0.0641 ± 0.0143
0.0112 ± 0.0086
Best
0.8038
0.8229


## 5.2. EEG Classification

The effectiveness of DELM has been confirmed, so the effectiveness of the application of DELM in EEG classification is tested in this part.


### 5.2.1. Visual Feedback Experiment (Healthy Subject)

The performances of DELM on the second BCI competition dataset IA are tested in this section, and this dataset comes from the visual feedback experiment (healthy subject) provided by University of Tuebingen [24].The datasets were taken from a healthy subject. The subject was asked to move a cursor up and down on a computer screen, while his cortical potentials were taken. Cortical positivity leads to a downward movement of the cursor on the screen. Cortical negativity leads to an upward movement of the cursor. Each trial lasted 6 s. The visual feedback was presented from second 2 to second 5.5. Only this 3.5-second interval of every trial is provided for training and testing. The sampling rate of 256 Hz and the recording length of 3.5 s result in 896 samples per channel for every trial, and the details are presented in Table3.Table 3
The details of the second BCI competition dataset IA.
Dataset
Number of samples
Number of attributes
Number of labels
Training samples
Testing samples
BCI competition II dataset IA
268
293
5376
2As shown in Figure6, we can make choices that the number of ELM hidden layer nodes on BCI competition II dataset IA is 3000; the regularized parameter C and the kernel parameter γ of KELM are 103 and 104. The structure of MLELM is 5376-500-500-3000-2, where the parameter C for layer 5376-500 is 21, the parameter C for layer 500-3000 is 28, and the parameter C for layer 3000-2 is 2−7. The structure of DELM is 5376-500-500-L-2, where the parameter C for layer 5376-500 is 10−1, the parameter C for layer L-2 is 10−1, and the kernel parameter γ is 102.Figure 6
Basic ELM and KELM for the BCI competition II dataset IA.
(a)
Basic ELM for the BCI competition II dataset IA
(b)
KELM for the BCI competition II dataset IAThe performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IA is shown in Table4. It is clearly observed that DELM testing accuracy is higher than MLELM, either the average or the maximum, and the best values of DELM testing accuracy are higher than ELM and KELM. MLELM training time is the longest, and the testing time of MLELM and DELM is less than ELM. The performance comparison of DELM with the results of BCI competition II dataset IA is shown in Table 5. It is clear that the average error value of DELM on BCI competition II dataset IA achieves 13.50%, but the min error value has reduced to 8.19%, which is much lower than the results of BCI competition II.Table 4
Performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IA.
Dataset
Algorithm
#
Training accuracy
Testing accuracy
Training time (s)
Testing time (s)
BCI competition II dataset IA
Basic ELM
Average
1.0000 ± 0
0.8609 ± 0.0187
3.3670 ± 0.0866
2.2361 ± 0.0498
Best
1.0000
0.9078
—
—
KELM
—
0.8582
0.9010
0.0754
0.1430
ML-ELM
Average
0.7849 ± 0.0213
0.8642 ± 0.0216
9.2012 ± 0.1444
0.4820 ± 0.0272
Best
0.8358
0.9113
—
—
DELM
Average
0.7515 ± 0.0161
0.8650 ± 0.0224
6.7438 ± 0.2099
0.2932 ± 0.0290
Best
0.7873
0.9181
—
—Table 5
Performance comparison of DELM with the results of BCI competition II on dataset IA.
#
Contributor
Error (%)
1
Brett Mensh
11.3
2
Guido Dornhege
11.6
3
Kai-Min Chung
11.9
4
Tzu-Kuo Huang
15.0
5
David Pinto
15.7
6
Juma Mbwana
17.1
7
Vladimir Bostanov
17.4
8
Ulrich Hoffmann
17.8
9
Deniz Erdogmus
19.1
10
Justin Sanchez
19.8
*
Ours (the average value of DELM)
13.50
*
Ours (the best value of DELM)
8.19


### 5.2.2. Visual Feedback Experiment (ALS Patient)

The performances of DELM on the second BCI competition dataset IB are tested in this section, and this dataset comes from the visual feedback experiment (ALS patient) provided by University of Tuebingen.The datasets were taken from an artificially respirated ALS patient. The subject was asked to move a cursor up and down on a computer screen, while his cortical potentials were taken. Cortical positivity leads to a downward movement of the cursor on the screen. Cortical negativity leads to an upward movement of the cursor. Each trial lasted 8 s. The visual feedback was presented from second 2 to second 6.5. Only this 4.5-second interval of every trial is provided for training and testing. The sampling rate of 256 Hz and the recording length of 4.5 s result in 1152 samples per channel for every trial, and the details are presented in Table6.Table 6
The details of the second BCI competition dataset IB.
Dataset
Number of samples
Number of attributes
Number of labels
Training samples
Testing samples
BCI competition II dataset IB
200
180
8064
2As shown in Figure7, we can make choices that the number of ELM hidden layer nodes on BCI competition II dataset IA is 2000; the regularized parameter C and the kernel parameter γ of KELM are 10−1 and 103. The structure of MLELM is 8064-500-500-2000-2, where the parameter C for layer 8064-500 is 101, the parameter C for layer 500-2000 is 108, and the parameter C for layer 2000-2 is 104. The structure of DELM is 8064-500-500-L-2, where the parameter C for layer 8064-500 is 10−2, the parameter C for layer L-2 is 10−8, and the kernel parameter γ is 101.Figure 7
Basic ELM and KELM for the BCI competition II dataset IA.
(a)
Basic ELM for the BCI competition II dataset IA
(b)
KELM for the BCI competition II dataset IAThe performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IB is shown in Table7. It is clearly observed that the best of DELM testing accuracy is not lower than MLELM, ELM, and KELM. MLELM training time is the longest, and the testing time of MLELM and DELM is less than ELM. The performance comparison of DELM with the results of BCI competition II dataset IA is shown in Table 8. It is clear that the average error value of DELM on BCI competition II dataset IA achieves 47.89%, but the min error value has reduced to 39.44%, which is much lower than the results of BCI competition II.Table 7
Performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IB.
Dataset
Algorithm
#
Training accuracy
Testing accuracy
Training time (s)
Testing time (s)
BCI competition II dataset IB
Basic ELM
Average
1.0000 ± 0
0.5172 ± 0.0395
3.4636 ± 0.1255
2.0602 ± 0.0670
Best
1.0000
0.6056
—
—
KELM
—
1.0000
0.5333
0.0738
0.1285
ML-ELM
Average
0.6145 ± 0.0306
0.5219 ± 0.0284
10.4225 ± 0.1134
0.3970 ± 0.0182
Best
0.6750
0.5833
—
—
DELM
Average
0.7151 ± 0.0485
0.5211 ± 0.0266
8.9814 ± 0.2085
0.2603 ± 0.0231
Best
0.8450
0.6056
—
—Table 8
Performance comparison of DELM with the results of BCI competition II on dataset IB.
#
Contributor
error
1
Vladimir Bostanov
45.6%
2
Tzu-Kuo Huang
46.7%
2
Juma Mbwana
46.7%
4
Kai-Min Chung
47.8%
5
Xichen Sun
48.3%
6
Amir Saffari
53.3%
7
Fabien Torre
54.4%
8
Brett Mensh
56.1%
*
Ours (the average value of DELM)
47.89%
*
Ours (the best value of DELM)
39.44%


## 5.2.1. Visual Feedback Experiment (Healthy Subject)

The performances of DELM on the second BCI competition dataset IA are tested in this section, and this dataset comes from the visual feedback experiment (healthy subject) provided by University of Tuebingen [24].The datasets were taken from a healthy subject. The subject was asked to move a cursor up and down on a computer screen, while his cortical potentials were taken. Cortical positivity leads to a downward movement of the cursor on the screen. Cortical negativity leads to an upward movement of the cursor. Each trial lasted 6 s. The visual feedback was presented from second 2 to second 5.5. Only this 3.5-second interval of every trial is provided for training and testing. The sampling rate of 256 Hz and the recording length of 3.5 s result in 896 samples per channel for every trial, and the details are presented in Table3.Table 3
The details of the second BCI competition dataset IA.
Dataset
Number of samples
Number of attributes
Number of labels
Training samples
Testing samples
BCI competition II dataset IA
268
293
5376
2As shown in Figure6, we can make choices that the number of ELM hidden layer nodes on BCI competition II dataset IA is 3000; the regularized parameter C and the kernel parameter γ of KELM are 103 and 104. The structure of MLELM is 5376-500-500-3000-2, where the parameter C for layer 5376-500 is 21, the parameter C for layer 500-3000 is 28, and the parameter C for layer 3000-2 is 2−7. The structure of DELM is 5376-500-500-L-2, where the parameter C for layer 5376-500 is 10−1, the parameter C for layer L-2 is 10−1, and the kernel parameter γ is 102.Figure 6
Basic ELM and KELM for the BCI competition II dataset IA.
(a)
Basic ELM for the BCI competition II dataset IA
(b)
KELM for the BCI competition II dataset IAThe performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IA is shown in Table4. It is clearly observed that DELM testing accuracy is higher than MLELM, either the average or the maximum, and the best values of DELM testing accuracy are higher than ELM and KELM. MLELM training time is the longest, and the testing time of MLELM and DELM is less than ELM. The performance comparison of DELM with the results of BCI competition II dataset IA is shown in Table 5. It is clear that the average error value of DELM on BCI competition II dataset IA achieves 13.50%, but the min error value has reduced to 8.19%, which is much lower than the results of BCI competition II.Table 4
Performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IA.
Dataset
Algorithm
#
Training accuracy
Testing accuracy
Training time (s)
Testing time (s)
BCI competition II dataset IA
Basic ELM
Average
1.0000 ± 0
0.8609 ± 0.0187
3.3670 ± 0.0866
2.2361 ± 0.0498
Best
1.0000
0.9078
—
—
KELM
—
0.8582
0.9010
0.0754
0.1430
ML-ELM
Average
0.7849 ± 0.0213
0.8642 ± 0.0216
9.2012 ± 0.1444
0.4820 ± 0.0272
Best
0.8358
0.9113
—
—
DELM
Average
0.7515 ± 0.0161
0.8650 ± 0.0224
6.7438 ± 0.2099
0.2932 ± 0.0290
Best
0.7873
0.9181
—
—Table 5
Performance comparison of DELM with the results of BCI competition II on dataset IA.
#
Contributor
Error (%)
1
Brett Mensh
11.3
2
Guido Dornhege
11.6
3
Kai-Min Chung
11.9
4
Tzu-Kuo Huang
15.0
5
David Pinto
15.7
6
Juma Mbwana
17.1
7
Vladimir Bostanov
17.4
8
Ulrich Hoffmann
17.8
9
Deniz Erdogmus
19.1
10
Justin Sanchez
19.8
*
Ours (the average value of DELM)
13.50
*
Ours (the best value of DELM)
8.19


## 5.2.2. Visual Feedback Experiment (ALS Patient)

The performances of DELM on the second BCI competition dataset IB are tested in this section, and this dataset comes from the visual feedback experiment (ALS patient) provided by University of Tuebingen.The datasets were taken from an artificially respirated ALS patient. The subject was asked to move a cursor up and down on a computer screen, while his cortical potentials were taken. Cortical positivity leads to a downward movement of the cursor on the screen. Cortical negativity leads to an upward movement of the cursor. Each trial lasted 8 s. The visual feedback was presented from second 2 to second 6.5. Only this 4.5-second interval of every trial is provided for training and testing. The sampling rate of 256 Hz and the recording length of 4.5 s result in 1152 samples per channel for every trial, and the details are presented in Table6.Table 6
The details of the second BCI competition dataset IB.
Dataset
Number of samples
Number of attributes
Number of labels
Training samples
Testing samples
BCI competition II dataset IB
200
180
8064
2As shown in Figure7, we can make choices that the number of ELM hidden layer nodes on BCI competition II dataset IA is 2000; the regularized parameter C and the kernel parameter γ of KELM are 10−1 and 103. The structure of MLELM is 8064-500-500-2000-2, where the parameter C for layer 8064-500 is 101, the parameter C for layer 500-2000 is 108, and the parameter C for layer 2000-2 is 104. The structure of DELM is 8064-500-500-L-2, where the parameter C for layer 8064-500 is 10−2, the parameter C for layer L-2 is 10−8, and the kernel parameter γ is 101.Figure 7
Basic ELM and KELM for the BCI competition II dataset IA.
(a)
Basic ELM for the BCI competition II dataset IA
(b)
KELM for the BCI competition II dataset IAThe performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IB is shown in Table7. It is clearly observed that the best of DELM testing accuracy is not lower than MLELM, ELM, and KELM. MLELM training time is the longest, and the testing time of MLELM and DELM is less than ELM. The performance comparison of DELM with the results of BCI competition II dataset IA is shown in Table 8. It is clear that the average error value of DELM on BCI competition II dataset IA achieves 47.89%, but the min error value has reduced to 39.44%, which is much lower than the results of BCI competition II.Table 7
Performance comparison of DELM with ELM, KELM, and MLELM on the BCI competition II dataset IB.
Dataset
Algorithm
#
Training accuracy
Testing accuracy
Training time (s)
Testing time (s)
BCI competition II dataset IB
Basic ELM
Average
1.0000 ± 0
0.5172 ± 0.0395
3.4636 ± 0.1255
2.0602 ± 0.0670
Best
1.0000
0.6056
—
—
KELM
—
1.0000
0.5333
0.0738
0.1285
ML-ELM
Average
0.6145 ± 0.0306
0.5219 ± 0.0284
10.4225 ± 0.1134
0.3970 ± 0.0182
Best
0.6750
0.5833
—
—
DELM
Average
0.7151 ± 0.0485
0.5211 ± 0.0266
8.9814 ± 0.2085
0.2603 ± 0.0231
Best
0.8450
0.6056
—
—Table 8
Performance comparison of DELM with the results of BCI competition II on dataset IB.
#
Contributor
error
1
Vladimir Bostanov
45.6%
2
Tzu-Kuo Huang
46.7%
2
Juma Mbwana
46.7%
4
Kai-Min Chung
47.8%
5
Xichen Sun
48.3%
6
Amir Saffari
53.3%
7
Fabien Torre
54.4%
8
Brett Mensh
56.1%
*
Ours (the average value of DELM)
47.89%
*
Ours (the best value of DELM)
39.44%


## 6. Conclusions

This paper explores the application of DELM in EEG classification and makes use of two BCI competition datasets to test the performances of DELM. Experimental results show that DELM has the advantage of the least training time and the good efficiency and DELM is an effective BCI classifier. Although DELM has these advantages, there are some places which should be improved, such as the number of all hidden layer nodes, each hidden layer activation function, and each layer parameterC that are difficult to determine. In this paper, DELM is used to classify preprocessed EEG data and the feature attributes of preprocessed EEG are not extracted, which has certain effects on the experimental results. Future research is to combine the EEG feature extraction methods and DELM, which will be applied to the EEG classification.



---

*Source: 129021-2015-05-27.xml*