# Evolutionary Voting-Based Extreme Learning Machines

Authors: Nan Liu; Jiuwen Cao; Zhiping Lin; Pin Pin Pek; Zhi Xiong Koh; Marcus Eng Hock Ong
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

Voting-based extreme learning machine (V-ELM) was proposed to improve learning efficiency where majority voting was employed. V-ELM assumes that all individual classifiers contribute equally to the decision ensemble. However, in many real-world scenarios, this assumption does not work well. In this paper, we aim to enhance V-ELM by introducing weights to distinguish the importance of each individual ELM classifier in decision making. Genetic algorithm is used for optimizing these weights. This evolutionary V-ELM is named as EV-ELM. Results on several benchmark databases show that EV-ELM achieves the highest classification accuracy compared with V-ELM and ELM.


---

## Body

## 1. Introduction

The extreme learning machine (ELM) [1, 2] was proposed as a new type of learning algorithm for single-hidden layer feedforward neural network (SLFN). ELM adopts a training mechanism that does not require parameter tuning so that it learns faster than traditional gradient-based neural networks while achieving even better classification/regression performance [3]. ELM has received increasing attentions in recent years; many efforts have been dedicated to improve its performance [4–7] and apply it in various applications [8–14]. Among these ELM extensions, ensemble learning-based methods [15–23] have found their advantages such as better accuracy and less variance compared with the original ELM algorithm.Ensemble learning has been popular for decades. Jain et al. [24] wrote a concise yet informative introduction to the classifier combination. Polikar [25] comprehensively reviewed the area of multiple classifier system (ensemble system) for decision making. Majority voting [26] is one of the most commonly used combining strategies. This rule seeks the class that receives the highest number of votes and assigns it to the predicted label for the testing pattern. Since each classifier in an ensemble does not necessarily contribute equally to the final decision, Littlestone and Warmuth [27] proposed weighing individual classifiers to make them discriminative. Ensemble learning-based ELM algorithms [16, 18, 19] were reported to successfully resolve the problems of predictive instability and overfitting.Among ensemble-based extensions, voting-based ELM (V-ELM) [18] was proposed to perform multiple independent ELM training using a simple and effective learning architecture. A decision was made based on majority voting. V-ELM not only enhanced the classification performance, but also lowered the variance. In this paper, we aim to investigate the introduction of weights for all individual ELM classifiers to enhance the V-ELM algorithm. The hypothesis is that each individual ELM classifier presents various levels of confidence in decision making. In our proposed method, each weight represents the importance of an ELM classifier. Final decision is made with the weighted majority voting scheme.The remainder of this paper is organized as follows. Section2 briefly reviews ELM and V-ELM algorithms. Section 3 presents the proposed evolutionary V-ELM (EV-ELM) algorithm. Section 4 demonstrates the performance of EV-ELM and compares it with ELM and V-ELM. Section 5 concludes this study.


## 2. Background

### 2.1. Extreme Learning Machine

In the process of SLFN learning, ELM randomly selects weights and biases for hidden nodes. Then it analytically determines the output weights by finding the least squares solution. Given a training set consisting ofN samples L
=
{
(
x
n
,
t
n
)
∣
x
n
∈
R
u
,
t
n
∈
R
v
,
n
=
1,2
,
.
.
.
,
N
}, where x
n is an u
×
1 input vector and t
n is an v
×
1 target vector, an SLFN with N
~ hidden nodes is formulated as
(1)
f
N
~
(
x
n
)
=
∑
i
=
1
N
~
‍
β
i
g
(
w
i
·
x
n
+
b
i
)
=
t
n
,
n
=
1,2
,
…
,
N
,
where the additive hidden node is employed. Weight vector w
i connects the ith hidden node and input neurons. In approximating N samples using N
~ hidden nodes, β
i, w
i, and b
i are supposed to exist if zero error is obtained. Consequently, (1) can be written as
(2)
H
β
^
=
T
,
where H
(
w
1
,
…
,
w
N
~
,
b
1
,
…
,
b
N
~
,
x
1
,
…
,
x
N
) is hidden layer output matrix of the network; h
n
i
=
g
(
w
i
·
x
n
+
b
i
) is the output of ith hidden neuron with respect to x
n, i
=
1,2
,
…
,
N
~ and n
=
1,2
,
…
,
N; β
^
=
[
β
1
,
…
,
β
N
~
]
T and T
=
[
t
1
,
…
,
t
N
]
T are output weight matrix and target matrix, respectively.The ELM algorithm can be summarized as three steps: (1) generate parametersw
i and b
i randomly for i
=
1
,
…
,
N
~; (2) calculate the hidden layer output matrix H; and (3) calculate the output weight using β
=
H
†
T. It has been shown in [28] that any continuous target functions in R
n can be universally approximated using single SLFN with randomly chosen additive hidden nodes.


### 2.2. Voting-Based Extreme Learning Machine

In ELM, randomized hidden nodes are used and remain unchanged during the training. Some testing samples could be misclassified in certain situations, for example, when they are near the classification boundary. To tackle this issue, V-ELM incorporates multiple individual ELMs and makes decisions with majority voting. V-ELM uses a fixed number of hidden nodes for all individual ELMs. All these ELMs are trained with the same dataset and the learning parameters of each ELM are randomly initialized. The predicted class label is then determined by majority voting on all results obtained from ELMs.


## 2.1. Extreme Learning Machine

In the process of SLFN learning, ELM randomly selects weights and biases for hidden nodes. Then it analytically determines the output weights by finding the least squares solution. Given a training set consisting ofN samples L
=
{
(
x
n
,
t
n
)
∣
x
n
∈
R
u
,
t
n
∈
R
v
,
n
=
1,2
,
.
.
.
,
N
}, where x
n is an u
×
1 input vector and t
n is an v
×
1 target vector, an SLFN with N
~ hidden nodes is formulated as
(1)
f
N
~
(
x
n
)
=
∑
i
=
1
N
~
‍
β
i
g
(
w
i
·
x
n
+
b
i
)
=
t
n
,
n
=
1,2
,
…
,
N
,
where the additive hidden node is employed. Weight vector w
i connects the ith hidden node and input neurons. In approximating N samples using N
~ hidden nodes, β
i, w
i, and b
i are supposed to exist if zero error is obtained. Consequently, (1) can be written as
(2)
H
β
^
=
T
,
where H
(
w
1
,
…
,
w
N
~
,
b
1
,
…
,
b
N
~
,
x
1
,
…
,
x
N
) is hidden layer output matrix of the network; h
n
i
=
g
(
w
i
·
x
n
+
b
i
) is the output of ith hidden neuron with respect to x
n, i
=
1,2
,
…
,
N
~ and n
=
1,2
,
…
,
N; β
^
=
[
β
1
,
…
,
β
N
~
]
T and T
=
[
t
1
,
…
,
t
N
]
T are output weight matrix and target matrix, respectively.The ELM algorithm can be summarized as three steps: (1) generate parametersw
i and b
i randomly for i
=
1
,
…
,
N
~; (2) calculate the hidden layer output matrix H; and (3) calculate the output weight using β
=
H
†
T. It has been shown in [28] that any continuous target functions in R
n can be universally approximated using single SLFN with randomly chosen additive hidden nodes.


## 2.2. Voting-Based Extreme Learning Machine

In ELM, randomized hidden nodes are used and remain unchanged during the training. Some testing samples could be misclassified in certain situations, for example, when they are near the classification boundary. To tackle this issue, V-ELM incorporates multiple individual ELMs and makes decisions with majority voting. V-ELM uses a fixed number of hidden nodes for all individual ELMs. All these ELMs are trained with the same dataset and the learning parameters of each ELM are randomly initialized. The predicted class label is then determined by majority voting on all results obtained from ELMs.


## 3. Evolutionary Voting-Based ELM

Given a learning setL consisting of samples {
(
x
n
,
y
n
)
}, n
=
1,2
,
…
,
N, where y
n is the class label. We assume that x is the input and y is predicted by φ
(
x
,
L
). In V-ELM, the aim is to better predict y using multiple ELMs than a single one. Suppose that φ
(
x
,
L
) predicts a class label j
∈
{
1,2
,
…
,
C
} and the prediction of kth classifier is D
k
,
j
∈
{
0,1
} where k
=
1,2
,
…
,
K, the ensemble decision can be defined as
(3)
φ
(
x
,
L
)
=
arg
max
⁡
j
=
1
C
∑
k
=
1
K
‍
D
k
,
j
.
The voting is plurality version which means that the output is the value with highest number of votes whether or not the sum of votes exceeds half.In many applications, not all the classifiers contribute equally to decision making. The overall performance of the ensemble system is able to be improved by weighing the decisions prior to combination [27]. In this section, an evolutionary voting-based ELM (EV-ELM) using weighted majority voting is proposed. The general algorithm is elaborated in Algorithm 1.Algorithm 1:Evolutionary voting based ELM.
Inputs: Training samples L
=
{
(
x
n
,
y
n
)
}
,
n
=
1
,
…
,
N with labels y
n
∈
{
1
,
…
,
C
}; P
(
t
): Population
of candidates at generationt; p: Population size of each generation; p
c: Crossover probability;
p
m: Mutation probability
Initialization: Set t
=
0 and initialize the population P
(
t
) at random.
Evolutionary Process: Evaluate the fitness values of P
(
t
) using (5). Increase t with 1 in each
iteration. To create a new generationP
(
t
+
1
), operations of selection, crossover and mutation
onP
(
t
) are used. Repeat the following steps until the termination criteria of genetic algorithm
(GA) is met.
(i) Firstly,(
1
-
p
c
)
p members of P
(
t
) are probabilistically selected to P
(
t
+
1
) according to the
fitness.
(ii) Secondly, the crossover operator is applied to half of not selected candidates inP
(
t
). The
offsprings after crossover are added toP
(
t
+
1
).
(iii) Lastly, a number of chromosomes with a probability ofp
m in P
(
t
+
1
) are subject to mutation.
Store weights{
ω
1
opt
,
…
,
ω
K
opt
} as the outputs where “opt” indicates “optimal”.
Decision Making: Given a testing sample x, use the following equation to predict its label
φ
ω
(
x
,
L
)
=
arg
max
⁡
j
=
1
C
⁡
∑
k
=
1
K
ω
k
opt
D
k
,
jWe denoteω
k as the weights for kth individual ELM. The mathematical representation of weighted voting algorithm is shown as
(4)
φ
ω
(
x
,
L
)
=
arg
max
⁡
j
=
1
C
∑
k
=
1
K
‍
ω
k
D
k
,
j
.
In the framework of weighted majority voting algorithm, the weight ω
k needs to be optimized to improve the generalization performance. If we know certain classifiers working better, we are able to assign larger weights to the corresponding ones. However, such knowledge is usually absent.Conventional parameter updating methods find the optimal weights to provide better generalization performance. But the optimization process has the risk of getting the local minima and maxima. Methods that discover global optimum can be implemented to further improve classification accuracy. Genetic algorithm (GA) [29] is a class of optimization procedures inspired by the biological mechanisms of reproduction. Many applications utilize the advantages of GA to find optimal solutions, for example, face recognition [30] and clustering techniques [31]. GA is implemented in this paper for demonstration purpose. In practice, many new emerging techniques are potential alternatives such as differential evolution [32] and particle swarm optimization [33].In order to use GA to select proper weights, the chromosomes are formed byω
1
,
ω
2
,
…
,
ω
K. At the beginning, a population of N
c chromosomes is generated randomly. Then, the fitness function f is calculated for each chromosome as
(5)
f
=
∑
k
=
1
K
ω
k
A
k
∑
k
=
1
K
ω
k
,
where ω
k and A
k are the weight and training accuracy for the kth ELM. Maximizing the fitness implies that by choosing appropriate weights we are able to achieve the best normalized training accuracy across all K ELM classifiers in the ensemble. Such a set of optimal weights provide the decision ensemble good generalization performance on unseen testing samples. The fitness function is the most important measurement to determine the composition of the next generation and to guide the entire evolutionary process.After chromosome selection, parts of the current population are inherited into the next generation. The remaining strings are reproduced and some of the parent chromosomes have undergone crossover operation. The crossover probability is defined asp
c. To extend the search space to find global optimum, mutation is applied to some of offsprings randomly with a very small mutation probability p
m. The mutation will introduce a degree of diversity to the population and prevent a premature convergence. The evolutionary process will be terminated after N
g generations, which are considered enough for convergence.


## 4. Performance Evaluation

Experiments were carried out in MATLAB 7 environment under a desktop equipped with Intel 3.2 GHz CPU and 4 G RAM. The learning and testing processes were repeated 50 times and the mean and standard deviations were reported in results. In the experiments, the range of weight was[
exp
⁡
(
0
)
,
exp
⁡
(
1
)
] where the exponential function was used to enhance the difference between lower-bound and upper-bound values of the weight. Moreover, the crossover probability p
c and mutation probability p
m were chosen as 0.65 and 0.004, respectively. The population size N
c was 100 and the number of generations in GA was 150 for all experiments.


### 4.1. Databases

We evaluated the proposed EV-ELM with two types of data: UCI machine learning data and face recognition data. The UCI data was used to test the methods in general-purpose classification problems; the face data was able to examine how good the methods were at handling data with the problem of small sample size (i.e., each pattern class had only a few samples). Details of these databases are presented in Table1. A total of 12 real world datasets were downloaded from the UCI database [34]. Five benchmark face databases were used, namely, ORL [35], UMIST [36], Yale [37], FERET [38], and Georgia Tech face database (GTFD) [39]. ORL, UMIST, and Yale formed a combo database for testing. The combo set consisted of training samples and 575 testing samples in total, and all images belonged to 75 different classes with large variations of illumination, poses, and facial expressions. The FERET database used in this paper was a preprocessed subset [40] composed of 2713 images from 320 subjects. In GTFD, each of 50 subjects had 15 images. Before the experimental evaluation, we cropped and resized images in Yale and GTFD databases to 112 × 92 to make their dimensions identical to those of samples in ORL and UMIST. Furthermore, we applied the discrete cosine transform (DCT) [41] to convert 2D face images to low-dimensional vectors of DCT coefficients.Table 1
Databases used in the experiments.
Database
Number of training data
Number of testing data
Number of attributes
Number of classes
UCI
Balance
400
225
4
3
Diabetes
576
192
8
2
Digit
7494
3498
16
10
Hayes
132
28
4
3
Heart
100
170
13
2
Iris
100
50
4
3
Monk1
124
432
6
2
Monk2
169
432
6
2
Monk3
122
432
6
2
Sonar
100
108
60
2
Waveform
3000
2000
21
3
Wine
100
78
13
3
Face
Combo
555
575
81
75
FERET
1280
1433
81
320
GTFD
400
350
81
50


### 4.2. Results and Discussion

Table2 presents the comparison results where both training time and testing accuracy are averaged across 50 repeats of the evaluation process. It is shown that ELM is the fastest learner but performs poorly in classification. V-ELM and EV-ELM achieve much better performance. Since both V-ELM and EV-ELM create an ensemble of individual ELM classifiers, they run slower compared with ELM unless a parallel computing structure is implemented. In all databases, EV-ELM outperforms V-ELM in terms of both accuracy and variance. However, this improvement is not as much as that between voting-based methods and the original ELM algorithm. The results also show that the evolutionary weighing method is able to increase classification accuracy while bringing down the variance. In general, EV-ELM needs more time than V-ELM to train a model. When in an application that online training is not required, EV-ELM is a good alternative to V-ELM.Table 2
Comparison results among ELM, V-ELM, and EV-ELM algorithms using benchmark UCI and face databases.
Database
Algorithms
Number of nodes
Number of ensembles
Training time (s)
Testing accuracy (%)
Standard deviation (%)
UCI
Balance
ELM
100
NA
0.06
89.56
1.35
V-ELM
100
20
1.22
90.24
0.68
EV-ELM
100
20
3.16
90.84
0.67
Diabetes
ELM
30
NA
0.06
76.65
1.73
V-ELM
30
20
0.35
78.67
0.69
EV-ELM
30
20
2.27
79.18
0.66
Digit
ELM
200
NA
1.15
96.98
0.23
V-ELM
200
20
28.51
97.18
0.08
EV-ELM
200
20
30.77
97.30
0.07
Hayes
ELM
60
NA
<0.01
74.64
5.63
V-ELM
60
20
0.41
78.42
3.58
EV-ELM
60
20
2.51
79.93
3.21
Heart
ELM
20
NA
<0.01
79.01
2.82
V-ELM
20
20
0.19
82.14
1.25
EV-ELM
20
20
1.99
82.78
1.12
Iris
ELM
20
NA
<0.01
97.20
1.39
V-ELM
20
20
0.12
98.00
0.70
EV-ELM
20
20
2.16
98.00
0.39
Monk1
ELM
100
NA
0.05
78.15
2.63
V-ELM
100
20
0.79
85.79
1.37
EV-ELM
100
20
2.82
87.23
1.17
Monk2
ELM
100
NA
0.06
78.84
2.07
V-ELM
100
20
0.91
83.51
0.82
EV-ELM
100
20
3.21
83.81
0.75
Monk3
ELM
100
NA
0.06
80.62
3.13
V-ELM
100
20
0.68
89.00
1.03
EV-ELM
100
20
2.68
89.48
0.97
Sonar
ELM
60
NA
<0.01
77.25
3.44
V-ELM
60
20
0.29
86.79
1.95
EV-ELM
60
20
2.23
87.17
1.77
Waveform
ELM
200
NA
0.44
84.87
0.52
V-ELM
200
20
12.71
86.44
0.21
EV-ELM
200
20
14.77
86.60
0.19
Wine
ELM
20
NA
<0.01
97.31
1.50
V-ELM
20
20
0.12
98.89
0.85
EV-ELM
20
20
2.21
99.36
0.69
Face
Combo
ELM
200
NA
0.21
84.88
1.06
V-ELM
200
20
5.16
87.80
0.60
EV-ELM
200
20
7.58
88.42
0.59
FERET
ELM
100
NA
0.12
41.85
0.80
V-ELM
100
20
7.19
49.06
0.49
EV-ELM
100
20
13.95
49.20
0.47
GTFD
ELM
200
NA
0.19
54.86
2.01
V-ELM
200
20
4.60
66.91
1.39
EV-ELM
200
20
6.77
67.14
1.26Figure1 illustrates five examples on the changes in classification performance during the evolutionary process. Within 150 generations, the GA process is usually able to converge. Reflected in the figure, the classification accuracy tends to become stable toward the end of the evolutionary process. The selected weights after 150 generations cannot guarantee achieving the best classification accuracy, but they are able to provide a consistent output. This characteristic is important as testing accuracy is not available during classifier training, and it cannot be used to guide the evolutionary process. Figure 2 depicts the changes of 3 weights as examples during the GA evolution. Initially, these weights are randomly generated. With the increasing of generations, they tend to converge to either upper- or lower-bound values so that the fitness is maximized. The variety among the weights creates a dynamic ELM ensemble for decision making.Five examples show the changes in classification performance during the evolutionary process. Thex-axis is the number of generations and the y-axis is classification accuracy during each generation. The results are based on UCI Heart dataset.
(a)
(b)
(c)
(d)
(e)Figure 2
There example weights and their value changes in value changes during the evolutionary process. The results are based on UCI Heart dataset.


## 4.1. Databases

We evaluated the proposed EV-ELM with two types of data: UCI machine learning data and face recognition data. The UCI data was used to test the methods in general-purpose classification problems; the face data was able to examine how good the methods were at handling data with the problem of small sample size (i.e., each pattern class had only a few samples). Details of these databases are presented in Table1. A total of 12 real world datasets were downloaded from the UCI database [34]. Five benchmark face databases were used, namely, ORL [35], UMIST [36], Yale [37], FERET [38], and Georgia Tech face database (GTFD) [39]. ORL, UMIST, and Yale formed a combo database for testing. The combo set consisted of training samples and 575 testing samples in total, and all images belonged to 75 different classes with large variations of illumination, poses, and facial expressions. The FERET database used in this paper was a preprocessed subset [40] composed of 2713 images from 320 subjects. In GTFD, each of 50 subjects had 15 images. Before the experimental evaluation, we cropped and resized images in Yale and GTFD databases to 112 × 92 to make their dimensions identical to those of samples in ORL and UMIST. Furthermore, we applied the discrete cosine transform (DCT) [41] to convert 2D face images to low-dimensional vectors of DCT coefficients.Table 1
Databases used in the experiments.
Database
Number of training data
Number of testing data
Number of attributes
Number of classes
UCI
Balance
400
225
4
3
Diabetes
576
192
8
2
Digit
7494
3498
16
10
Hayes
132
28
4
3
Heart
100
170
13
2
Iris
100
50
4
3
Monk1
124
432
6
2
Monk2
169
432
6
2
Monk3
122
432
6
2
Sonar
100
108
60
2
Waveform
3000
2000
21
3
Wine
100
78
13
3
Face
Combo
555
575
81
75
FERET
1280
1433
81
320
GTFD
400
350
81
50


## 4.2. Results and Discussion

Table2 presents the comparison results where both training time and testing accuracy are averaged across 50 repeats of the evaluation process. It is shown that ELM is the fastest learner but performs poorly in classification. V-ELM and EV-ELM achieve much better performance. Since both V-ELM and EV-ELM create an ensemble of individual ELM classifiers, they run slower compared with ELM unless a parallel computing structure is implemented. In all databases, EV-ELM outperforms V-ELM in terms of both accuracy and variance. However, this improvement is not as much as that between voting-based methods and the original ELM algorithm. The results also show that the evolutionary weighing method is able to increase classification accuracy while bringing down the variance. In general, EV-ELM needs more time than V-ELM to train a model. When in an application that online training is not required, EV-ELM is a good alternative to V-ELM.Table 2
Comparison results among ELM, V-ELM, and EV-ELM algorithms using benchmark UCI and face databases.
Database
Algorithms
Number of nodes
Number of ensembles
Training time (s)
Testing accuracy (%)
Standard deviation (%)
UCI
Balance
ELM
100
NA
0.06
89.56
1.35
V-ELM
100
20
1.22
90.24
0.68
EV-ELM
100
20
3.16
90.84
0.67
Diabetes
ELM
30
NA
0.06
76.65
1.73
V-ELM
30
20
0.35
78.67
0.69
EV-ELM
30
20
2.27
79.18
0.66
Digit
ELM
200
NA
1.15
96.98
0.23
V-ELM
200
20
28.51
97.18
0.08
EV-ELM
200
20
30.77
97.30
0.07
Hayes
ELM
60
NA
<0.01
74.64
5.63
V-ELM
60
20
0.41
78.42
3.58
EV-ELM
60
20
2.51
79.93
3.21
Heart
ELM
20
NA
<0.01
79.01
2.82
V-ELM
20
20
0.19
82.14
1.25
EV-ELM
20
20
1.99
82.78
1.12
Iris
ELM
20
NA
<0.01
97.20
1.39
V-ELM
20
20
0.12
98.00
0.70
EV-ELM
20
20
2.16
98.00
0.39
Monk1
ELM
100
NA
0.05
78.15
2.63
V-ELM
100
20
0.79
85.79
1.37
EV-ELM
100
20
2.82
87.23
1.17
Monk2
ELM
100
NA
0.06
78.84
2.07
V-ELM
100
20
0.91
83.51
0.82
EV-ELM
100
20
3.21
83.81
0.75
Monk3
ELM
100
NA
0.06
80.62
3.13
V-ELM
100
20
0.68
89.00
1.03
EV-ELM
100
20
2.68
89.48
0.97
Sonar
ELM
60
NA
<0.01
77.25
3.44
V-ELM
60
20
0.29
86.79
1.95
EV-ELM
60
20
2.23
87.17
1.77
Waveform
ELM
200
NA
0.44
84.87
0.52
V-ELM
200
20
12.71
86.44
0.21
EV-ELM
200
20
14.77
86.60
0.19
Wine
ELM
20
NA
<0.01
97.31
1.50
V-ELM
20
20
0.12
98.89
0.85
EV-ELM
20
20
2.21
99.36
0.69
Face
Combo
ELM
200
NA
0.21
84.88
1.06
V-ELM
200
20
5.16
87.80
0.60
EV-ELM
200
20
7.58
88.42
0.59
FERET
ELM
100
NA
0.12
41.85
0.80
V-ELM
100
20
7.19
49.06
0.49
EV-ELM
100
20
13.95
49.20
0.47
GTFD
ELM
200
NA
0.19
54.86
2.01
V-ELM
200
20
4.60
66.91
1.39
EV-ELM
200
20
6.77
67.14
1.26Figure1 illustrates five examples on the changes in classification performance during the evolutionary process. Within 150 generations, the GA process is usually able to converge. Reflected in the figure, the classification accuracy tends to become stable toward the end of the evolutionary process. The selected weights after 150 generations cannot guarantee achieving the best classification accuracy, but they are able to provide a consistent output. This characteristic is important as testing accuracy is not available during classifier training, and it cannot be used to guide the evolutionary process. Figure 2 depicts the changes of 3 weights as examples during the GA evolution. Initially, these weights are randomly generated. With the increasing of generations, they tend to converge to either upper- or lower-bound values so that the fitness is maximized. The variety among the weights creates a dynamic ELM ensemble for decision making.Five examples show the changes in classification performance during the evolutionary process. Thex-axis is the number of generations and the y-axis is classification accuracy during each generation. The results are based on UCI Heart dataset.
(a)
(b)
(c)
(d)
(e)Figure 2
There example weights and their value changes in value changes during the evolutionary process. The results are based on UCI Heart dataset.


## 5. Conclusions

In this paper, we proposed an enhanced V-ELM method. Weights were introduced to distinguish the difference among various individual ELM classifiers and the genetic algorithm was used for optimization. Experimental results demonstrated the effectiveness of EV-ELM in terms of classification accuracy. However, slow training speed prohibits the use of EV-ELM in applications that require online training. This study is a preliminary research on optimizing ELM ensembles; many evolutionary algorithms are potentially useful in optimizing weights. Furthermore, reducing training time is of great interest in future work.



---

*Source: 808292-2014-08-14.xml*