# An Improved Teaching-Learning-Based Optimization with the Social Character of PSO for Global Optimization

Authors: Feng Zou; Debao Chen; Jiangtao Wang
Journal: Computational Intelligence and Neuroscience
 (2016)

---

## Abstract

An improved teaching-learning-based optimization with combining of the social character of PSO (TLBO-PSO), which is considering the teacher’s behavior influence on the students and the mean grade of the class, is proposed in the paper to find the global solutions of function optimization problems. In this method, the teacher phase of TLBO is modified; the new position of the individual is determined by the old position, the mean position, and the best position of current generation. The method overcomes disadvantage that the evolution of the original TLBO might stop when the mean position of students equals the position of the teacher. To decrease the computation cost of the algorithm, the process of removing the duplicate individual in original TLBO is not adopted in the improved algorithm. Moreover, the probability of local convergence of the improved method is decreased by the mutation operator. The effectiveness of the proposed method is tested on some benchmark functions, and the results are competitive with respect to some other methods.


---

## Body

## 1. Introduction

Global optimization is a concerned research area in science and engineering. Many real-world optimization applications can be formulated as global optimization problems. To efficiently solve the global optimization problems, the efficient and robust optimization algorithms are needed. The traditional methods often fail to solve complex global optimization problems. A detailed overview of the research progress in deterministic global optimization can be found in [1]. To overcome the difficulties of traditional methods, some well-known metaheuristics are developed for solving global optimization problems during the last four decades. Among existing metaheuristics, particle swarm optimization (PSO) algorithm [2] plays very important role in solving global optimization problems. PSO inspired by the social behaviors of birds has been successfully utilized to optimize continuous nonlinear functions [3], but the standard PSO algorithm (SPSO) might trap into local optima when solving complex multimodal problems. To improve the global performance of PSO, some variants are developed. Linear decreasing inertia weight particle swarm optimization (LDWPSO) [4] was introduced by Shi and Eberhart to overcome the lack of velocity control in standard PSO; the inertia weight of the algorithm decreases from a large value to a small one with increasing of evolution. To improve the convergence accuracy of PSO, some modified operators are adopted to help the swarm escape from the local optima especially when the best fitness of the swarm is not changed in some continuous iteration [5–8]. To improve the convergence speed of PSO, some modifications for updating rule of particles were presented in [9], and the improved algorithm was tested on some benchmark functions. Adaptive particle swarm optimization algorithm was introduced by Zhan to improve the performance of PSO, many operators were proposed to help the swarm jump out of the local optima and the algorithms have been evaluated on 12 benchmark functions [10, 11]. Some detail surveys of development to PSO are introduced for the interested readers in [12–14].Though swarm intelligence optimization algorithms have been successfully used to solve global optimization problems, the main limitation of the previously mentioned algorithms is that many parameters (often more than two parameters) should be determined in updating process of individuals, and the efficiency of algorithms are usually affected by these parameters. For example, there are three parameters (w, c
1, and c
2) that should be determined in updating equations of PSO. Moreover, the optimal parameters of the algorithms are often difficult to be determined. To decrease the effects of parameters for the algorithms, teaching-learning-based (TLBO) algorithm [15] is proposed recently, and it has been used in some real applications [16–19]. Some variants of the TLBO algorithm have been presented for improving the performance of the original TLBO [20, 21].Under the framework of population-based optimizations, many variations of evolutionary optimization algorithms have been designed. Each of these algorithms performs well in certain cases and none of them are dominating one another. The key reason for employing the hybridization is that the hybrid algorithm can take advantage of the strengths of each individual technique while simultaneously overcoming its main limitations. On top of this idea, many hybrid algorithms have been presented [22–27]. In the paper, to improve the performance of TLBO algorithm for solving global optimization problems, an improved TLBO algorithm with combining of the social character of PSO, named TLBO-PSO, is proposed. In the improved TLBO algorithm, the teacher improves not only the performance of the mean grade of the whole class but also the performance of every student. The proposed algorithm has been evaluated on some benchmark functions, and the results are compared with some other algorithms.The paper is organized as follows. Section2 provides a brief description of the standard PSO algorithm. Original teaching-learning-based (TLBO) algorithm is introduced in Section 3. Section 4 provides the detail procedure of the improved teaching-learning-based optimization algorithm with combining of the social character of PSO (TLBO-PSO). Some experiments are given in Section 5. Section 6 concludes the paper.


## 2. Particle Swarm Optimizers

In the standard PSO (SPSO) algorithm, a swarm of particles are represented as potential solutions; each particle searches for an optimal solution to the objective function in the search space. Theith particle is associated with two vectors, that is, the velocity vector V
i
=
[
v
i
1
,
v
i
2
,
…
,
v
i
D
] and the position vector X
i
=
[
x
i
1
,
x
i
2
,
…
,
x
i
D
], where D is the dimensions of the variable. Each particle dynamically updates its position in a limited searching domain according to the best position of current iteration and the best position which it has achieved so far. The velocity and position of the ith particle are updated as follows:(1)
v
i
d
=
w
v
i
d
+
c
1
r
1
P
b
e
s
t
i
d
-
x
i
d
+
c
2
r
2
G
b
e
s
t
d
-
x
i
d
,
(2)
x
i
d
=
x
i
d
+
v
i
d
,where c
1 and c
2 are the acceleration coefficients, the value of them often equals 2, r
1 and r
2 are random numbers between 0 and 1. w is the inertia weight which influences the convergent character of the swarm. Large inertial weight benefits global searching performance, while small one facilitates local searching. x
i
d is the position of the ith particle on the dth dimension, P
b
e
s
t
i
d is the best position which the ith has achieved so far, and G
b
e
s
t
d is the best position of the swarm in current iteration. In general, the velocity of all the particles is limited by the maximum velocity (V
m
a
x), and positions of them are limited by the maximum position (P
m
a
x) and the minimum position (P
m
i
n). To improve the performance of PSO, the method LDWPSO in which the inertia weight decreases linearly from a relatively large value to a small one is proposed [4], and the changed weight with evolution iteration is shown as follows:(3)
w
=
w
m
a
x
-
g
e
n
∗
w
m
a
x
-
w
m
i
n
max
⁡
g
e
n
,where w
m
a
x
=
0.9, w
m
i
n
=
0.4 are the maximum and minimum values of inertia weight, respectively. gen is the current generation, max
⁡
g
e
n is the maximum evolutionary iteration. The adaptive weights make swarm have good global searching ability at the beginning of iterations and good local searching ability near the end of runs. Algorithm 1 presents the detail steps of LDWPSO algorithm.Algorithm 1:LDWPSO algorithm.
Initialize:
Initializemax
⁡
gen, w
max
⁡, w
min
⁡, V
m
a
x, P
m
a
x, P
m
i
n, Population size (Ps), Dimension size (D) and the initial swarm
Optimize:
for gen = 1 :max
⁡
gen
Calculate thethe inertia weight w according to (3)
fori
=
1 : Ps
Calculate fitness of all particles;
Calculate the best positionG
b
e
s
t
d of current generation;
Calculate the best positionP
b
e
s
t
i
d of ith particle which it has achieved so far;
ford
=
1 : D
Update the velocity of ith particle according to (1)
If v
i
d
>
V
m
a
x then v
i
d
=
V
m
a
x
If v
i
d
<
-
V
m
a
x then v
i
d
=
-
V
m
a
x
Update the position of ith particle according to (2)
If x
i
d
>
P
m
a
x then x
i
d
=
P
m
a
x
If x
i
d
<
P
m
i
n then x
i
d
=
P
m
i
n
end
end
end


## 3. Teaching-Learning-Based Optimization (TLBO) Algorithm

TLBO is one of the recently proposed population-based algorithms and it simulates the teaching-learning process of the class [15]. The main idea of the algorithm is based on the influence of a teacher on the output of learners in a class and the interaction between the learners. The TLBO algorithm does not require any specific parameters. TLBO requires only common controlling parameters like population size and number of generations for its working. The use of teaching in the algorithm is to improve the average grades of the class. The algorithm contains two phases: teacher phase and learner phase. The detail description of the algorithm can be found in [15]. In this paper, only the two main phases of TLBO are introduced as follows.


### 3.1. Teacher Phase

In the teacher phase of TLBO algorithm, the task of the teacher is to increase the mean grades of the class. Suppose that an objective function isf
(
X
) with n-dimensional variables, the ith student can be represented as X
i
=
[
x
i
1
,
x
i
2
,
…
,
x
i
n
]. At any iteration g, assume that the population size of the class is m, the mean result of students in current iteration is X
g
m
e
a
n
=
1
/
m
[
∑
i
=
1
m
x
i
1
,
∑
i
=
1
m
x
i
2
,
…
,
∑
i
=
1
m
x
i
n
]. The student with the best fitness is chosen as the teacher of current iteration; it is represented as X
t
e
a
c
h
e
r. All the students will update their position as follows:(4)
X
i
,
n
e
w
=
X
i
,
o
l
d
+
r
1
X
t
e
a
c
h
e
r
-
T
F
X
g
m
e
a
n
,where X
i
,
n
e
w and X
i
,
o
l
d are the new and the old position of the ith student and r
1 is the random number in the range [
0,1
]. If the new solution is better than the old one, the old position of individual will be replaced by the new position. The value of T
F is randomly decided by the algorithm according to(5)
T
F
=
r
o
u
n
d
1
+
rand
⁡
0,1
2
-
1
.


### 3.2. Learner Phase

In learner phase of TLBO algorithm, learners can increase their knowledge from others. A learner interacts randomly with other learners for enhancing his or her knowledge. The learning of learner phase can be expressed as follows. For theith individual X
i in the jth generation, randomly select the kth individual X
k which is different from X
i, and the updated formula of X
i is defined in (6) and (7).IfX
i is better than X
k according to their fitness, then (6)
X
i
,
n
e
w
=
X
i
,
o
l
d
+
r
i
X
i
-
X
k
.Else(7)
X
i
,
n
e
w
=
X
i
,
o
l
d
+
r
i
X
k
-
X
i
,where r
i is the random number in the range [
0,1
].If the new positionX
i
,
n
e
w is better than the old one X
i
,
o
l
d, the old position X
i
,
o
l
d is replaced by the new X
i
,
n
e
w; otherwise, the position of the ith individual is not changed. The detail algorithm is shown in Algorithm 2.Algorithm 2:TLBO().
InitializeN (number of learners) and D (number of dimensions)
Initialize learnersX and evaluate all learners X
Donate the best learner asX
teacher and the mean of all learners X as X
g
mean
while(stopping condition not met)
for each learner X
i of the class % Teaching phase
T
F = round(1 + rand(0,1))
X
i
,
new
=
X
i
,
old
+
r
1
X
teacher
-
T
F
X
g
mean
Accept X
i
,
new if f
(
X
i
,
new
) is better than f
(
X
i
,
old
)
endfor
for each learner X
i of the class % Learning phase
Randomly select one learner X
k, such that i
≠
k
if
f
(
X
i
) better f
(
X
k
)
X
i
,
new
=
X
i
,
old
+
r
i
X
i
-
X
k
else
X
i
,
new
=
X
i
,
old
+
r
i
X
k
-
X
i
endif
Accept X
i
,
new if f
(
X
i
,
new
) is better than f
(
X
i
,
old
)
endfor
Update X
teacher and X
g
mean
endwhile


## 3.1. Teacher Phase

In the teacher phase of TLBO algorithm, the task of the teacher is to increase the mean grades of the class. Suppose that an objective function isf
(
X
) with n-dimensional variables, the ith student can be represented as X
i
=
[
x
i
1
,
x
i
2
,
…
,
x
i
n
]. At any iteration g, assume that the population size of the class is m, the mean result of students in current iteration is X
g
m
e
a
n
=
1
/
m
[
∑
i
=
1
m
x
i
1
,
∑
i
=
1
m
x
i
2
,
…
,
∑
i
=
1
m
x
i
n
]. The student with the best fitness is chosen as the teacher of current iteration; it is represented as X
t
e
a
c
h
e
r. All the students will update their position as follows:(4)
X
i
,
n
e
w
=
X
i
,
o
l
d
+
r
1
X
t
e
a
c
h
e
r
-
T
F
X
g
m
e
a
n
,where X
i
,
n
e
w and X
i
,
o
l
d are the new and the old position of the ith student and r
1 is the random number in the range [
0,1
]. If the new solution is better than the old one, the old position of individual will be replaced by the new position. The value of T
F is randomly decided by the algorithm according to(5)
T
F
=
r
o
u
n
d
1
+
rand
⁡
0,1
2
-
1
.


## 3.2. Learner Phase

In learner phase of TLBO algorithm, learners can increase their knowledge from others. A learner interacts randomly with other learners for enhancing his or her knowledge. The learning of learner phase can be expressed as follows. For theith individual X
i in the jth generation, randomly select the kth individual X
k which is different from X
i, and the updated formula of X
i is defined in (6) and (7).IfX
i is better than X
k according to their fitness, then (6)
X
i
,
n
e
w
=
X
i
,
o
l
d
+
r
i
X
i
-
X
k
.Else(7)
X
i
,
n
e
w
=
X
i
,
o
l
d
+
r
i
X
k
-
X
i
,where r
i is the random number in the range [
0,1
].If the new positionX
i
,
n
e
w is better than the old one X
i
,
o
l
d, the old position X
i
,
o
l
d is replaced by the new X
i
,
n
e
w; otherwise, the position of the ith individual is not changed. The detail algorithm is shown in Algorithm 2.Algorithm 2:TLBO().
InitializeN (number of learners) and D (number of dimensions)
Initialize learnersX and evaluate all learners X
Donate the best learner asX
teacher and the mean of all learners X as X
g
mean
while(stopping condition not met)
for each learner X
i of the class % Teaching phase
T
F = round(1 + rand(0,1))
X
i
,
new
=
X
i
,
old
+
r
1
X
teacher
-
T
F
X
g
mean
Accept X
i
,
new if f
(
X
i
,
new
) is better than f
(
X
i
,
old
)
endfor
for each learner X
i of the class % Learning phase
Randomly select one learner X
k, such that i
≠
k
if
f
(
X
i
) better f
(
X
k
)
X
i
,
new
=
X
i
,
old
+
r
i
X
i
-
X
k
else
X
i
,
new
=
X
i
,
old
+
r
i
X
k
-
X
i
endif
Accept X
i
,
new if f
(
X
i
,
new
) is better than f
(
X
i
,
old
)
endfor
Update X
teacher and X
g
mean
endwhile


## 4. Teaching-Learning-Based Optimization Algorithm with PSO (TLBO-PSO)

### 4.1. The Main Idea of TLBO-PSO

As previously reviewed, the main idea of TLBO is to imitate the teaching-learning process in a class. The teacher tries to disseminate knowledge among the learners to increase the knowledge level of the whole class, and the learners also study knowledge from the others to improve its grade. Algorithm displays that all individuals update their positions based on the distance between one or two times of the mean solution and the teacher in teacher phase. An individual also renewed its position based on the distance between it and a randomly selected individual from the class. Equation (4) indicates that the teacher only improves the grades of the students by using the mean grade of the class; the distance between the teacher and the students is not considered in teacher phase. In PSO, the difference between the current best individual and the individual can help the individual improve its performance. Based on this idea, the method in PSO is introduced into TLBO to improve the learning efficiency of the TLBO algorithm. The main change for TLBO is represented in updating equation. Equation (4) in TLBO is modified as(8)
X
i
,
n
e
w
=
X
i
,
o
l
d
+
r
1
X
t
e
a
c
h
e
r
-
T
F
X
g
m
e
a
n
+
r
2
X
t
e
a
c
h
e
r
-
X
i
,
o
l
d
,where r
1, r
2 are the random number in the range [
0
,
1
]. With this modification, the performance of TLBO algorithm might be improved.In original TLBO algorithm, all genes of each individual should be compared with those of all other individuals for removing the duplicate individual. This operator is a heavy computation cost process, and the function evaluations required are not clearly known. In our opinion, duplication testing is not needed in every generation especially in the beginning of evolution. Assume that individualsi and k have the same genes in t generation, and the new position of these two individuals might be different when T
F is a random number (1 or 2). At the beginning of evolution, all individuals generate better positions easily. Random operator for T
F may benefit for maintaining diversity of class, but, in the anaphase of evolution, the positions of individuals might be close to each other. When the mean solution equals the best solution, the individual does not change (T
F
=
1) or the large change of genes might destroy the individual in large degree (T
F
=
2) so that the better individual is difficult to be generated. To decrease the computational effort of comparing all the individuals, the removing of the duplicate individual process in the original TLBO is deleted in the improved TLBO algorithm and a mutation operator according to the best fitness of successive generations is introduced in the paper. If the best fitness of continuous n generations is not changed or changed slightly, an individual will be randomly selected according to mutation possibility p
c to be mutated. To maintain the global performance of the algorithm, the best individual is not mutated. The subroutine for the mutation is described as shown in Algorithm 3.Algorithm 3:Subroutine for mutation.
pop = sort(pop);
if abs(bestfit(gen) − bestfit(gen − 1)) <ε then m
=
m
+
1; else m
=
0;
if (m == n)
m
=
0;
forI
=
2 : popsize
if rand(1) < p
c
k = ceil(rand(1) ∗ dimsize);
pop(i
,
k) = pop(i
,
k) + α ∗ rands(1,1);
if pop(i
,
k) > x
max
⁡ then pop(
i
,
k) = x
max
⁡;
if pop(i
,
k) < x
min
⁡ then pop(i
,
k) = x
min
⁡;
end
end
end
endIn Algorithm3, n is the setting generation, p
c is the mutation possibility, and ε is a small number which is given by the designer. α is a mutation parameter between 0.01 and 0.1.


### 4.2. The Steps of TLBO-PSO

Step 1.
Set the maximumX
m
a
x and minimum X
m
i
n of position, the maximal evolution generation g
e
n
m
a
x, mutation possibility p
c and mutation parameter α, the population size popsize, and the dimension size of the task. Initialize the initial population pop as follows:(9)
p
o
p
=
X
m
i
n
+
r
∗
X
m
a
x
-
X
m
i
n
,where r is the random number in the range [
0,1
].Step 2.
Evaluate the individual, select the best individualX
t
e
a
c
h
e
r as the teacher, and calculate the mean solution X
g
m
e
a
n of the population.Step 3.
For each individual, update its position according to (8). If X
i
,
n
e
w is better than X
i
,
o
l
d, then X
i
,
o
l
d
=
X
i
,
n
e
w.Step 4.
For each individual, randomly select another individual, update its position according to (6) and (7), and choose the better solution from X
i
,
o
l
d and X
i
,
n
e
w as the new position of the individual.Step 5.
Execute mutation operator for the population according to Algorithm3.Step 6.
If the ended condition of TLBO-PSO is not satisfied, the algorithm will go back to Step2, or it is terminated.


## 4.1. The Main Idea of TLBO-PSO

As previously reviewed, the main idea of TLBO is to imitate the teaching-learning process in a class. The teacher tries to disseminate knowledge among the learners to increase the knowledge level of the whole class, and the learners also study knowledge from the others to improve its grade. Algorithm displays that all individuals update their positions based on the distance between one or two times of the mean solution and the teacher in teacher phase. An individual also renewed its position based on the distance between it and a randomly selected individual from the class. Equation (4) indicates that the teacher only improves the grades of the students by using the mean grade of the class; the distance between the teacher and the students is not considered in teacher phase. In PSO, the difference between the current best individual and the individual can help the individual improve its performance. Based on this idea, the method in PSO is introduced into TLBO to improve the learning efficiency of the TLBO algorithm. The main change for TLBO is represented in updating equation. Equation (4) in TLBO is modified as(8)
X
i
,
n
e
w
=
X
i
,
o
l
d
+
r
1
X
t
e
a
c
h
e
r
-
T
F
X
g
m
e
a
n
+
r
2
X
t
e
a
c
h
e
r
-
X
i
,
o
l
d
,where r
1, r
2 are the random number in the range [
0
,
1
]. With this modification, the performance of TLBO algorithm might be improved.In original TLBO algorithm, all genes of each individual should be compared with those of all other individuals for removing the duplicate individual. This operator is a heavy computation cost process, and the function evaluations required are not clearly known. In our opinion, duplication testing is not needed in every generation especially in the beginning of evolution. Assume that individualsi and k have the same genes in t generation, and the new position of these two individuals might be different when T
F is a random number (1 or 2). At the beginning of evolution, all individuals generate better positions easily. Random operator for T
F may benefit for maintaining diversity of class, but, in the anaphase of evolution, the positions of individuals might be close to each other. When the mean solution equals the best solution, the individual does not change (T
F
=
1) or the large change of genes might destroy the individual in large degree (T
F
=
2) so that the better individual is difficult to be generated. To decrease the computational effort of comparing all the individuals, the removing of the duplicate individual process in the original TLBO is deleted in the improved TLBO algorithm and a mutation operator according to the best fitness of successive generations is introduced in the paper. If the best fitness of continuous n generations is not changed or changed slightly, an individual will be randomly selected according to mutation possibility p
c to be mutated. To maintain the global performance of the algorithm, the best individual is not mutated. The subroutine for the mutation is described as shown in Algorithm 3.Algorithm 3:Subroutine for mutation.
pop = sort(pop);
if abs(bestfit(gen) − bestfit(gen − 1)) <ε then m
=
m
+
1; else m
=
0;
if (m == n)
m
=
0;
forI
=
2 : popsize
if rand(1) < p
c
k = ceil(rand(1) ∗ dimsize);
pop(i
,
k) = pop(i
,
k) + α ∗ rands(1,1);
if pop(i
,
k) > x
max
⁡ then pop(
i
,
k) = x
max
⁡;
if pop(i
,
k) < x
min
⁡ then pop(i
,
k) = x
min
⁡;
end
end
end
endIn Algorithm3, n is the setting generation, p
c is the mutation possibility, and ε is a small number which is given by the designer. α is a mutation parameter between 0.01 and 0.1.


## 4.2. The Steps of TLBO-PSO

Step 1.
Set the maximumX
m
a
x and minimum X
m
i
n of position, the maximal evolution generation g
e
n
m
a
x, mutation possibility p
c and mutation parameter α, the population size popsize, and the dimension size of the task. Initialize the initial population pop as follows:(9)
p
o
p
=
X
m
i
n
+
r
∗
X
m
a
x
-
X
m
i
n
,where r is the random number in the range [
0,1
].Step 2.
Evaluate the individual, select the best individualX
t
e
a
c
h
e
r as the teacher, and calculate the mean solution X
g
m
e
a
n of the population.Step 3.
For each individual, update its position according to (8). If X
i
,
n
e
w is better than X
i
,
o
l
d, then X
i
,
o
l
d
=
X
i
,
n
e
w.Step 4.
For each individual, randomly select another individual, update its position according to (6) and (7), and choose the better solution from X
i
,
o
l
d and X
i
,
n
e
w as the new position of the individual.Step 5.
Execute mutation operator for the population according to Algorithm3.Step 6.
If the ended condition of TLBO-PSO is not satisfied, the algorithm will go back to Step2, or it is terminated.


## 5. Simulation Experiments

To test the performance of the improved TLBO-PSO algorithm, nine benchmark functions are simulated in this section. These nine benchmark functions are listed in Table1. In particular, in order to compare the performance of the proposed TLBO-PSO with variants of TLBO and PSO, CLPSO [22], TLBO [15], and ETLBO [21] are selected and simulated.Table 1
Nine tested functions.
Function
Formula
Range
f
m
i
n
Acceptance
f
1 (Sphere)
f
1
x
=
∑
i
=
1
D
x
i
2
[
-
100,100
]
0
1
E
-
6
f
2 (Quadric)
f
2
x
=
∑
i
=
1
D
∑
j
=
1
i
x
j
2
[
-
100,100
]
0
1
E
-
6
f
3 (Sum Square)
f
3
x
=
∑
i
=
1
D
i
x
i
2
[
-
100,100
]
0
1
E
-
6
f
4 (Zakharov)
f
4
x
=
∑
i
=
1
D
x
i
2
+
∑
i
=
1
D
0.5
i
x
i
2
+
∑
i
=
1
D
0.5
i
x
i
4
[
-
10,10
]
0
1
E
-
6
f
5 (Rosenbrock)
f
5
x
=
∑
i
=
1
D
-
1
100
x
i
2
-
x
i
+
1
2
+
x
i
-
1
2
[
-
2.048,2.048
]
0
1
E
-
6
f
6 (Ackley)
f
6
x
=
20
-
20
exp
⁡
-
1
5
1
D
∑
i
=
1
D
x
i
2
-
exp
⁡
1
D
∑
i
=
1
D
cos
⁡
2
π
x
i
+
e
[
-
32.768,32.768
]
0
1
E
-
6
f
7 (Rastrigin)
f
7
x
=
∑
i
=
1
D
x
i
2
-
10
cos
⁡
2
π
x
i
+
10
[
-
5.12,5.12
]
0
1
E
-
6
f
8 (Weierstrass)
f
8
x
=
∑
i
=
1
D
∑
k
=
0
k
m
a
x
a
k
cos
⁡
2
π
b
k
x
i
+
0.5
-
D
∑
k
=
0
k
m
a
x
a
k
cos
⁡
2
π
b
k
×
0.5
a
=
0.5
b
=
3
k
m
a
x
=
20
[
-
0.5,0.5
]
0
1
E
-
1
f
9 (Griewank)
f
9
x
=
∑
i
=
1
D
x
i
2
4000
-
∏
i
=
1
n
cos
⁡
x
i
i
+
1
[
-
600,600
]
0
1
E
-
1


### 5.1. Parameters Setting

To reduce statistical errors, each function in the paper is independently simulated 30 runs, and their mean results are used in the comparison. The value of function is defined as the fitness function. All the experiments are carried out on the same machine with a Celeron 2.26 GHz CPU, 512-MB memory, and Windows XP operating system with Matlab 7.0. All functions are simulated in 10 and 30 dimensions. The nine functions are summarized in Table1. “Range” is the lower and upper bounds of the variables. “f
m
i
n” is the theory global minimum solution. “Acceptance” is the acceptable solutions of different functions. For CLPSO, TLBO, and ETLBO algorithms, the training parameters are the same as those used in the corresponding references except that the maximal FEs are 50000 and the size of population is 30. The size of elitism is 2 in ETLBO algorithm. When the best fitness is changed to be smaller than ε, the large value of mutation will possibly be chosen. In our experiments, the mutation possibility p
c is 0.6, ε is 0.01, and α is 0.05.


### 5.2. Comparisons on the Solution Accuracy

The performance of different algorithms for 10- and 30-dimensional functions in terms of the best solution, the mean (Mean), and standard deviation (STD) of the solutions obtained in the 30 independent runs is listed in Tables2 and 3. Boldface in the tables indicates the best solution among those obtained by all four contenders.Table 2
The mean best solutions and standard deviations by various methods for 10D functions.
Methods
CLPSO
TLBO
ETLBO
TLBO-PSO
Mean
STD
Mean
STD
Mean
STD
Mean
STD
f
1
4.68
E
-
19
4.90
E
-
19
2.12
E
-
185
0.00
E
+
00
1.51
E
-
167
0.00
E
+
00
1.07
E
-
314
0.00
E
+
00
f
2
6.73
E
-
01
4.40
E
-
01
5.79
E
-
80
1.71
E
-
79
5.10
E
-
78
1.22
E
-
77
9.62
E
-
172
0.00
E
+
00
f
3
1.14
E
-
20
8.02
E
-
21
1.02
E
-
185
0.00
E
+
00
5.91
E
-
169
0.00
E
+
00
2.65
E
-
315
0.00
E
+
00
f
4
3.32
E
-
03
3.25
E
-
03
4.44
E
-
87
1.36
E
-
86
9.99
E
-
85
1.89
E
-
84
5.04
E
-
182
0.00
E
+
00
f
5
2.25
E
+
00
1.00
E
+
00
6.54
E
-
01
6.56
E
-
01
2.76
E
-
01
2.78
E
-
01
4.18
E
+
00
7.46
E
-
01
f
6
3.75
E
-
10
2.83
E
-
10
3.55
E
-
15
0.00
E
+
00
3.20
E
-
15
1.12
E
-
15
2.84
E
-
15
1.50
E
-
15
f
7
1.63
E
-
09
2.42
E
-
09
2.17
E
+
00
1.86
E
+
00
2.33
E
+
00
1.76
E
+
00
5.26
E
+
00
4.63
E
+
00
f
8
1.16
E
-
11
1.55
E
-
11
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
f
9
1.06
E
-
03
2.29
E
-
03
9.96
E
-
03
1.02
E
-
02
8.45
E
-
03
1.61
E
-
02
3.54
E
-
02
5.27
E
-
02Table 3
The mean best solutions and standard deviations by various methods for 30D functions.
Methods
CLPSO
TLBO
ETLBO
TLBO-PSO
Mean
STD
Mean
STD
Mean
STD
Mean
STD
f
1
1.50
E
-
03
5.04
E
-
04
1.63
E
-
152
2.38
E
-
152
5.70
E
-
132
1.13
E
-
131
6.67
E
-
240
0.00
E
+
00
f
2
9.37
E
+
03
2.23
E
+
03
1.78
E
-
33
3.34
E
-
33
4.58
E
-
33
5.40
E
-
33
1.98
E
-
76
4.73
E
-
76
f
3
1.39
E
-
04
4.19
E
-
05
1.45
E
-
152
2.79
E
-
152
7.13
E
-
132
1.78
E
-
131
1.60
E
-
240
0.00
E
+
00
f
4
1.45
E
+
02
4.10
E
+
01
5.72
E
-
18
7.60
E
-
18
1.56
E
-
16
3.00
E
-
16
1.46
E
-
43
3.04
E
-
43
f
5
3.60
E
+
01
1.03
E
+
01
2.42
E
+
01
6.48
E
-
01
2.46
E
+
01
7.50
E
-
01
2.65
E
+
01
1.18
E
+
00
f
6
3.19
E
-
02
1.27
E
-
02
3.55
E
-
15
0.00
E
+
00
3.91
E
-
15
1.12
E
-
15
3.55
E
-
15
0.00
E
+
00
f
7
9.88
E
+
00
2.48
E
+
00
1.31
E
+
01
5.78
E
+
00
1.68
E
+
01
8.12
E
+
00
2.41
E
+
01
1.40
E
+
01
f
8
6.53
E
-
02
1.08
E
-
02
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
f
9
8.76
E
-
03
3.43
E
-
03
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00The results for 10-dimensional functions in Table2 display that the improved TLBO (TLBO-PSO) outperforms all the other algorithms in terms of the mean best solutions and standard deviations (STD) for functions f
1, f
2, f
3, and f
4. For function f
5, ETLBO has the best performance. For function f
6, the mean best solution of TLBO-PSO is the best, and the STD of TLBO is the smallest. For functions f
7 and f
9, CLPSO has the best performance. For function f
8, three TLBOs have the same solutions in terms of the mean best solutions and standard deviations (STD).The results for 30-dimensional functions in Table3 indicate that TLBO-PSO also has the best performance in terms of the mean best solutions and standard deviations (STD) for functions f
1, f
2, f
3, and f
4 with 30 dimensions. TLBO has the best performance for function f
5. For function f
6, TLBO and ETLBO have the same performance in terms of the mean best solutions and standard deviations (STD). For function f
7, the mean best solution of TLBO is the smallest, and the standard deviation of TLBO-PSO is the smallest. Three TLBOs have the same solutions for functions f
8 and f
9.


### 5.3. Comparisons on the Convergence Speed and Successful Ratios

The mean number of function evolutions (FEs) is often used to measure the convergence speed of algorithms. In this paper, the mean value of FEs is used to measure the speed of all algorithms. The average FEs (when the algorithm is globally convergent) of all algorithms for nine functions with 10 and 30 dimensions are shown in Tables4 and 5. Boldface in the tables indicates the best result among those obtained by all algorithms. When the algorithm is not globally convergent in all 30 runs, the mFEs is represented as “NaN.” The successful ratios of different algorithms for the nine functions are also shown in the tables.Table 4
The mean FEs needed to reach an acceptable solution and reliability “ratio” being the percentage of trial runs reaching acceptable solutions for 10-dimensional functions.
Methods
CLPSO
TLBO
ETLBO
TLBO-PSO
mFEs
Ratios
mFEs
Ratios
mFEs
Ratios
mFEs
Ratios
f
1
27041.1
100.0%
2744.9
100.0%
3044.1
100.0%
1640
100.0%
f
2
NaN
0.0%
5748
100.0%
6106.3
100.0%
2858.2
100.0%
f
3
24058.3
100.0%
2336.1
100.0%
2631.1
100.0%
1449.2
100.0%
f
4
NaN
0.0%
5999.3
100.0%
6040.2
100.0%
3084.3
100.0%
f
5
NaN
0.0%
45112
20.0%
43793.5
40.0%
NaN
0.0%
f
6
38089.8
100.0%
4201.4
100.0%
4577.4
100.0%
2480.4
100.0%
f
7
26783.3
100.0%
23632.8
50.0%
17727.2
50.0%
12833
40.0%
f
8
41009.2
100.0%
6110.1
100.0%
6686.2
100.0%
3278.9
100.0%
f
9
24268
100.0%
4123.3
100.0%
5467
100.0%
2796.2
90.0%Table 5
The mean FEs needed to reach an acceptable solution and reliability “ratio” being the percentage of trial runs reaching acceptable solutions for 30-dimensional functions.
Methods
CLPSO
TLBO
ETLBO
TLBO-PSO
mFEs
Ratios
mFEs
Ratios
mFEs
Ratios
mFEs
Ratios
f
1
0.0%
3502.8
100.0%
4039.5
100.0%
2327.9
100.0%
0.0%
f
2
0.0%
13845.3
100.0%
13699.8
100.0%
6802.7
100.0%
0.0%
f
3
0.0%
3214.1
100.0%
3716.9
100.0%
2093
100.0%
0.0%
f
4
0.0%
26934.9
100.0%
27876.2
100.0%
14067.3
100.0%
0.0%
f
5
0.0%
NaN
0.0%
NaN
0.0%
NaN
0.0%
0.0%
f
6
0.0%
5039.1
100.0%
5744.6
100.0%
3305.4
100.0%
0.0%
f
7
0.0%
NaN
0.0%
NaN
0.0%
4629
10.0%
0.0%
f
8
0.0%
7188.8
100.0%
8125.9
100.0%
4315.3
100.0%
0.0%
f
9
100.0%
2046.8
100.0%
2361
100.0%
1492.5
100.0%
100.0%Tables4 and 5 display that mFEs of TLBO-PSO are the smallest for large part of functions except that for function f
5. The merit in terms of mean FEs for 10-dimensional function f
5 with ETLBO is the best and the four algorithms cannot converge to the acceptable solution for 30-dimensional function. For function f
6, all algorithms can converge to the global optima with 100% successful ratios on 30 dimensions except CLPSO. For function f
7, CLPSO can converge to optima with 100% successful ratios for 10-dimensional function. They all cannot converge to 30-dimensional function except that the successful ratio of TLBO-PSO is 10%. The convergence speed of average best fitness of four algorithms is shown in Figure 1. The figures indicate that the TLBO-PSO has the best performance for large part of functions. According to the theorem of “no free lunch” [28], one algorithm cannot offer better performance than all the others on every aspect or on every kind of problem. This is also observed in our experimental results. For example, the merit of TLBO-PSO is worse than those of TLBO and ETLBO for 10-dimension function but it is better than large part of algorithms for other functions.Figure 1
Convergence performance of the four different methods for 30-dimensional functions.


## 5.1. Parameters Setting

To reduce statistical errors, each function in the paper is independently simulated 30 runs, and their mean results are used in the comparison. The value of function is defined as the fitness function. All the experiments are carried out on the same machine with a Celeron 2.26 GHz CPU, 512-MB memory, and Windows XP operating system with Matlab 7.0. All functions are simulated in 10 and 30 dimensions. The nine functions are summarized in Table1. “Range” is the lower and upper bounds of the variables. “f
m
i
n” is the theory global minimum solution. “Acceptance” is the acceptable solutions of different functions. For CLPSO, TLBO, and ETLBO algorithms, the training parameters are the same as those used in the corresponding references except that the maximal FEs are 50000 and the size of population is 30. The size of elitism is 2 in ETLBO algorithm. When the best fitness is changed to be smaller than ε, the large value of mutation will possibly be chosen. In our experiments, the mutation possibility p
c is 0.6, ε is 0.01, and α is 0.05.


## 5.2. Comparisons on the Solution Accuracy

The performance of different algorithms for 10- and 30-dimensional functions in terms of the best solution, the mean (Mean), and standard deviation (STD) of the solutions obtained in the 30 independent runs is listed in Tables2 and 3. Boldface in the tables indicates the best solution among those obtained by all four contenders.Table 2
The mean best solutions and standard deviations by various methods for 10D functions.
Methods
CLPSO
TLBO
ETLBO
TLBO-PSO
Mean
STD
Mean
STD
Mean
STD
Mean
STD
f
1
4.68
E
-
19
4.90
E
-
19
2.12
E
-
185
0.00
E
+
00
1.51
E
-
167
0.00
E
+
00
1.07
E
-
314
0.00
E
+
00
f
2
6.73
E
-
01
4.40
E
-
01
5.79
E
-
80
1.71
E
-
79
5.10
E
-
78
1.22
E
-
77
9.62
E
-
172
0.00
E
+
00
f
3
1.14
E
-
20
8.02
E
-
21
1.02
E
-
185
0.00
E
+
00
5.91
E
-
169
0.00
E
+
00
2.65
E
-
315
0.00
E
+
00
f
4
3.32
E
-
03
3.25
E
-
03
4.44
E
-
87
1.36
E
-
86
9.99
E
-
85
1.89
E
-
84
5.04
E
-
182
0.00
E
+
00
f
5
2.25
E
+
00
1.00
E
+
00
6.54
E
-
01
6.56
E
-
01
2.76
E
-
01
2.78
E
-
01
4.18
E
+
00
7.46
E
-
01
f
6
3.75
E
-
10
2.83
E
-
10
3.55
E
-
15
0.00
E
+
00
3.20
E
-
15
1.12
E
-
15
2.84
E
-
15
1.50
E
-
15
f
7
1.63
E
-
09
2.42
E
-
09
2.17
E
+
00
1.86
E
+
00
2.33
E
+
00
1.76
E
+
00
5.26
E
+
00
4.63
E
+
00
f
8
1.16
E
-
11
1.55
E
-
11
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
f
9
1.06
E
-
03
2.29
E
-
03
9.96
E
-
03
1.02
E
-
02
8.45
E
-
03
1.61
E
-
02
3.54
E
-
02
5.27
E
-
02Table 3
The mean best solutions and standard deviations by various methods for 30D functions.
Methods
CLPSO
TLBO
ETLBO
TLBO-PSO
Mean
STD
Mean
STD
Mean
STD
Mean
STD
f
1
1.50
E
-
03
5.04
E
-
04
1.63
E
-
152
2.38
E
-
152
5.70
E
-
132
1.13
E
-
131
6.67
E
-
240
0.00
E
+
00
f
2
9.37
E
+
03
2.23
E
+
03
1.78
E
-
33
3.34
E
-
33
4.58
E
-
33
5.40
E
-
33
1.98
E
-
76
4.73
E
-
76
f
3
1.39
E
-
04
4.19
E
-
05
1.45
E
-
152
2.79
E
-
152
7.13
E
-
132
1.78
E
-
131
1.60
E
-
240
0.00
E
+
00
f
4
1.45
E
+
02
4.10
E
+
01
5.72
E
-
18
7.60
E
-
18
1.56
E
-
16
3.00
E
-
16
1.46
E
-
43
3.04
E
-
43
f
5
3.60
E
+
01
1.03
E
+
01
2.42
E
+
01
6.48
E
-
01
2.46
E
+
01
7.50
E
-
01
2.65
E
+
01
1.18
E
+
00
f
6
3.19
E
-
02
1.27
E
-
02
3.55
E
-
15
0.00
E
+
00
3.91
E
-
15
1.12
E
-
15
3.55
E
-
15
0.00
E
+
00
f
7
9.88
E
+
00
2.48
E
+
00
1.31
E
+
01
5.78
E
+
00
1.68
E
+
01
8.12
E
+
00
2.41
E
+
01
1.40
E
+
01
f
8
6.53
E
-
02
1.08
E
-
02
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
f
9
8.76
E
-
03
3.43
E
-
03
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00
0.00
E
+
00The results for 10-dimensional functions in Table2 display that the improved TLBO (TLBO-PSO) outperforms all the other algorithms in terms of the mean best solutions and standard deviations (STD) for functions f
1, f
2, f
3, and f
4. For function f
5, ETLBO has the best performance. For function f
6, the mean best solution of TLBO-PSO is the best, and the STD of TLBO is the smallest. For functions f
7 and f
9, CLPSO has the best performance. For function f
8, three TLBOs have the same solutions in terms of the mean best solutions and standard deviations (STD).The results for 30-dimensional functions in Table3 indicate that TLBO-PSO also has the best performance in terms of the mean best solutions and standard deviations (STD) for functions f
1, f
2, f
3, and f
4 with 30 dimensions. TLBO has the best performance for function f
5. For function f
6, TLBO and ETLBO have the same performance in terms of the mean best solutions and standard deviations (STD). For function f
7, the mean best solution of TLBO is the smallest, and the standard deviation of TLBO-PSO is the smallest. Three TLBOs have the same solutions for functions f
8 and f
9.


## 5.3. Comparisons on the Convergence Speed and Successful Ratios

The mean number of function evolutions (FEs) is often used to measure the convergence speed of algorithms. In this paper, the mean value of FEs is used to measure the speed of all algorithms. The average FEs (when the algorithm is globally convergent) of all algorithms for nine functions with 10 and 30 dimensions are shown in Tables4 and 5. Boldface in the tables indicates the best result among those obtained by all algorithms. When the algorithm is not globally convergent in all 30 runs, the mFEs is represented as “NaN.” The successful ratios of different algorithms for the nine functions are also shown in the tables.Table 4
The mean FEs needed to reach an acceptable solution and reliability “ratio” being the percentage of trial runs reaching acceptable solutions for 10-dimensional functions.
Methods
CLPSO
TLBO
ETLBO
TLBO-PSO
mFEs
Ratios
mFEs
Ratios
mFEs
Ratios
mFEs
Ratios
f
1
27041.1
100.0%
2744.9
100.0%
3044.1
100.0%
1640
100.0%
f
2
NaN
0.0%
5748
100.0%
6106.3
100.0%
2858.2
100.0%
f
3
24058.3
100.0%
2336.1
100.0%
2631.1
100.0%
1449.2
100.0%
f
4
NaN
0.0%
5999.3
100.0%
6040.2
100.0%
3084.3
100.0%
f
5
NaN
0.0%
45112
20.0%
43793.5
40.0%
NaN
0.0%
f
6
38089.8
100.0%
4201.4
100.0%
4577.4
100.0%
2480.4
100.0%
f
7
26783.3
100.0%
23632.8
50.0%
17727.2
50.0%
12833
40.0%
f
8
41009.2
100.0%
6110.1
100.0%
6686.2
100.0%
3278.9
100.0%
f
9
24268
100.0%
4123.3
100.0%
5467
100.0%
2796.2
90.0%Table 5
The mean FEs needed to reach an acceptable solution and reliability “ratio” being the percentage of trial runs reaching acceptable solutions for 30-dimensional functions.
Methods
CLPSO
TLBO
ETLBO
TLBO-PSO
mFEs
Ratios
mFEs
Ratios
mFEs
Ratios
mFEs
Ratios
f
1
0.0%
3502.8
100.0%
4039.5
100.0%
2327.9
100.0%
0.0%
f
2
0.0%
13845.3
100.0%
13699.8
100.0%
6802.7
100.0%
0.0%
f
3
0.0%
3214.1
100.0%
3716.9
100.0%
2093
100.0%
0.0%
f
4
0.0%
26934.9
100.0%
27876.2
100.0%
14067.3
100.0%
0.0%
f
5
0.0%
NaN
0.0%
NaN
0.0%
NaN
0.0%
0.0%
f
6
0.0%
5039.1
100.0%
5744.6
100.0%
3305.4
100.0%
0.0%
f
7
0.0%
NaN
0.0%
NaN
0.0%
4629
10.0%
0.0%
f
8
0.0%
7188.8
100.0%
8125.9
100.0%
4315.3
100.0%
0.0%
f
9
100.0%
2046.8
100.0%
2361
100.0%
1492.5
100.0%
100.0%Tables4 and 5 display that mFEs of TLBO-PSO are the smallest for large part of functions except that for function f
5. The merit in terms of mean FEs for 10-dimensional function f
5 with ETLBO is the best and the four algorithms cannot converge to the acceptable solution for 30-dimensional function. For function f
6, all algorithms can converge to the global optima with 100% successful ratios on 30 dimensions except CLPSO. For function f
7, CLPSO can converge to optima with 100% successful ratios for 10-dimensional function. They all cannot converge to 30-dimensional function except that the successful ratio of TLBO-PSO is 10%. The convergence speed of average best fitness of four algorithms is shown in Figure 1. The figures indicate that the TLBO-PSO has the best performance for large part of functions. According to the theorem of “no free lunch” [28], one algorithm cannot offer better performance than all the others on every aspect or on every kind of problem. This is also observed in our experimental results. For example, the merit of TLBO-PSO is worse than those of TLBO and ETLBO for 10-dimension function but it is better than large part of algorithms for other functions.Figure 1
Convergence performance of the four different methods for 30-dimensional functions.


## 6. Conclusions

An improved TLBO-PSO algorithm which is considering the difference between the solutions of the best individual and the individual that want to be renewed is designed in the paper. The mutation operator is introduced to improve the global convergence performance the algorithm. The performance of TLBO-PSO is improved for larger part of functions especially for 30-dimension functions in terms of convergence accuracy and mFEs. The local convergence of TLBO-PSO is major caused by the lost diversity in the later stage of the evolution.Further works include researches into adaptive selection of parameters to make the algorithm more efficient. Moreover, it needs to seek a better method to improve the TLBO algorithm for functions with optimal parameters displaced from all zeroes. Furthermore, the algorithm may be applied to constrained, dynamic optimization domain. It is expected that TLBO-PSO will be used in real-world optimization problems.



---

*Source: 4561507-2016-01-14.xml*