# An Adaptive Hybrid Algorithm Based on Particle Swarm Optimization and Differential Evolution for Global Optimization

Authors: Xiaobing Yu; Jie Cao; Haiyan Shan; Li Zhu; Jun Guo
Journal: The Scientific World Journal
 (2014)

---

## Abstract

Particle swarm optimization (PSO) and differential evolution (DE) are both efficient and powerful population-based stochastic search techniques for solving optimization problems, which have been widely applied in many scientific and engineering fields. Unfortunately, both of them can easily fly into local optima and lack the ability of jumping out of local optima. A novel adaptive hybrid algorithm based on PSO and DE (HPSO-DE) is formulated by developing a balanced parameter between PSO and DE. Adaptive mutation is carried out on current population when the population clusters around local optima. The HPSO-DE enjoys the advantages of PSO and DE and maintains diversity of the population. Compared with PSO, DE, and their variants, the performance of HPSO-DE is competitive. The balanced parameter sensitivity is discussed in detail.


---

## Body

## 1. Introduction

Evolutionary algorithms (EAs), inspired by the natural evolution of species, have been successfully applied to solve numerous optimization problems in diverse fields [1]. Particle swarm optimization (PSO) and differential evolution (DE) are two stochastic, population-based optimization EAs [2].PSO was introduced by Kennedy and Eberhart in 1995 [3, 4]. PSO uses a simple mechanism that mimics swarm behavior in birds flocking and fish schooling to guide the particles to search for globally optimal solutions. As PSO is easy to implement, it has rapidly progressed in recent years with many successful applications in solving real-world optimization problems. During the last decade, PSO algorithm has been paid much attention to in various areas [5–19] due to its effectiveness in handling optimization problems. Unfortunately, the PSO algorithm suffers from the premature convergence problem which does exist in complex optimization issues. In [17–19], some methods for tuning parameters including inertia weights and acceleration coefficients for PSO have been proposed to enhance PSO’s search performance. A comprehensive learning PSO (CLPSO) algorithm was proposed in [14], which shows its superiority in dealing with multimodal functions.DE is a simple yet powerful EA for global optimization introduced by Storn and Price [20]. The DE algorithm has gradually become more popular and has been used in many practical cases, mainly because it has demonstrated good convergence properties and is principally easy to understand [21]. DE has been successfully applied in diverse fields of engineering [22–26]. The performance of the conventional DE algorithm highly depends on the chosen trial vector generation strategy and associated parameter values used. Inappropriate choice of strategies and parameters may lead to premature convergence, which have been extensively demonstrated in [27]. In the past decade, DE researchers have suggested many empirical guidelines for choosing trial vector generation strategies and their associated control parameter settings [1, 28–31].Although DE and PSO have been successfully applied to a wide range of problems including test and real life problems, both have certain shortcomings associated with them which sometimes deteriorate the performance of algorithms. The major problem is the lack of diversity resulting in a suboptimal solution or a slow convergence rate [32]. In order to improve the performance of these algorithms, one of the class of modified algorithms consists of the hybridization of algorithms, where the two algorithms are combined together to form a new algorithm. DE is applied to each particle for a finite number of iterations to determine the best particle which is then included into the population [33]. DE is applied to the best particle obtained by PSO [34]. A hybrid version of PSO and DE is proposed which is named Barebones DE [35]. The evolution candidate solution is generated either by DE or by PSO according to some fixed probability distribution [36]. A hybrid metaheuristic is designed so as to preserve the strengths of both algorithms [32].However, it is worth mentioning that, in almost all the hybrid works mentioned above, the convergence rate is not fast enough or the global search performance is not satisfactory. To achieve the goal, a novel adaptive hybrid algorithm based on PSO and DE (HPSO-DE) is formulated by developing a balanced parameter between PSO and DE. Adaptive mutation is carried out to current population when the population clusters around local optima. The HPSO-DE enjoys the advantages of PSO and DE and maintains diversity of the population. With the efforts, the HPSO-DE has the ability to jump out of the local optima.


## 2. PSO and DE

### 2.1. PSO

In the standard PSO, a swarm consists ofm individuals (called particles) that fly around in an n
-dimensional search space. The position of the ith particle at the tth iteration is used to evaluate the particle and represent the candidate solution for the optimization problem. It can be represented as X
i
t
=
[
x
i
1
t
,
x
i
2
t
,
…
,
x
i
n
t
], where x
i
j
t is the position value of the ith particle with respect to the jth dimension (
j
=
1,2
,
…
,
n
). During the search process, the position of a particle is guided by two factors: the best position visited by itself (p
best) denoted by P
i
t
=
[
p
i
1
t
,
p
i
2
t
,
…
,
p
i
n
t
] and the position of the best particle found so far in the swarm (g
best) denoted by G
t
=
[
g
1
t
,
g
2
t
,
…
,
g
n
t
]. The new velocity (denoted by V
i
t
=
[
v
i
1
t
,
v
i
2
t
,
…
,
v
i
n
t
]) and position of particle i at the next iteration are calculated according to
(1)
v
i
j
t
+
1
=
w
×
v
i
j
t
+
c
1
×
r
1
×
(
p
i
j
t
-
x
i
j
t
)
+
c
2
×
r
2
×
(
g
j
t
-
x
i
j
t
)
,
(2)
x
i
j
t
+
1
=
x
i
j
t
+
v
i
j
t
+
1
,
where w is the inertia weight, c
1, and c
2 are, respectively, the cognitive and social learning parameters, and r
1, and r
2 are random numbers between (
0
,
1
). Based on the above equations, the particle can fly through search space toward p
best and g
best in a navigated way [16, 17].


### 2.2. PSO Variants

#### 2.2.1. PSO-w

In the PSO algorithm, proper control of global exploration and local exploitation is an important issue. In general, the higher values of inertia weightw help in exploring the search space more thoroughly in the process and benefit the global search, while lower values help in the local search around the current search space. The major concern of this linear PSO is to avoid the premature convergence in the early period of the search and to enhance convergence to the global optimum solution during the latter period of the search. The concept of linearly decreasing inertia weight was introduced in [17] and is given by
(3)
w
=
w
max
⁡
-
(
w
max
⁡
-
w
min
⁡
)
(
iter
iter
max
⁡
)
,
where iter is the current iteration number and iter
max
⁡ is the maximum number of iteration. Usually the value of w is between 0.9 and 0.4. Therefore, the particle is to use lager inertia weight during the initial exploration and gradually reduce its value as the search proceeds in further iterations. According to the research [37], the inertia weight is adjusted by (4). The nonlinear descending can achieve faster convergence speed than that with linear inertia weight:
(4)
w
=
(
w
1
-
w
2
)
×
(
iter
-
iter
max
⁡
)
2
(
iter
max
⁡
)
2
+
w
2
,
where w
1 and w
2 are the initial and final inertia weight.


#### 2.2.2. PSO-TVAC

Although linear PSO can locate satisfactory solution at a markedly fast speed, its ability to fine-tune the optimum solution is limited, mainly due to the lack of diversity at the latter stage of evolution process. In population-based optimization methods, the guideline is to encourage the individuals to roam through the entire search space during the early period of the search, without clustering around local optima. During the later period, convergence towards the global optima is encouraged [3]. With this view, a novel strategy in which time-varying acceleration coefficients are employed by changing the acceleration coefficients with time is proposed [16, 38]. With a large cognitive component and small social component at the beginning, particles are allowed to move around the search space, instead of moving toward the population best. On the other hand, a small cognitive component and a large social component allow the particles to converge to the global optima in the latter part of the optimization. This approach is referred to as PSO-TVAC. This modification can be mathematically represented as follows:
(5)
c
1
=
(
c
1
f
-
c
1
i
)
(
iter
iter
max
⁡
)
+
c
1
i
,
c
2
=
(
c
2
f
-
c
2
i
)
(
iter
iter
max
⁡
)
+
c
2
i
,
where c
1
i, c
1
f, c
2
i, and c
2
f are initial and final values of cognitive and social acceleration factors, respectively; usually c
1
i
=
c
2
f
=
2.5 and c
1
f
=
c
2
i
=
0.5 [38].


### 2.3. DE

DE is proposed by Storn and Price [20]. It is an effective, robust, and simple global optimization algorithm. According to frequently reported experimental studies, DE has shown better performance than many other EAs in terms of convergence speed and robustness over several benchmark functions and real-world problems [39].In DE, there are three operators: mutation, crossover, and selection. Initially, a population is generated randomly with uniform distribution; then the mutation, crossover, and selection operators are applied to generate a new population. Trial vector generation is a crucial step in DE process. The two operators mutation and crossover are used to generate the trial vectors. The selection operator is used to select the best trial vector for the next generation. The initialization and DE operators are explained briefly as follows [40].DE starts with a population ofNP  D-dimensional candidate solutions which may be represented as X
i
,
G
(
i
=
1,2
,
…
,
NP
)
=
{
x
i
,
G
1
,
x
i
,
G
2
,
…
,
x
i
,
G
D
}, where index i denotes the ith individual of the population, G denotes the generation to which the population belongs, and D is the dimension of the population.The initial population should try to cover the entire search space as much as possible by uniformly randomizing individuals within the search space constrained by the minimumX
min
⁡
=
{
x
min
⁡
1
,
x
min
⁡
2
,
…
,
x
min
⁡
D
} and maximum X
max
⁡
=
{
x
max
⁡
1
,
x
max
⁡
2
,
…
,
x
max
⁡
D
} bounds. Thus, the initial population can be described as follows:
(6)
x
i
,
0
=
x
min
⁡
+
rand
(
0,1
)
×
(
x
max
⁡
-
x
min
⁡
)
,
where rand
(
0,1
)
∈
[
0,1
] is a uniformly distributed random variable [40, 44].(1)  Mutation. After initialization, DE utilizes the mutation operation to generate a trial vector V
i
,
G
=
{
v
i
,
G
1
,
v
i
,
G
2
,
…
,
v
i
,
G
D
} with respect to each individual in the current population. V
i
,
G can be produced by certain mutation strategy. For example, the five most frequently mutation strategies implemented in the DE are listed as follows [1]:DE/rand/1:(7)
V
i
,
G
=
X
r
1
i
,
G
+
F
·
(
X
r
2
i
,
G
-
X
r
3
i
,
G
)
,
DE/best/1:(8)
V
i
,
G
=
X
best
,
G
+
F
·
(
X
r
1
i
,
G
-
X
r
2
i
,
G
)
,
DE/rand-to-best/1:(9)
V
i
,
G
=
X
i
,
G
+
F
·
(
X
best
,
G
-
X
i
,
G
)
+
F
·
(
X
r
1
i
,
G
-
X
r
2
i
,
G
)
,
DE/best/2:(10)
V
i
,
G
=
X
best
,
G
+
F
·
(
X
r
1
i
,
G
-
X
r
2
i
,
G
)
+
F
·
(
X
r
3
i
,
G
-
X
r
4
i
,
G
)
,
DE/rand/2:(11)
V
i
,
G
=
X
r
1
i
,
G
+
F
·
(
X
r
2
i
,
G
-
X
r
3
i
,
G
)
+
F
·
(
X
r
4
i
,
G
-
X
r
5
i
,
G
)
.The indicesr
1
i, r
2
i, r
3
i, r
4
i, and r
5
i are mutually exclusive integers randomly generated within the range [
0,1
], which are also different from the index i [1]. F is the mutation scale factor which is used in controlling the amplification of the differential variation [40].(2)  Crossover. The crossover operation is introduced to increase the diversity of the target vectors. After the mutation phase, the crossover operation is applied to V
i
,
G
=
{
v
i
,
G
1
,
v
i
,
G
2
,
…
,
v
i
,
G
D
} and X
i
,
G
=
{
X
i
,
G
1
,
X
i
,
G
2
,
…
,
X
i
,
G
D
} to generate a trial vector U
i
,
G
=
{
u
i
,
G
1
,
u
i
,
G
2
,
…
,
u
i
,
G
D
} as follows:
(12)
u
i
,
G
j
=
{
v
i
,
G
j
if
rand
j
[
0,1
)
≤
CR
or
(
j
=
j
rand
)
,
x
i
,
G
j
others
.
CR
∈
[
0,1
] is the crossover constant, which has to be determined by the user. j
rand
∈
[
1
,
D
] is a randomly chosen index which ensures that the trial vector U
i
,
G will differ from X
i
,
G by at least one parameter.(3)  Selection. If the generated trial vector u
i
,
G
j exceeds the corresponding upper and lower bounds, we randomly and uniformly reinitialize it within the search range. Then the fitness values of all trial vectors are evaluated [45].After that, a greedy selection scheme is employed:(13)
x
i
,
G
+
1
=
{
u
i
,
G
if
f
(
u
i
,
G
)
≤
f
(
x
i
,
G
)
,
x
i
,
G
otherwise
.If the trial vectoru
i
,
G yields a better cost function value than x
i
,
G, the u
i
,
G will replace x
i
,
G and enter the population of the next generation; otherwise, the old value x
i
,
G is retained.


### 2.4. DE Variants

In order to improve the performance of DE, some adaptive DE variants are proposed. jDE was proposed based on the self-adaptation of the scale factorF and the crossover rate CR [29]. The jDE is represented by a D-dimensional vector X
i
,
G
(
i
=
1,2
,
…
,
NP
). New control parameters or factors F
i
,
G
+
1 and CR
i
,
G
+
1 are calculated as
(14)
F
i
,
G
+
1
=
{
F
l
+
rand
1
×
F
u
if
rand
2
<
τ
1
,
F
i
,
G
otherwise
,
CR
i
,
G
+
1
=
{
rand
3
if
rand
4
<
τ
2
,
CR
i
,
G
otherwise
,
and they produce factors F and CR in a new parent vector. rand
j
(
j
∈
{
1,2
,
3,4
}
) are uniform random values within the range [
0,1
]. τ
1 and τ
2 represent probabilities to adjust factors F and CR, respectively. Generally, τ
1
=
τ
2
=
0.1, F
l
=
0.1, F
u
=
0.9. SaDE gave the first attempt to simultaneously adopt more than one mutation scheme in DE [1]. The main propose is to reduce the problem-solving risk by distributing available computational resources to multiple search techniques with different biases. SaNSDE can be regarded as an improved version of the SaDE. Its mutation is executed in the same way as SaDE except that only two mutation schemes are used, and the scale factor F in the adopted mutation schemes is generated according to either a Gaussian distribution or a Cauchy distribution [30]. JADE is another recent DE variant, in which a new mutation scheme named “/DE/current-to-pbest” is adopted [31].


## 2.1. PSO

In the standard PSO, a swarm consists ofm individuals (called particles) that fly around in an n
-dimensional search space. The position of the ith particle at the tth iteration is used to evaluate the particle and represent the candidate solution for the optimization problem. It can be represented as X
i
t
=
[
x
i
1
t
,
x
i
2
t
,
…
,
x
i
n
t
], where x
i
j
t is the position value of the ith particle with respect to the jth dimension (
j
=
1,2
,
…
,
n
). During the search process, the position of a particle is guided by two factors: the best position visited by itself (p
best) denoted by P
i
t
=
[
p
i
1
t
,
p
i
2
t
,
…
,
p
i
n
t
] and the position of the best particle found so far in the swarm (g
best) denoted by G
t
=
[
g
1
t
,
g
2
t
,
…
,
g
n
t
]. The new velocity (denoted by V
i
t
=
[
v
i
1
t
,
v
i
2
t
,
…
,
v
i
n
t
]) and position of particle i at the next iteration are calculated according to
(1)
v
i
j
t
+
1
=
w
×
v
i
j
t
+
c
1
×
r
1
×
(
p
i
j
t
-
x
i
j
t
)
+
c
2
×
r
2
×
(
g
j
t
-
x
i
j
t
)
,
(2)
x
i
j
t
+
1
=
x
i
j
t
+
v
i
j
t
+
1
,
where w is the inertia weight, c
1, and c
2 are, respectively, the cognitive and social learning parameters, and r
1, and r
2 are random numbers between (
0
,
1
). Based on the above equations, the particle can fly through search space toward p
best and g
best in a navigated way [16, 17].


## 2.2. PSO Variants

### 2.2.1. PSO-w

In the PSO algorithm, proper control of global exploration and local exploitation is an important issue. In general, the higher values of inertia weightw help in exploring the search space more thoroughly in the process and benefit the global search, while lower values help in the local search around the current search space. The major concern of this linear PSO is to avoid the premature convergence in the early period of the search and to enhance convergence to the global optimum solution during the latter period of the search. The concept of linearly decreasing inertia weight was introduced in [17] and is given by
(3)
w
=
w
max
⁡
-
(
w
max
⁡
-
w
min
⁡
)
(
iter
iter
max
⁡
)
,
where iter is the current iteration number and iter
max
⁡ is the maximum number of iteration. Usually the value of w is between 0.9 and 0.4. Therefore, the particle is to use lager inertia weight during the initial exploration and gradually reduce its value as the search proceeds in further iterations. According to the research [37], the inertia weight is adjusted by (4). The nonlinear descending can achieve faster convergence speed than that with linear inertia weight:
(4)
w
=
(
w
1
-
w
2
)
×
(
iter
-
iter
max
⁡
)
2
(
iter
max
⁡
)
2
+
w
2
,
where w
1 and w
2 are the initial and final inertia weight.


### 2.2.2. PSO-TVAC

Although linear PSO can locate satisfactory solution at a markedly fast speed, its ability to fine-tune the optimum solution is limited, mainly due to the lack of diversity at the latter stage of evolution process. In population-based optimization methods, the guideline is to encourage the individuals to roam through the entire search space during the early period of the search, without clustering around local optima. During the later period, convergence towards the global optima is encouraged [3]. With this view, a novel strategy in which time-varying acceleration coefficients are employed by changing the acceleration coefficients with time is proposed [16, 38]. With a large cognitive component and small social component at the beginning, particles are allowed to move around the search space, instead of moving toward the population best. On the other hand, a small cognitive component and a large social component allow the particles to converge to the global optima in the latter part of the optimization. This approach is referred to as PSO-TVAC. This modification can be mathematically represented as follows:
(5)
c
1
=
(
c
1
f
-
c
1
i
)
(
iter
iter
max
⁡
)
+
c
1
i
,
c
2
=
(
c
2
f
-
c
2
i
)
(
iter
iter
max
⁡
)
+
c
2
i
,
where c
1
i, c
1
f, c
2
i, and c
2
f are initial and final values of cognitive and social acceleration factors, respectively; usually c
1
i
=
c
2
f
=
2.5 and c
1
f
=
c
2
i
=
0.5 [38].


## 2.2.1. PSO-w

In the PSO algorithm, proper control of global exploration and local exploitation is an important issue. In general, the higher values of inertia weightw help in exploring the search space more thoroughly in the process and benefit the global search, while lower values help in the local search around the current search space. The major concern of this linear PSO is to avoid the premature convergence in the early period of the search and to enhance convergence to the global optimum solution during the latter period of the search. The concept of linearly decreasing inertia weight was introduced in [17] and is given by
(3)
w
=
w
max
⁡
-
(
w
max
⁡
-
w
min
⁡
)
(
iter
iter
max
⁡
)
,
where iter is the current iteration number and iter
max
⁡ is the maximum number of iteration. Usually the value of w is between 0.9 and 0.4. Therefore, the particle is to use lager inertia weight during the initial exploration and gradually reduce its value as the search proceeds in further iterations. According to the research [37], the inertia weight is adjusted by (4). The nonlinear descending can achieve faster convergence speed than that with linear inertia weight:
(4)
w
=
(
w
1
-
w
2
)
×
(
iter
-
iter
max
⁡
)
2
(
iter
max
⁡
)
2
+
w
2
,
where w
1 and w
2 are the initial and final inertia weight.


## 2.2.2. PSO-TVAC

Although linear PSO can locate satisfactory solution at a markedly fast speed, its ability to fine-tune the optimum solution is limited, mainly due to the lack of diversity at the latter stage of evolution process. In population-based optimization methods, the guideline is to encourage the individuals to roam through the entire search space during the early period of the search, without clustering around local optima. During the later period, convergence towards the global optima is encouraged [3]. With this view, a novel strategy in which time-varying acceleration coefficients are employed by changing the acceleration coefficients with time is proposed [16, 38]. With a large cognitive component and small social component at the beginning, particles are allowed to move around the search space, instead of moving toward the population best. On the other hand, a small cognitive component and a large social component allow the particles to converge to the global optima in the latter part of the optimization. This approach is referred to as PSO-TVAC. This modification can be mathematically represented as follows:
(5)
c
1
=
(
c
1
f
-
c
1
i
)
(
iter
iter
max
⁡
)
+
c
1
i
,
c
2
=
(
c
2
f
-
c
2
i
)
(
iter
iter
max
⁡
)
+
c
2
i
,
where c
1
i, c
1
f, c
2
i, and c
2
f are initial and final values of cognitive and social acceleration factors, respectively; usually c
1
i
=
c
2
f
=
2.5 and c
1
f
=
c
2
i
=
0.5 [38].


## 2.3. DE

DE is proposed by Storn and Price [20]. It is an effective, robust, and simple global optimization algorithm. According to frequently reported experimental studies, DE has shown better performance than many other EAs in terms of convergence speed and robustness over several benchmark functions and real-world problems [39].In DE, there are three operators: mutation, crossover, and selection. Initially, a population is generated randomly with uniform distribution; then the mutation, crossover, and selection operators are applied to generate a new population. Trial vector generation is a crucial step in DE process. The two operators mutation and crossover are used to generate the trial vectors. The selection operator is used to select the best trial vector for the next generation. The initialization and DE operators are explained briefly as follows [40].DE starts with a population ofNP  D-dimensional candidate solutions which may be represented as X
i
,
G
(
i
=
1,2
,
…
,
NP
)
=
{
x
i
,
G
1
,
x
i
,
G
2
,
…
,
x
i
,
G
D
}, where index i denotes the ith individual of the population, G denotes the generation to which the population belongs, and D is the dimension of the population.The initial population should try to cover the entire search space as much as possible by uniformly randomizing individuals within the search space constrained by the minimumX
min
⁡
=
{
x
min
⁡
1
,
x
min
⁡
2
,
…
,
x
min
⁡
D
} and maximum X
max
⁡
=
{
x
max
⁡
1
,
x
max
⁡
2
,
…
,
x
max
⁡
D
} bounds. Thus, the initial population can be described as follows:
(6)
x
i
,
0
=
x
min
⁡
+
rand
(
0,1
)
×
(
x
max
⁡
-
x
min
⁡
)
,
where rand
(
0,1
)
∈
[
0,1
] is a uniformly distributed random variable [40, 44].(1)  Mutation. After initialization, DE utilizes the mutation operation to generate a trial vector V
i
,
G
=
{
v
i
,
G
1
,
v
i
,
G
2
,
…
,
v
i
,
G
D
} with respect to each individual in the current population. V
i
,
G can be produced by certain mutation strategy. For example, the five most frequently mutation strategies implemented in the DE are listed as follows [1]:DE/rand/1:(7)
V
i
,
G
=
X
r
1
i
,
G
+
F
·
(
X
r
2
i
,
G
-
X
r
3
i
,
G
)
,
DE/best/1:(8)
V
i
,
G
=
X
best
,
G
+
F
·
(
X
r
1
i
,
G
-
X
r
2
i
,
G
)
,
DE/rand-to-best/1:(9)
V
i
,
G
=
X
i
,
G
+
F
·
(
X
best
,
G
-
X
i
,
G
)
+
F
·
(
X
r
1
i
,
G
-
X
r
2
i
,
G
)
,
DE/best/2:(10)
V
i
,
G
=
X
best
,
G
+
F
·
(
X
r
1
i
,
G
-
X
r
2
i
,
G
)
+
F
·
(
X
r
3
i
,
G
-
X
r
4
i
,
G
)
,
DE/rand/2:(11)
V
i
,
G
=
X
r
1
i
,
G
+
F
·
(
X
r
2
i
,
G
-
X
r
3
i
,
G
)
+
F
·
(
X
r
4
i
,
G
-
X
r
5
i
,
G
)
.The indicesr
1
i, r
2
i, r
3
i, r
4
i, and r
5
i are mutually exclusive integers randomly generated within the range [
0,1
], which are also different from the index i [1]. F is the mutation scale factor which is used in controlling the amplification of the differential variation [40].(2)  Crossover. The crossover operation is introduced to increase the diversity of the target vectors. After the mutation phase, the crossover operation is applied to V
i
,
G
=
{
v
i
,
G
1
,
v
i
,
G
2
,
…
,
v
i
,
G
D
} and X
i
,
G
=
{
X
i
,
G
1
,
X
i
,
G
2
,
…
,
X
i
,
G
D
} to generate a trial vector U
i
,
G
=
{
u
i
,
G
1
,
u
i
,
G
2
,
…
,
u
i
,
G
D
} as follows:
(12)
u
i
,
G
j
=
{
v
i
,
G
j
if
rand
j
[
0,1
)
≤
CR
or
(
j
=
j
rand
)
,
x
i
,
G
j
others
.
CR
∈
[
0,1
] is the crossover constant, which has to be determined by the user. j
rand
∈
[
1
,
D
] is a randomly chosen index which ensures that the trial vector U
i
,
G will differ from X
i
,
G by at least one parameter.(3)  Selection. If the generated trial vector u
i
,
G
j exceeds the corresponding upper and lower bounds, we randomly and uniformly reinitialize it within the search range. Then the fitness values of all trial vectors are evaluated [45].After that, a greedy selection scheme is employed:(13)
x
i
,
G
+
1
=
{
u
i
,
G
if
f
(
u
i
,
G
)
≤
f
(
x
i
,
G
)
,
x
i
,
G
otherwise
.If the trial vectoru
i
,
G yields a better cost function value than x
i
,
G, the u
i
,
G will replace x
i
,
G and enter the population of the next generation; otherwise, the old value x
i
,
G is retained.


## 2.4. DE Variants

In order to improve the performance of DE, some adaptive DE variants are proposed. jDE was proposed based on the self-adaptation of the scale factorF and the crossover rate CR [29]. The jDE is represented by a D-dimensional vector X
i
,
G
(
i
=
1,2
,
…
,
NP
). New control parameters or factors F
i
,
G
+
1 and CR
i
,
G
+
1 are calculated as
(14)
F
i
,
G
+
1
=
{
F
l
+
rand
1
×
F
u
if
rand
2
<
τ
1
,
F
i
,
G
otherwise
,
CR
i
,
G
+
1
=
{
rand
3
if
rand
4
<
τ
2
,
CR
i
,
G
otherwise
,
and they produce factors F and CR in a new parent vector. rand
j
(
j
∈
{
1,2
,
3,4
}
) are uniform random values within the range [
0,1
]. τ
1 and τ
2 represent probabilities to adjust factors F and CR, respectively. Generally, τ
1
=
τ
2
=
0.1, F
l
=
0.1, F
u
=
0.9. SaDE gave the first attempt to simultaneously adopt more than one mutation scheme in DE [1]. The main propose is to reduce the problem-solving risk by distributing available computational resources to multiple search techniques with different biases. SaNSDE can be regarded as an improved version of the SaDE. Its mutation is executed in the same way as SaDE except that only two mutation schemes are used, and the scale factor F in the adopted mutation schemes is generated according to either a Gaussian distribution or a Cauchy distribution [30]. JADE is another recent DE variant, in which a new mutation scheme named “/DE/current-to-pbest” is adopted [31].


## 3. Proposed HPSO-DE

Similar to other EAs, both of PSO and DE are the population-based iterative algorithms. The PSO and DE can easily get trapped in the local optima when solving complex multimodal problems. These weaknesses have restricted wider applications of them. Therefore, avoiding the local optima has become one of the most important and appealing goals in algorithms research. To achieve the goal, an adaptive hybrid algorithm based on PSO and DE (HPSO-DE) for global optimization is proposed. The algorithm focuses on the convergence of population. When a particle discovers a current optima position, the other particles will draw together to the particle. If the position is the local optima, the algorithm will be convergence and clustered in local optima. The premature may appear. Suppose that the population size of HPSO-DE isNP, the fitness value of ith particle is f
i, and the average fitness value is f
avg. The convergence degree is defined as follows:
(15)
d
=
∑
i
=
1
N
P
(
f
i
-
f
avg
max
⁡
⁡
{
1
,
max
⁡
1
≤
i
≤
N
(
f
i
-
f
avg
)
)
2
.The parameterd reflects the convergence degree. When the parameter d is large, the algorithm is in random search. On the other hand, the algorithm will get into local optima and the premature maybe occur. In order to evaluate the parameter d, d
c is given as follows, where p is the mutation probability.
(16)
p
=
{
k
,
d
<
d
c
,
0
,
others
.Generally,d
c
∈
[
0.5,2
]. If the parameter d is less than d
c, the mutation probability p is equal to k. The mutation including PSO and DE is as follows, respectively:
(17)
g
best
=
(
1
+
0.5
η
)
g
best
,
(18)
X
i
,
G
=
(
1
+
0.5
η
)
X
i
,
G
,
where the parameter η obeys Gauss (
0,1
) distribution.The strategy makes HPSO-DE enjoy the advantages of two algorithms and maintain diversity of the population. With the efforts, the HPSO-DE has the ability of jumping out of the local optima. The main procedure of HPSO-DE is presented in Algorithm1.Algorithm 1
Step  1. Initialize parameters: PSO-Flag=false; DE-Flag=false; the mutation probability of PSO: PS
O
p;
the mutation probability of DE:D
E
p; the balanced parameter between PSO and DE: p.
Convergence evaluation parameterd
c.
Step  2. generation counterG = 0 and randomly initialize a population of NP individuals
P
G
=
{
X
1
,
G
,
.
.
.
,
X
NP
,
G
} with X
i
,
G
=
{
x
i
,
G
1
,
.
.
.
,
x
i
,
G
D
}, i
=
1
,
.
.
.
,
NP uniformly distributed in the range
[
X
min
⁡
,
X
max
⁡
], where X
min
⁡
=
{
x
min
⁡
1
,
x
min
⁡
2
,
.
.
.
,
x
min
⁡
D
} and X
max
⁡
=
{
x
max
⁡
1
,
x
max
⁡
2
,
.
.
.
,
x
max
⁡
D
}.
Step  3. Evaluation the population and Identify the best position.
Step  4. While stopping criterion is not satisfied
Generate Random number rand in (0, 1).
If rand <p
SetDE-Flag=true; %%Active jDE for current population
fori = 1 to the NP do
UpdateF and R according to (14);
GenerateV
i
,
G using (7)
Generate the trial vectorU
i
,
G by (12)
End for
Else
SetPSO-Flag=true; %%Active PSO for current population
Updatew using (4)
fori = 1 to the NP do
Update particle Velocity and Position according to (1), (2) respectively.
Set the values of position to the trial vectorU
i
,
G.
End for
End if
Step  4.1. Randomly reinitialize the trial vector  U
i
,
G within the search
space if any variable is outside its boundaries.
Step  4.2.  Selection
fori = 1 to NP
Evaluate the trial vectorU
i
,
G
Iff
(
U
i
,
G
)
≤
f
(
X
i
,
G
)
X
i
,
G
+
1
=
U
i
,
G
,
f
(
X
i
,
G
+
1
)
=
f
(
U
i
,
G
)
P
i
,
p
best
=
U
i
,
G
,
f
(
P
i
,
p
best
)
=
f
(
U
i
,
G
)
Iff
(
U
i
,
G
)
<
f
(
X
best
,
G
)
X
best
,
G
=
U
i
,
G
,
f
(
X
best
,
G
)
=
f
(
U
i
,
G
)
p
g
=
U
i
,
G
,
f
(
p
g
)
=
f
(
U
i
,
G
)
End if
End if
End for
Step  4.3. Calculate parameter d using (15)
If parameter meets the requirement of (16)
Generate a random number rand in (0, 1).
If rand is less thanPS
O
p and PSO-Flag is true
Updateg
best using (17)
End if
If rand is less thanD
E
p and DE-Flag is true
UpdateX
i
,
G
+
1 using (18)
End if
End if
Step  4.4. Increment the generation count G
=
G
+
1;
ResetPSO-Flag=false; DE-Flag=false;
Step  5. End whileOne of the most important parameters for the proposed HPSO-DE is the balanced parameterp. In the following, we will make an integrated analysis of the key parameter by comparing the performance of the HPSO-DE in the optimization of several representative functions.


## 4. Numerical Experiments and Results

### 4.1. Test Functions

16 benchmark functions are used to test the performance of HPSO-DE to assure a fair comparison. If the number of test problems is smaller, it will be very difficult to make a general conclusion. Using a test set which is too small also has the potential risk that the algorithm is biased (optimized) toward the chosen set of problems. Such bias might not be useful for other problems of interest. The benchmark functions are given in Table1. It denotes the ranges of the variables and the value of the global optimum. Functions f
1
-
f
16 are high-dimensional problems. Functions f
1
-
f
6 are unimodal. Function f
7 is a noisy quadratic function. Functions f
8
-
f
16 are multimodal functions where the number of local minima increases exponentially with the problem dimension [29]. f
15
-
f
18 are rotated functions. An orthogonal matrix M is generated to rotate a function. The original variable x is left-multiplied by the orthogonal matrix M to get the new rotated variable y
=
M
×
x. This variable y is used to compute the fitness value f. When one dimension in x is changed, all dimensions in y will be influenced:
(19)
f
1
=
∑
i
=
1
D
x
i
2
,
f
2
=
∑
i
=
1
D
i
x
i
2
,
f
3
=
∑
i
=
1
D
|
x
i
|
+
∏
i
=
1
D
|
x
i
|
,
f
4
=
∑
i
=
1
D
(
∑
j
=
1
i
x
j
)
2
,
f
5
=
max
⁡
i
⁡
{
|
x
i
|
,
1
≤
x
i
≤
D
}
,
f
6
=
∑
i
=
1
D
-
1
[
100
(
x
i
+
1
-
x
i
2
)
2
+
(
x
i
-
1
)
2
]
,
f
7
=
∑
i
=
1
D
i
x
i
4
+
random
[
0,1
)
,
f
8
=
∑
i
=
1
D
[
x
i
2
-
10
cos
⁡
(
2
π
x
i
)
+
10
]
,
f
9
(
x
)
=
∑
i
=
1
D
(
y
i
2
-
10
cos
⁡
(
2
π
y
i
)
+
10
)
,
y
i
=
{
x
i
,
|
x
i
|
<
0.5
,
round
(
2
x
i
)
2
,
|
x
i
|
≥
0.5
,
f
10
=
-
20
exp
⁡
(
-
0.2
1
D
∑
i
=
1
D
x
i
2
)
-
exp
⁡
(
1
D
∑
i
=
1
D
cos
⁡
(
2
π
x
i
)
)
+
20
+
e
,
f
11
=
1
4000
∑
i
=
1
D
x
i
2
-
∏
i
=
1
D
cos
⁡
(
x
i
i
1
/
2
)
+
1
,
f
12
(
x
)
=
π
n
{
(
y
n
-
1
)
2
∑
i
=
1
D
(
y
i
-
1
)
2
10
si
n
2
(
π
y
i
)
+
∑
i
=
1
D
(
y
i
-
1
)
2
[
1
+
10
si
n
2
(
π
y
i
+
1
)
]
+
(
y
n
-
1
)
2
∑
i
=
1
D
(
y
i
-
1
)
2
}
+
∑
i
=
1
D
u
(
x
i
,
10,100,4
)
,
y
i
=
(
1
+
1
4
(
x
i
+
1
)
)
,
u
(
x
i
,
a
,
k
,
m
)
=
{
k
(
x
i
-
a
)
m
,
x
i
>
a
,
0
,
-
a
≤
x
i
≤
a
,
k
(
-
x
i
-
a
)
m
,
x
i
<
-
a
,
f
13
(
x
)
=
0.1
{
∑
i
=
1
D
-
1
(
x
i
-
1
)
2
si
n
2
(
3
π
x
1
)
+
∑
i
=
1
D
-
1
(
x
i
-
1
)
2
[
1
+
si
n
2
(
3
π
x
i
+
1
)
]
+
(
x
D
-
1
)
2
[
1
+
si
n
2
(
2
π
x
D
)
]
∑
i
=
1
D
-
1
(
x
i
-
1
)
2
}
+
∑
i
=
1
D
u
(
x
i
,
10,100,4
)
,
f
14
=
∑
i
=
1
D
(
∑
k
=
0
k
max
⁡
[
a
k
cos
⁡
(
2
π
b
k
(
x
i
+
0.5
)
)
]
-
D
∑
k
=
0
k
max
⁡
[
a
k
cos
⁡
(
2
π
b
k
*
0.5
)
]
)
,
a
=
0.5
,
b
=
3
,
k
max
⁡
=
20
,
f
15
=
∑
i
=
1
D
[
y
i
2
-
10
cos
⁡
(
2
π
y
i
)
+
10
]
,
y
=
M
×
x
,
f
16
(
x
)
=
∑
i
=
1
D
(
z
i
2
-
10
cos
⁡
(
2
π
z
i
)
+
10
)
,
z
i
=
{
y
i
,
|
y
i
|
<
0.5
,
round
(
2
y
i
)
2
,
|
y
i
|
≥
0.5
,
y
=
M
×
x
,
f
17
=
-
20
exp
⁡
(
-
0.2
1
D
∑
i
=
1
D
y
i
2
)
f
17
=
-
exp
⁡
(
1
D
∑
i
=
1
D
cos
⁡
(
2
π
y
i
)
)
+
20
+
e
,
f
17
=
-
20
(
-
0.2
1
D
∑
i
=
1
D
y
i
2
)
y
=
M
×
x
,
f
18
=
1
4000
∑
i
=
1
D
y
i
2
-
∏
i
=
1
D
cos
⁡
(
y
i
i
1
/
2
)
+
1
,
y
=
M
×
x
.Table 1
Benchmark configurations.
Function
Name
Search space
Global optimalf
(
x
*
)
x
*
f
1
Sphere
[
-
100
,
100
]
0
0
f
2
Weighted sphere
[
-
100
,
100
]
0
0
f
3
Schwefel’s Problem 2.22
[
-
10
,
10
]
0
0
f
4
Shifted Schwefel’s Problem 1.2
[
-
100
,
100
]
0
0
f
5
Schwefel’s Problem 2.21
[
-
100
,
100
]
0
0
f
6
Rosenbrock
[
-
30
,
30
]
0
(1, 1, …, 1)
f
7
Quartic
[
-
1.28
,
1.28
]
0
0
f
8
Rastrigin
[
-
5
,
5
]
0
0
f
9
Noncontinuous Rastigin
[
-
5
,
5
]
0
0
f
10
Ackley
[
-
32
,
32
]
0
0
f
11
Griewank
[
-
600
,
600
]
0
0
f
12
Penalized 1
[
-
50
,
50
]
0
0
f
13
Penalized 2
[
-
50
,
50
]
0
0
f
14
Weierstrass
[
-
0.5
,
0.5
]
0
0
f
15
Rotated Rastrigin
[
-
5
,
5
]
0
0
f
16
Rotated noncontinuous Rastrigin
[
-
5
,
5
]
0
0
f
17
Rotated Ackley
[
-
32
,
32
]
0
0
f
18
Rotated Griewank
[
-
600
,
600
]
0
0


### 4.2. Algorithms for Comparison

Experiments are conducted on a suite of 16 numerical functions to evaluate seven algorithms including the proposed HPSO-DE algorithm. For functions, 30-dimensional (30D) function is tested. The maximum number of function evaluations (FEs) is set to 300 000 andNP is 100. All experiments are run 25 times independently. The seven algorithms in comparison are listed in Table 2.Table 2
Algorithms initialization.
Algorithm
Parameters
Reference
SPSO
w
=
0.729, c
1
=
c
2
=
1.49
[41]
PSO-w
w = 0.9 : 0.4, c
1
=
c
2
=
2
[17]
PSO-TVAC
w = 0.9 : 0.4, c
1 = 2.5 : 0.5, c
2 = 0.5 : 2.5
[16, 38]
PSO-FDR
c
1
=
c
2
=
1, c
3
=
2
[42]
DE
F
=
0.5, CR
=
0.9
[20, 43]
jDE
F
l
=
0.1, F
u
=
0.9, τ
1
=
τ
2
=
0.1
[29]
HPSO-DE
PS
O
p
=
0.3, D
E
p
=
0.01, d
c
=
1.5, p
=
0
This paper


### 4.3. Comparisons on the Solution Accuracy

The mean and standard deviation (Std) of the solutions in 25 independent runs are listed in Table3. The best result among these algorithms is indicated by boldface in the table. Figures 1, 2, and 3 show the comparisons in terms of convergence, mean solutions, and evolution processes in solving 16 benchmark functions.Table 3
Result comparisons among seven algorithms on test functions.
Functions
Algorithms
PSO
PSO-w
PSO-FDR
PSO-TVAC
DE
jDE
HPSO-DE
f
1
Mean
1.3
1
E
-
75
1.6
1
E
-
22
2.3
9
E
-
95
4.14
E
-
83
1.0
4
E
-
30
3.8
4
E
-
31
0.0
E
+
00
Std.
4.03
E
-
149
7.47
E
-
44
1.2
9
E
-
188
1.6
2
E
-
164
6.81
E
-
61
2.14
E
-
61
0.0
E
+
00
f
2
Mean
7.44
E
-
76
2
.
80
E
-
21
9.2
8
E
-
94
4.76
E
-
83
1.3
3
E
-
29
5.8
8
E
-
21
0.0
E
+
00
Std.
4.3
5
E
-
150
1.4
7
E
-
41
2.0
9
E
-
185
2.0
E
-
164
1.3
6
E
-
58
8.63
E
-
40
0.0
E
+
00
f
3
Mean
2.50
E
-
41
1.99
E
-
15
6.10
E
-
45
3.88
E
-
36
7.51
E
-
15
5
.
5
7
E
-
03
8
0.0
E
+
00
Std.
3.3
6
E
-
81
1.1
5
E
-
29
6.6
4
E
-
88
3.6
8
E
-
70
5.05
E
-
29
1.4
5
E
-
75
0.0
E
+
00
f
4
Mean
1.83
E
-
7
4
.
70
E
+
1
6.77
E
-
7
5.74
E
-
12
1.1
7
E
-
30
1
.
80
E
-
60
0.0
E
+
00
Std.
2.21
E
-
13
9
.
39
E
+
2
5.42
E
-
13
1.80
E
-
22
1.92
E
-
60
6.01
E
-
120
0.0
E
+
00
f
5
Mean
2.6
3
E
-
7
1.0
E
+
0
4.2
6
E
-
6
1.91
E
-
7
1
.
59
E
-
2
5
.
92
E
-
2
0.0
E
+
00
Std.
5.7
9
E
-
14
3.0
E
-
1
5.1
2
E
-
11
2.5
2
E
-
13
8.5
9
E
-
4
3
.
3
1
E
-
2
0.0
E
+
00
f
6
Mean
2
.
2
4
E
+
1
4.89
E
+
1
1
.
4
1
E
+
1
1.10
E
+
1
1.10
E
+
1
3.0
4
E
+
0
2.86
E
+
2
Std.
3.69
E
+
2
1.5
3
E
+
3
3
.
1
4
E
+
2
3
.
50
E
+
2
5
.
09
E
+
2
2.39
E
+
0
9.0
E
-
3
f
7
Mean
2
.
0
E
-
3
1.2
E
-
2
4.0
E
-
3
3.0
E
-
3
4.0
E
-
3
7.0
E
-
3
1
.
2
E
-
5
Std.
1.16
E
-
6
2.60
E
-
5
3.18
E
-
6
2.0
1
E
-
6
2.3
3
E
-
6
1
.
70
E
+
0
1.04
9
E
-
10
f
8
Mean
4
.
3
1
E
+
1
2
.
44
E
+
1
2
.
45
E
+
1
2
.
57
E
+
1
1
.
42
E
+
2
0.0
E
+
00
0.0
E
+
00
Std.
1
.
31
E
+
2
4
.
94
E
+
1
3
.
27
E
+
1
6
.
10
E
+
1
5
.
11
E
+
2
0.0
E
+
00
0.0
E
+
00
f
9
Mean
1
.
37
E
+
1
1
.
86
E
+
1
1
.
24
E
+
1
2
.
33
E
+
1
1
.
16
E
+
2
4
.
0
E
-
2
0.0
E
+
00
Std.
7
.
75
E
+
1
1
.
90
E
+
1
1
.
61
E
+
1
2
.
43
E
+
1
2
.
82
E
+
2
4
.
0
E
-
2
0.0
E
+
00
f
10
Mean
1.14
E
-
1
2.9
2
-
E
12
1.8
2
E
-
14
3.08
E
-
14
6.2
9
E
-
15
4.44
E
-
15
8.88
2
E
-
16
Std.
1
.
55
E
-
1
6.88
E
-
24
1
.
30
E
-
29
8
.
60
E
-
29
3.28
E
-
30
0.0
E
+
00
0.0
E
+
00
f
11
Mean
1
.
5
E
-
2
1.5
E
-
2
1.0
E
-
2
1.9
E
-
2
0.0
E
+
00
0.0
E
+
00
0.0
E
+
00
Std.
2.4
4
E
-
4
3.0
E
-
4
8.97
E
-
5
4
.
40
E
-
4
0.0
E
+
00
0.0
E
+
00
0.0
E
+
00
f
12
Mean
9
.
1
E
-
2
4.0
E
-
3
1.57
E
-
32
2.90
E
-
2
1.87
E
-
31
1.5
8
E
-
32
1
.
3
E
-
2
Std.
3
.
2
E
-
2
4.0
E
-
4
3.12
E
-
95
4.0
E
-
3
3.3
9
E
-
62
3.12
E
-
95
3.30
E
-
5
f
13
Mean
6
.
34
E
+
1
1
.
45
E
+
1
1.3
5
E
-
32
4.44
E
+
0
1.4
9
E
+
0
8.33
E
-
22
2
.
7
6
E
-
1
Std.
1.07
E
+
3
2
.
1
3
E
+
2
3.12
E
-
95
5
.
2
6
E
+
1
1
.
68
E
+
1
1.73
E
-
41
1
.
2
E
-
2
f
14
Mean
1.67
E
+
0
2.28
E
-
5
6.65
E
-
2
5.5
E
-
2
2
.
79
E
+
1
0.0
E
+
00
0.0
E
+
00
Std.
2.0
3
E
+
0
5.10
E
-
9
1
.
18
E
-
2
3.0
E
-
3
2
.
7
8
E
+
1
0.0
E
+
00
0.0
E
+
00
f
15
Mean
9
.
2
5
E
+
1
8
.
24
E
+
1
5
.
2
3
E
+
1
8
.
0
1
E
+
1
1
.
86
E
+
2
1
.
0
7
E
+
2
0.0
E
+
00
Std.
4
.
84
E
+
2
5
.
01
E
+
2
5
.
99
E
+
2
6
.
77
E
+
2
5
.
82
E
+
1
1
.
97
E
+
2
0.0
E
+
00
f
16
Mean
1
.
1
7
E
+
2
1
.
20
E
+
2
7
.
52
E
+
1
9
.
38
E
+
1
1.52
E
+
2
7
.
29
E
+
1
0.0
E
+
00
Std.
1
.
33
E
+
3
1.00
E
+
3
8
.
81
E
+
2
7
.
02
E
+
2
9.76
E
+
1
1
.
28
E
+
2
0.0
E
+
00
f
17
Mean
2
.
09
E
+
1
2
.
09
E
+
1
2
.
09
E
+
1
2
.
09
E
+
1
2.09
E
+
1
2.10
E
+
1
4.18
E
+
0
Std.
2.6
E
-
3
4.3
E
-
3
2.2
E
-
3
2.1
E
-
3
1.80
E
-
3
2.2
E
-
3
7
.
29
E
+
1
f
18
Mean
1
.
15
E
-
2
1
.
17
E
-
2
1
.
87
E
-
2
1.76
E
-
2
1
.
60
E
-
16
6
.
4
7
E
-
1
0.0
E
+
00
Std.
1
E
-
4
1
E
-
4
1
E
-
4
2
E
-
4
2.0
6
E
-
31
2
.
7
4
E
-
1
0.0
E
+
00Figure 1
Performance of the algorithms forf
1, f
2, f
3, f
4, f
5, f
6, f
7, and f
8.Figure 2
Performance of the algorithms forf
9, f
10
f
11, f
12, f
13, f
14, f
15, and f
16.Figure 3
Performance of the algorithms forf
17 and f
18.From Table3 and Figures 1–3, it is very clear that the hybrid proposed algorithm has the strong ability to jump out of the local optima. It can effectively prevent the premature convergence and significantly enhance the convergence rate and accuracy. It provides best performance on the f
1
,
f
2
,
f
3
,
f
4
,
f
5
,
f
8
,
f
9
,
f
11
,
f
14
,
f
15
,
f
16, and f
18, which reachse the highest accuracy on them. The jDE ranks the second on f
8, and f
11 and performs a little better than HPSO-DE on f
6. PSO-FDR performs best on f
12
,
f
13.One can observe that the proposed method can search the optimum and maintain a higher convergence speed. The capabilities of avoiding local optima and finding global optimum of these functions indicate the superiority of HPSO-DE.


### 4.4. Comparisons on Convergent Rate and Successful Percentage

The convergent rate for achieving the global optimum is another key point for testing the algorithm performance. The success of an algorithm means that this algorithm can result in a function value not worse than the prespecified optimal value, that is, for all problems with the number of function evaluations less than the pre specified maximum number. The success rate (SR) is calculated as the number of successful runs divided by the total number of runs.In Table4, we summarize the SR of each algorithm and the average number of function evaluations over successful runs (FESS). An experiment is considered as successful if the best solution is found with sufficient accuracy: 10
-
8.Table 4
Convergence speed and algorithm reliability comparisons.
Functions
Algorithms
PSO
PSO-w
PSO-FDR
PSO-TVAC
DE
jDE
HPSO-DE
f
1
FESS
4
.
5056
E
+
4
2
.
3642
+
E
5
1
.
4738
E
+
5
1
.
014
2
E
+
5
1.1055
E
+
5
1.0883
E
+
5
2.2075
E
+
4
SR
100
%
100
%
100
%
100
%
100
%
100
%
100
%
f
2
FESS
4
.
9428
E
+
4
2
.
4217
E
+
5
1
.
5194
E
+
5
1
.
043
5
E
+
5
1.1905
E
+
5
6.4498
E
+
4
2.3002
E
+
4
SR
100
%
100
%
100
%
100
%
100
%
100
%
100
%
f
3
FESS
6
.
8336
E
+
4
2
.
5089
E
+
5
1
.
638
2
E
+
5
1
.
1524
E
+
5
1.8979
E
+
5
7.6644
E
+
4
3.2762
E
+
4
SR
100
%
100
%
100
%
100
%
100
%
100
%
100
%
f
4
FESS
2
.
907
E
+
5
—
—
2
.
4757
E
+
5
1.1041
E
+
5
5.9383
E
+
4
2.2075
E
+
4
SR
12
%
0%
0%
100%
100%
100%
100
%
f
5
FESS
2
.
987
E
+
5
—
—
2
.
8725
E
+
5
—
—
3.2629
E
+
4
SR
4%
0%
0%
24%
0%
0%
100
%
f
8
FESS
—
—
—
—
—
7.3477
E
+
4
2.3084
E
+
4
SR
0%
0%
0%
0%
0%
100
%
100
%
f
9
FESS
—
—
—
—
8.7133
E
+
4
2.2356
E
+
4
SR
0%
0%
0%
0%
96%
100
%
f
10
FESS
7.337
E
+
4
2
.
7072
E
+
5
1
.
722
6
E
+
5
1
.
2118
E
+
5
1.7113
E
+
5
9.0429
E
+
4
3.2502
E
+
4
SR
92
%
100%
100%
100%
100%
100%
100
%
f
11
FESS
4
.
6325
E
+
4
2.3907
E
+
5
1.5296
E
+
5
1.0411
E
+
5
1.1398
E
+
5
6.2468
E
+
4
2.0659
E
+
4
SR
32
%
28%
36%
36%
100%
100%
100%
f
12
FESS
4.6978
E
+
4
2.4440
E
+
5
1
.
3834
E
+
5
1
.
087
2
E
+
5
1.0199
E
+
5
5.0201
E
+
4
—
SR
72
%
96%
100%
80%
100%
100%
0%
f
13
FESS
—
2
.
975
E
+
5
1
.
6684
E
+
5
1.4981
E
+
5
2.5882
E
+
5
5.7271
E
+
4
—
SR
0%
4%
100%
60%
52%
100%
0%
f
14
FESS
—
2.7346
E
+
5
—
—
—
8.435
E
+
4
3.85E+4
SR
0%
88%
0%
0%
0%
100%
100
%
f
15
FESS
—
—
—
—
—
—
2.1025
E
+
4
SR
0%
0%
0%
0%
0%
0%
100
%
f
16
FESS
—
—
—
—
—
—
2.2563
E
+
4
SR
0%
0%
0%
0%
0%
0%
100
%
f
17
FESS
—
—
—
—
—
—
6.8878
E
+
4
SR
0%
0%
0%
0%
0%
0%
80
%
f
18
FESS
1.0931
E
+
5
—
—
1
.
4448
E
+
5
1.834
E
+
5
—
2.268
E
+
4
SR
36%
0%
0%
20%
100%
0%
100%Table4 shows that HPSO-DE needs least FESS to achieve the acceptable solution on most of functions, which reveals that proposed algorithm has a higher convergent rate than other algorithms. DE and jDE outperform HPSO-DE on the f
12 and f
13; SPSO, LPSO, and PSO-TVAC have much worse SR and accuracy than HPSO-DE on the test functions. In addition, HPSO-DE can achieve accepted value with a good convergence speed and accuracy on most of the functions, as seen from Figures 1–3 and Table 3.In summary, the HPSO-DE performs best on functions and has good search ability. Owing to the proposed techniques, the HPSO-DE processes capabilities of fast convergence speed, the highest successful rate, and the best search accuracy among these algorithms.


### 4.5. Parameter Study

The balanced parameterp needs to be optimized. In this section, we investigate the impact of this parameter on HPSO-DE. The HPSO-DE algorithm runs 25 times on each function with four different balanced parameters of 0, 0.1, 0.2, and 0.3. The influence of balanced parameters on accuracy of HPSO-DE algorithm is investigated by comparing the optima values that HPSO-DE obtains for different balanced parameters.Figures4, 5, and 6 show the box plots of minimal values that HPSO-DE obtains with four different balanced parameters. The box has lines at the lower quartile, median, and upper quartile values. The whiskers are lines extending from each end of the box to show the extent of the remaining data. Outliers are data with values beyond the ends of the whiskers.Figure 4
HPSO-DE with different balance parameters and other six algorithms onf
1, f
2, f
3, f
4, f
5, and f
6.Figure 5
HPSO-DE with different balance parameters and other six algorithms onf
7, f
8, f
9, f
10, f
11, f
12, f
13, and f
14.Figure 6
HPSO-DE with different balance parameters and other six algorithms onf
15, f
16, f
17, and f
18.From Figures4–6, one can observe that the accuracy of HPSO-DE is less sensitive to the balanced parameter on most of functions except f
12
,
f
13, and f
17 when balanced parameter is between 0 and 0.3.


### 4.6. Comparison with JADE

The JADE algorithm is tested on a set of standard test functions in [31]. HPSO-DE is compared with JADE on 30D test functions chosen from [31]. The parameter settings are the same as in [31]. Maximum generations are listed in Table 5. The middle results of 50 independent runs are summarized in the table (results for JADE are taken from [31]), which show that the proposed algorithm obviously performs better than the JADE algorithm.Table 5
Results of the comparison between HPSO-DE and JADE.
Functions
Gen
HPSO-DEMean (Std Dev.)
JADE w/o archiveMean (Std Dev.)
JADE with archiveMean (Std Dev.)
f
1
1500
2
.
9
E
-
121
(
1.0
E
-
240
)
1.8
E
-
60 (8.4
E
-
60
)
1.3
E
-
54 (9.2
E
-
54
)
f
3
2000
1.7
E
-
78
(4
.
5
E
-
155
)
1.8
E
-
25 (8.8
E
-
25
)
3.9
E
-
22 (2.7
E
-
21
)
f
4
5000
0.0
E
+
00
(0.0
E
+
00
)
5.7
E
-
61 (2.7
E
-
60
)
6.0
E
-
87 (1.9
E
-
86
)
f
5
5000
1.7
E
-
185
(0
0.0
E
+
00
)
8.2
E
-
24 (4.0
E
-
23
)
4.3
E
-
66 (1.2
E
-
65
)
f
6
3000
2
.
9
+
E
2 (1
.
1
E
-
2
)
8.0
E
-
02
(5.6
E
-
01
)
3.2
E
-
01 (1.1
E
+
00
)
20000
2
.
9
+
E
2 (3
.
4
E
-
2
)
8.0
E
-
02
(5.6
E
-
01
)
3.2
E
-
01 (1.1
E
+
00
)
f
7
3000
2.1
E
-
5
(2
.
2
E
-
10
)
6.4
E
-
04 (2.5
E
-
04
)
6.8
E
-
04 (2.5
E
-
04
)
f
8
1000
0.0
E
+
00
(0.0
E
+
00
)
1.0
E
-
04 (6.0
E
-
05
)
1.4
E
-
04 (6.5
E
-
05
)
5000
0.0
E
+
00
(0.0
E
+
00
)
0.0
E
+
00
(0.0
E
+
00
)
0.0
E
+
00
(0.0
E
+
00
)
f
10
500
8
.
9
E
-
16
(
0.0
E
+
00
)
8.2
E
-
10 (6.9
E
-
10
)
3.0
E
-
09 (2.2
E
-
09
)
2000
8
.
9
E
-
16
(0.0
E
+
00
)
4.4
E
-
15 (0.0
E
+
00
)
4.4
E
-
15 (0.0
E
+
00
)
f
11
500
0.0
E
+
00
(0.0
E
+
00
)
9.9
E
-
08 (6.0
E
-
07
)
2.0
E
-
04 (1.4
E
-
03
)
3000
0.0
E
+
00
(0.0
E
+
00
)
0.0
E
+
00
(0.0
E
+
00
)
2.0
E
-
04 (1.4
E
-
03
)
f
12
500
1.8
E
-
2 (1
E
-
4
)
4.6
E
-
17
(1.9
E
-
16
)
3.8
E
-
16 (8.3
E
-
16
)
1500
1.7
E
-
2 (4
E
-
4
)
1.6
E
-
32
(5.5
E
-
48
)
1.6
E
-
32
(5.5
E
-
48
)
f
13
500
4
.
4
E
-
1 (2
.
1
E
-
2
)
2.0
E
-
16
(6.5
E
-
16
)
1.2
E
-
15 (2.8
E
-
15
)
1500
3.1
E
-
1 (1
.
6
E
-
2
)
1.4
E
-
32
(1.1
E
-
47
)
1.4
E
-
32
(1.1
E
-
47
)


## 4.1. Test Functions

16 benchmark functions are used to test the performance of HPSO-DE to assure a fair comparison. If the number of test problems is smaller, it will be very difficult to make a general conclusion. Using a test set which is too small also has the potential risk that the algorithm is biased (optimized) toward the chosen set of problems. Such bias might not be useful for other problems of interest. The benchmark functions are given in Table1. It denotes the ranges of the variables and the value of the global optimum. Functions f
1
-
f
16 are high-dimensional problems. Functions f
1
-
f
6 are unimodal. Function f
7 is a noisy quadratic function. Functions f
8
-
f
16 are multimodal functions where the number of local minima increases exponentially with the problem dimension [29]. f
15
-
f
18 are rotated functions. An orthogonal matrix M is generated to rotate a function. The original variable x is left-multiplied by the orthogonal matrix M to get the new rotated variable y
=
M
×
x. This variable y is used to compute the fitness value f. When one dimension in x is changed, all dimensions in y will be influenced:
(19)
f
1
=
∑
i
=
1
D
x
i
2
,
f
2
=
∑
i
=
1
D
i
x
i
2
,
f
3
=
∑
i
=
1
D
|
x
i
|
+
∏
i
=
1
D
|
x
i
|
,
f
4
=
∑
i
=
1
D
(
∑
j
=
1
i
x
j
)
2
,
f
5
=
max
⁡
i
⁡
{
|
x
i
|
,
1
≤
x
i
≤
D
}
,
f
6
=
∑
i
=
1
D
-
1
[
100
(
x
i
+
1
-
x
i
2
)
2
+
(
x
i
-
1
)
2
]
,
f
7
=
∑
i
=
1
D
i
x
i
4
+
random
[
0,1
)
,
f
8
=
∑
i
=
1
D
[
x
i
2
-
10
cos
⁡
(
2
π
x
i
)
+
10
]
,
f
9
(
x
)
=
∑
i
=
1
D
(
y
i
2
-
10
cos
⁡
(
2
π
y
i
)
+
10
)
,
y
i
=
{
x
i
,
|
x
i
|
<
0.5
,
round
(
2
x
i
)
2
,
|
x
i
|
≥
0.5
,
f
10
=
-
20
exp
⁡
(
-
0.2
1
D
∑
i
=
1
D
x
i
2
)
-
exp
⁡
(
1
D
∑
i
=
1
D
cos
⁡
(
2
π
x
i
)
)
+
20
+
e
,
f
11
=
1
4000
∑
i
=
1
D
x
i
2
-
∏
i
=
1
D
cos
⁡
(
x
i
i
1
/
2
)
+
1
,
f
12
(
x
)
=
π
n
{
(
y
n
-
1
)
2
∑
i
=
1
D
(
y
i
-
1
)
2
10
si
n
2
(
π
y
i
)
+
∑
i
=
1
D
(
y
i
-
1
)
2
[
1
+
10
si
n
2
(
π
y
i
+
1
)
]
+
(
y
n
-
1
)
2
∑
i
=
1
D
(
y
i
-
1
)
2
}
+
∑
i
=
1
D
u
(
x
i
,
10,100,4
)
,
y
i
=
(
1
+
1
4
(
x
i
+
1
)
)
,
u
(
x
i
,
a
,
k
,
m
)
=
{
k
(
x
i
-
a
)
m
,
x
i
>
a
,
0
,
-
a
≤
x
i
≤
a
,
k
(
-
x
i
-
a
)
m
,
x
i
<
-
a
,
f
13
(
x
)
=
0.1
{
∑
i
=
1
D
-
1
(
x
i
-
1
)
2
si
n
2
(
3
π
x
1
)
+
∑
i
=
1
D
-
1
(
x
i
-
1
)
2
[
1
+
si
n
2
(
3
π
x
i
+
1
)
]
+
(
x
D
-
1
)
2
[
1
+
si
n
2
(
2
π
x
D
)
]
∑
i
=
1
D
-
1
(
x
i
-
1
)
2
}
+
∑
i
=
1
D
u
(
x
i
,
10,100,4
)
,
f
14
=
∑
i
=
1
D
(
∑
k
=
0
k
max
⁡
[
a
k
cos
⁡
(
2
π
b
k
(
x
i
+
0.5
)
)
]
-
D
∑
k
=
0
k
max
⁡
[
a
k
cos
⁡
(
2
π
b
k
*
0.5
)
]
)
,
a
=
0.5
,
b
=
3
,
k
max
⁡
=
20
,
f
15
=
∑
i
=
1
D
[
y
i
2
-
10
cos
⁡
(
2
π
y
i
)
+
10
]
,
y
=
M
×
x
,
f
16
(
x
)
=
∑
i
=
1
D
(
z
i
2
-
10
cos
⁡
(
2
π
z
i
)
+
10
)
,
z
i
=
{
y
i
,
|
y
i
|
<
0.5
,
round
(
2
y
i
)
2
,
|
y
i
|
≥
0.5
,
y
=
M
×
x
,
f
17
=
-
20
exp
⁡
(
-
0.2
1
D
∑
i
=
1
D
y
i
2
)
f
17
=
-
exp
⁡
(
1
D
∑
i
=
1
D
cos
⁡
(
2
π
y
i
)
)
+
20
+
e
,
f
17
=
-
20
(
-
0.2
1
D
∑
i
=
1
D
y
i
2
)
y
=
M
×
x
,
f
18
=
1
4000
∑
i
=
1
D
y
i
2
-
∏
i
=
1
D
cos
⁡
(
y
i
i
1
/
2
)
+
1
,
y
=
M
×
x
.Table 1
Benchmark configurations.
Function
Name
Search space
Global optimalf
(
x
*
)
x
*
f
1
Sphere
[
-
100
,
100
]
0
0
f
2
Weighted sphere
[
-
100
,
100
]
0
0
f
3
Schwefel’s Problem 2.22
[
-
10
,
10
]
0
0
f
4
Shifted Schwefel’s Problem 1.2
[
-
100
,
100
]
0
0
f
5
Schwefel’s Problem 2.21
[
-
100
,
100
]
0
0
f
6
Rosenbrock
[
-
30
,
30
]
0
(1, 1, …, 1)
f
7
Quartic
[
-
1.28
,
1.28
]
0
0
f
8
Rastrigin
[
-
5
,
5
]
0
0
f
9
Noncontinuous Rastigin
[
-
5
,
5
]
0
0
f
10
Ackley
[
-
32
,
32
]
0
0
f
11
Griewank
[
-
600
,
600
]
0
0
f
12
Penalized 1
[
-
50
,
50
]
0
0
f
13
Penalized 2
[
-
50
,
50
]
0
0
f
14
Weierstrass
[
-
0.5
,
0.5
]
0
0
f
15
Rotated Rastrigin
[
-
5
,
5
]
0
0
f
16
Rotated noncontinuous Rastrigin
[
-
5
,
5
]
0
0
f
17
Rotated Ackley
[
-
32
,
32
]
0
0
f
18
Rotated Griewank
[
-
600
,
600
]
0
0


## 4.2. Algorithms for Comparison

Experiments are conducted on a suite of 16 numerical functions to evaluate seven algorithms including the proposed HPSO-DE algorithm. For functions, 30-dimensional (30D) function is tested. The maximum number of function evaluations (FEs) is set to 300 000 andNP is 100. All experiments are run 25 times independently. The seven algorithms in comparison are listed in Table 2.Table 2
Algorithms initialization.
Algorithm
Parameters
Reference
SPSO
w
=
0.729, c
1
=
c
2
=
1.49
[41]
PSO-w
w = 0.9 : 0.4, c
1
=
c
2
=
2
[17]
PSO-TVAC
w = 0.9 : 0.4, c
1 = 2.5 : 0.5, c
2 = 0.5 : 2.5
[16, 38]
PSO-FDR
c
1
=
c
2
=
1, c
3
=
2
[42]
DE
F
=
0.5, CR
=
0.9
[20, 43]
jDE
F
l
=
0.1, F
u
=
0.9, τ
1
=
τ
2
=
0.1
[29]
HPSO-DE
PS
O
p
=
0.3, D
E
p
=
0.01, d
c
=
1.5, p
=
0
This paper


## 4.3. Comparisons on the Solution Accuracy

The mean and standard deviation (Std) of the solutions in 25 independent runs are listed in Table3. The best result among these algorithms is indicated by boldface in the table. Figures 1, 2, and 3 show the comparisons in terms of convergence, mean solutions, and evolution processes in solving 16 benchmark functions.Table 3
Result comparisons among seven algorithms on test functions.
Functions
Algorithms
PSO
PSO-w
PSO-FDR
PSO-TVAC
DE
jDE
HPSO-DE
f
1
Mean
1.3
1
E
-
75
1.6
1
E
-
22
2.3
9
E
-
95
4.14
E
-
83
1.0
4
E
-
30
3.8
4
E
-
31
0.0
E
+
00
Std.
4.03
E
-
149
7.47
E
-
44
1.2
9
E
-
188
1.6
2
E
-
164
6.81
E
-
61
2.14
E
-
61
0.0
E
+
00
f
2
Mean
7.44
E
-
76
2
.
80
E
-
21
9.2
8
E
-
94
4.76
E
-
83
1.3
3
E
-
29
5.8
8
E
-
21
0.0
E
+
00
Std.
4.3
5
E
-
150
1.4
7
E
-
41
2.0
9
E
-
185
2.0
E
-
164
1.3
6
E
-
58
8.63
E
-
40
0.0
E
+
00
f
3
Mean
2.50
E
-
41
1.99
E
-
15
6.10
E
-
45
3.88
E
-
36
7.51
E
-
15
5
.
5
7
E
-
03
8
0.0
E
+
00
Std.
3.3
6
E
-
81
1.1
5
E
-
29
6.6
4
E
-
88
3.6
8
E
-
70
5.05
E
-
29
1.4
5
E
-
75
0.0
E
+
00
f
4
Mean
1.83
E
-
7
4
.
70
E
+
1
6.77
E
-
7
5.74
E
-
12
1.1
7
E
-
30
1
.
80
E
-
60
0.0
E
+
00
Std.
2.21
E
-
13
9
.
39
E
+
2
5.42
E
-
13
1.80
E
-
22
1.92
E
-
60
6.01
E
-
120
0.0
E
+
00
f
5
Mean
2.6
3
E
-
7
1.0
E
+
0
4.2
6
E
-
6
1.91
E
-
7
1
.
59
E
-
2
5
.
92
E
-
2
0.0
E
+
00
Std.
5.7
9
E
-
14
3.0
E
-
1
5.1
2
E
-
11
2.5
2
E
-
13
8.5
9
E
-
4
3
.
3
1
E
-
2
0.0
E
+
00
f
6
Mean
2
.
2
4
E
+
1
4.89
E
+
1
1
.
4
1
E
+
1
1.10
E
+
1
1.10
E
+
1
3.0
4
E
+
0
2.86
E
+
2
Std.
3.69
E
+
2
1.5
3
E
+
3
3
.
1
4
E
+
2
3
.
50
E
+
2
5
.
09
E
+
2
2.39
E
+
0
9.0
E
-
3
f
7
Mean
2
.
0
E
-
3
1.2
E
-
2
4.0
E
-
3
3.0
E
-
3
4.0
E
-
3
7.0
E
-
3
1
.
2
E
-
5
Std.
1.16
E
-
6
2.60
E
-
5
3.18
E
-
6
2.0
1
E
-
6
2.3
3
E
-
6
1
.
70
E
+
0
1.04
9
E
-
10
f
8
Mean
4
.
3
1
E
+
1
2
.
44
E
+
1
2
.
45
E
+
1
2
.
57
E
+
1
1
.
42
E
+
2
0.0
E
+
00
0.0
E
+
00
Std.
1
.
31
E
+
2
4
.
94
E
+
1
3
.
27
E
+
1
6
.
10
E
+
1
5
.
11
E
+
2
0.0
E
+
00
0.0
E
+
00
f
9
Mean
1
.
37
E
+
1
1
.
86
E
+
1
1
.
24
E
+
1
2
.
33
E
+
1
1
.
16
E
+
2
4
.
0
E
-
2
0.0
E
+
00
Std.
7
.
75
E
+
1
1
.
90
E
+
1
1
.
61
E
+
1
2
.
43
E
+
1
2
.
82
E
+
2
4
.
0
E
-
2
0.0
E
+
00
f
10
Mean
1.14
E
-
1
2.9
2
-
E
12
1.8
2
E
-
14
3.08
E
-
14
6.2
9
E
-
15
4.44
E
-
15
8.88
2
E
-
16
Std.
1
.
55
E
-
1
6.88
E
-
24
1
.
30
E
-
29
8
.
60
E
-
29
3.28
E
-
30
0.0
E
+
00
0.0
E
+
00
f
11
Mean
1
.
5
E
-
2
1.5
E
-
2
1.0
E
-
2
1.9
E
-
2
0.0
E
+
00
0.0
E
+
00
0.0
E
+
00
Std.
2.4
4
E
-
4
3.0
E
-
4
8.97
E
-
5
4
.
40
E
-
4
0.0
E
+
00
0.0
E
+
00
0.0
E
+
00
f
12
Mean
9
.
1
E
-
2
4.0
E
-
3
1.57
E
-
32
2.90
E
-
2
1.87
E
-
31
1.5
8
E
-
32
1
.
3
E
-
2
Std.
3
.
2
E
-
2
4.0
E
-
4
3.12
E
-
95
4.0
E
-
3
3.3
9
E
-
62
3.12
E
-
95
3.30
E
-
5
f
13
Mean
6
.
34
E
+
1
1
.
45
E
+
1
1.3
5
E
-
32
4.44
E
+
0
1.4
9
E
+
0
8.33
E
-
22
2
.
7
6
E
-
1
Std.
1.07
E
+
3
2
.
1
3
E
+
2
3.12
E
-
95
5
.
2
6
E
+
1
1
.
68
E
+
1
1.73
E
-
41
1
.
2
E
-
2
f
14
Mean
1.67
E
+
0
2.28
E
-
5
6.65
E
-
2
5.5
E
-
2
2
.
79
E
+
1
0.0
E
+
00
0.0
E
+
00
Std.
2.0
3
E
+
0
5.10
E
-
9
1
.
18
E
-
2
3.0
E
-
3
2
.
7
8
E
+
1
0.0
E
+
00
0.0
E
+
00
f
15
Mean
9
.
2
5
E
+
1
8
.
24
E
+
1
5
.
2
3
E
+
1
8
.
0
1
E
+
1
1
.
86
E
+
2
1
.
0
7
E
+
2
0.0
E
+
00
Std.
4
.
84
E
+
2
5
.
01
E
+
2
5
.
99
E
+
2
6
.
77
E
+
2
5
.
82
E
+
1
1
.
97
E
+
2
0.0
E
+
00
f
16
Mean
1
.
1
7
E
+
2
1
.
20
E
+
2
7
.
52
E
+
1
9
.
38
E
+
1
1.52
E
+
2
7
.
29
E
+
1
0.0
E
+
00
Std.
1
.
33
E
+
3
1.00
E
+
3
8
.
81
E
+
2
7
.
02
E
+
2
9.76
E
+
1
1
.
28
E
+
2
0.0
E
+
00
f
17
Mean
2
.
09
E
+
1
2
.
09
E
+
1
2
.
09
E
+
1
2
.
09
E
+
1
2.09
E
+
1
2.10
E
+
1
4.18
E
+
0
Std.
2.6
E
-
3
4.3
E
-
3
2.2
E
-
3
2.1
E
-
3
1.80
E
-
3
2.2
E
-
3
7
.
29
E
+
1
f
18
Mean
1
.
15
E
-
2
1
.
17
E
-
2
1
.
87
E
-
2
1.76
E
-
2
1
.
60
E
-
16
6
.
4
7
E
-
1
0.0
E
+
00
Std.
1
E
-
4
1
E
-
4
1
E
-
4
2
E
-
4
2.0
6
E
-
31
2
.
7
4
E
-
1
0.0
E
+
00Figure 1
Performance of the algorithms forf
1, f
2, f
3, f
4, f
5, f
6, f
7, and f
8.Figure 2
Performance of the algorithms forf
9, f
10
f
11, f
12, f
13, f
14, f
15, and f
16.Figure 3
Performance of the algorithms forf
17 and f
18.From Table3 and Figures 1–3, it is very clear that the hybrid proposed algorithm has the strong ability to jump out of the local optima. It can effectively prevent the premature convergence and significantly enhance the convergence rate and accuracy. It provides best performance on the f
1
,
f
2
,
f
3
,
f
4
,
f
5
,
f
8
,
f
9
,
f
11
,
f
14
,
f
15
,
f
16, and f
18, which reachse the highest accuracy on them. The jDE ranks the second on f
8, and f
11 and performs a little better than HPSO-DE on f
6. PSO-FDR performs best on f
12
,
f
13.One can observe that the proposed method can search the optimum and maintain a higher convergence speed. The capabilities of avoiding local optima and finding global optimum of these functions indicate the superiority of HPSO-DE.


## 4.4. Comparisons on Convergent Rate and Successful Percentage

The convergent rate for achieving the global optimum is another key point for testing the algorithm performance. The success of an algorithm means that this algorithm can result in a function value not worse than the prespecified optimal value, that is, for all problems with the number of function evaluations less than the pre specified maximum number. The success rate (SR) is calculated as the number of successful runs divided by the total number of runs.In Table4, we summarize the SR of each algorithm and the average number of function evaluations over successful runs (FESS). An experiment is considered as successful if the best solution is found with sufficient accuracy: 10
-
8.Table 4
Convergence speed and algorithm reliability comparisons.
Functions
Algorithms
PSO
PSO-w
PSO-FDR
PSO-TVAC
DE
jDE
HPSO-DE
f
1
FESS
4
.
5056
E
+
4
2
.
3642
+
E
5
1
.
4738
E
+
5
1
.
014
2
E
+
5
1.1055
E
+
5
1.0883
E
+
5
2.2075
E
+
4
SR
100
%
100
%
100
%
100
%
100
%
100
%
100
%
f
2
FESS
4
.
9428
E
+
4
2
.
4217
E
+
5
1
.
5194
E
+
5
1
.
043
5
E
+
5
1.1905
E
+
5
6.4498
E
+
4
2.3002
E
+
4
SR
100
%
100
%
100
%
100
%
100
%
100
%
100
%
f
3
FESS
6
.
8336
E
+
4
2
.
5089
E
+
5
1
.
638
2
E
+
5
1
.
1524
E
+
5
1.8979
E
+
5
7.6644
E
+
4
3.2762
E
+
4
SR
100
%
100
%
100
%
100
%
100
%
100
%
100
%
f
4
FESS
2
.
907
E
+
5
—
—
2
.
4757
E
+
5
1.1041
E
+
5
5.9383
E
+
4
2.2075
E
+
4
SR
12
%
0%
0%
100%
100%
100%
100
%
f
5
FESS
2
.
987
E
+
5
—
—
2
.
8725
E
+
5
—
—
3.2629
E
+
4
SR
4%
0%
0%
24%
0%
0%
100
%
f
8
FESS
—
—
—
—
—
7.3477
E
+
4
2.3084
E
+
4
SR
0%
0%
0%
0%
0%
100
%
100
%
f
9
FESS
—
—
—
—
8.7133
E
+
4
2.2356
E
+
4
SR
0%
0%
0%
0%
96%
100
%
f
10
FESS
7.337
E
+
4
2
.
7072
E
+
5
1
.
722
6
E
+
5
1
.
2118
E
+
5
1.7113
E
+
5
9.0429
E
+
4
3.2502
E
+
4
SR
92
%
100%
100%
100%
100%
100%
100
%
f
11
FESS
4
.
6325
E
+
4
2.3907
E
+
5
1.5296
E
+
5
1.0411
E
+
5
1.1398
E
+
5
6.2468
E
+
4
2.0659
E
+
4
SR
32
%
28%
36%
36%
100%
100%
100%
f
12
FESS
4.6978
E
+
4
2.4440
E
+
5
1
.
3834
E
+
5
1
.
087
2
E
+
5
1.0199
E
+
5
5.0201
E
+
4
—
SR
72
%
96%
100%
80%
100%
100%
0%
f
13
FESS
—
2
.
975
E
+
5
1
.
6684
E
+
5
1.4981
E
+
5
2.5882
E
+
5
5.7271
E
+
4
—
SR
0%
4%
100%
60%
52%
100%
0%
f
14
FESS
—
2.7346
E
+
5
—
—
—
8.435
E
+
4
3.85E+4
SR
0%
88%
0%
0%
0%
100%
100
%
f
15
FESS
—
—
—
—
—
—
2.1025
E
+
4
SR
0%
0%
0%
0%
0%
0%
100
%
f
16
FESS
—
—
—
—
—
—
2.2563
E
+
4
SR
0%
0%
0%
0%
0%
0%
100
%
f
17
FESS
—
—
—
—
—
—
6.8878
E
+
4
SR
0%
0%
0%
0%
0%
0%
80
%
f
18
FESS
1.0931
E
+
5
—
—
1
.
4448
E
+
5
1.834
E
+
5
—
2.268
E
+
4
SR
36%
0%
0%
20%
100%
0%
100%Table4 shows that HPSO-DE needs least FESS to achieve the acceptable solution on most of functions, which reveals that proposed algorithm has a higher convergent rate than other algorithms. DE and jDE outperform HPSO-DE on the f
12 and f
13; SPSO, LPSO, and PSO-TVAC have much worse SR and accuracy than HPSO-DE on the test functions. In addition, HPSO-DE can achieve accepted value with a good convergence speed and accuracy on most of the functions, as seen from Figures 1–3 and Table 3.In summary, the HPSO-DE performs best on functions and has good search ability. Owing to the proposed techniques, the HPSO-DE processes capabilities of fast convergence speed, the highest successful rate, and the best search accuracy among these algorithms.


## 4.5. Parameter Study

The balanced parameterp needs to be optimized. In this section, we investigate the impact of this parameter on HPSO-DE. The HPSO-DE algorithm runs 25 times on each function with four different balanced parameters of 0, 0.1, 0.2, and 0.3. The influence of balanced parameters on accuracy of HPSO-DE algorithm is investigated by comparing the optima values that HPSO-DE obtains for different balanced parameters.Figures4, 5, and 6 show the box plots of minimal values that HPSO-DE obtains with four different balanced parameters. The box has lines at the lower quartile, median, and upper quartile values. The whiskers are lines extending from each end of the box to show the extent of the remaining data. Outliers are data with values beyond the ends of the whiskers.Figure 4
HPSO-DE with different balance parameters and other six algorithms onf
1, f
2, f
3, f
4, f
5, and f
6.Figure 5
HPSO-DE with different balance parameters and other six algorithms onf
7, f
8, f
9, f
10, f
11, f
12, f
13, and f
14.Figure 6
HPSO-DE with different balance parameters and other six algorithms onf
15, f
16, f
17, and f
18.From Figures4–6, one can observe that the accuracy of HPSO-DE is less sensitive to the balanced parameter on most of functions except f
12
,
f
13, and f
17 when balanced parameter is between 0 and 0.3.


## 4.6. Comparison with JADE

The JADE algorithm is tested on a set of standard test functions in [31]. HPSO-DE is compared with JADE on 30D test functions chosen from [31]. The parameter settings are the same as in [31]. Maximum generations are listed in Table 5. The middle results of 50 independent runs are summarized in the table (results for JADE are taken from [31]), which show that the proposed algorithm obviously performs better than the JADE algorithm.Table 5
Results of the comparison between HPSO-DE and JADE.
Functions
Gen
HPSO-DEMean (Std Dev.)
JADE w/o archiveMean (Std Dev.)
JADE with archiveMean (Std Dev.)
f
1
1500
2
.
9
E
-
121
(
1.0
E
-
240
)
1.8
E
-
60 (8.4
E
-
60
)
1.3
E
-
54 (9.2
E
-
54
)
f
3
2000
1.7
E
-
78
(4
.
5
E
-
155
)
1.8
E
-
25 (8.8
E
-
25
)
3.9
E
-
22 (2.7
E
-
21
)
f
4
5000
0.0
E
+
00
(0.0
E
+
00
)
5.7
E
-
61 (2.7
E
-
60
)
6.0
E
-
87 (1.9
E
-
86
)
f
5
5000
1.7
E
-
185
(0
0.0
E
+
00
)
8.2
E
-
24 (4.0
E
-
23
)
4.3
E
-
66 (1.2
E
-
65
)
f
6
3000
2
.
9
+
E
2 (1
.
1
E
-
2
)
8.0
E
-
02
(5.6
E
-
01
)
3.2
E
-
01 (1.1
E
+
00
)
20000
2
.
9
+
E
2 (3
.
4
E
-
2
)
8.0
E
-
02
(5.6
E
-
01
)
3.2
E
-
01 (1.1
E
+
00
)
f
7
3000
2.1
E
-
5
(2
.
2
E
-
10
)
6.4
E
-
04 (2.5
E
-
04
)
6.8
E
-
04 (2.5
E
-
04
)
f
8
1000
0.0
E
+
00
(0.0
E
+
00
)
1.0
E
-
04 (6.0
E
-
05
)
1.4
E
-
04 (6.5
E
-
05
)
5000
0.0
E
+
00
(0.0
E
+
00
)
0.0
E
+
00
(0.0
E
+
00
)
0.0
E
+
00
(0.0
E
+
00
)
f
10
500
8
.
9
E
-
16
(
0.0
E
+
00
)
8.2
E
-
10 (6.9
E
-
10
)
3.0
E
-
09 (2.2
E
-
09
)
2000
8
.
9
E
-
16
(0.0
E
+
00
)
4.4
E
-
15 (0.0
E
+
00
)
4.4
E
-
15 (0.0
E
+
00
)
f
11
500
0.0
E
+
00
(0.0
E
+
00
)
9.9
E
-
08 (6.0
E
-
07
)
2.0
E
-
04 (1.4
E
-
03
)
3000
0.0
E
+
00
(0.0
E
+
00
)
0.0
E
+
00
(0.0
E
+
00
)
2.0
E
-
04 (1.4
E
-
03
)
f
12
500
1.8
E
-
2 (1
E
-
4
)
4.6
E
-
17
(1.9
E
-
16
)
3.8
E
-
16 (8.3
E
-
16
)
1500
1.7
E
-
2 (4
E
-
4
)
1.6
E
-
32
(5.5
E
-
48
)
1.6
E
-
32
(5.5
E
-
48
)
f
13
500
4
.
4
E
-
1 (2
.
1
E
-
2
)
2.0
E
-
16
(6.5
E
-
16
)
1.2
E
-
15 (2.8
E
-
15
)
1500
3.1
E
-
1 (1
.
6
E
-
2
)
1.4
E
-
32
(1.1
E
-
47
)
1.4
E
-
32
(1.1
E
-
47
)


## 5. Conclusions

In this paper, a novel algorithm HPSO-DE is proposed by developing a balanced parameter between PSO and DE. The population is generated either by DE or by PSO according to the balanced parameter. Adaptive mutation is carried out to current population when the population clusters around local optima. The strategy makes HPSO-DE have the advantages of two algorithms and maintain diversity of the population. In comparison with the PSO, DE, and their variants, the proposed algorithm is more effective in obtaining better quality solutions, works in a more effective way, and finds better quality solutions more frequently.



---

*Source: 215472-2014-02-09.xml*