# Double Flight-Modes Particle Swarm Optimization

Authors: Wang Yong; Li Jing-yang; Li Chun-lei
Journal: Journal of Optimization
 (2013)

---

## Abstract

Getting inspiration from the real birds in flight, we propose a new particle swarm optimization algorithm that we call the double flight modes particle swarm optimization (DMPSO) in this paper. In the DMPSO, each bird (particle) can use both rotational flight mode and nonrotational flight mode to fly, while it is searching for food in its search space. There is a King in the swarm of birds, and the King controls each bird’s flight behavior in accordance with certain rules all the time. Experiments were conducted on benchmark functions such as Schwefel, Rastrigin, Ackley, Step, Griewank, and Sphere. The experimental results show that the DMPSO not only has marked advantage of global convergence property but also can effectively avoid the premature convergence problem and has good performance in solving the complex and high-dimensional optimization problems.


---

## Body

## 1. Introduction

Particle swarm optimization (PSO) was developed by Kennedy and Eberhart in 1995 [1], based on the swarm behavior of birds in searching for food. Since then, PSO has got more and more attention from the researchers in the domain of information and has generated much wider interests, because of its simplicity of implementation, and less domain knowledge required. However, the original PSO still has the phenomenon of the premature convergence problem, which exists in most of the stochastic optimization algorithms. In order to improve the performance of the PSO, many scholars have proposed various approaches to improve the performance of the PSO such as listed in the paper [2–22]. The methods presented by the authors mentioned in the paper [2–22] can be summed up into two strategies. The first strategy is to add the group quantity of information through increasing the population size of swarm, in order to achieve the purpose of improving the performance of algorithm. However, this strategy cannot fundamentally overcome the premature convergence problem and will certainly lead to the increase in running time of computation. The second strategy is, under the condition of not increasing the population size of swarm, to excavate or to increase every particle’s latent capacity to achieve the goal of improving the performance of algorithm. Although these approaches mentioned in the paper [2–22] can improve the performance of the PSO to some extent but cannot fundamentally solve the premature convergence problem which exists in the original PSO.In this paper, we intend to present a new particle swarm optimization, namely, the double flight modes particle swarm optimization (DMPSO for short), based on the flight characteristics of birds. The rest of this paper is organized as follows. In Section2, we briefly introduce the original PSO, the PSO-W, and the CLPSO. Section 3 describes the double flight-modes particle swarm optimization. In Section 4, we conduct our simulation experiments on some test functions for each algorithm and compare the performance of the DMPSO with that of the original PSO, with that of the PSO-W, and with that of the CLPSO. We give our conclusions in Section 5.


## 2. Particle Swarm Optimizers

### 2.1. The Original PSO

Particle swarm optimization (PSO) was developed by Kennedy and Eberhart [1]. PSO emulates the swarm behavior and the individuals are treated as points in the D-dimensional search space. Each individual is named as a “particle” which represents a potential solution to a problem. The position and the velocity of the ith particle are represented as Xi(t)=(xi1(t),…,xiD(t)), and Vi(t)=(vi1(t),…,viD(t)) respectively. The best previous position (the position yielding the best fitness value) of the ith particle is represented as Pbesti(t)=(pi1(t),…,piD(t)). The best position discovered by the whole population is represented as Gbest(t)=(gb1(t),…,gbD(t)). Then the vector Vi(t+1) and the position Xi(t+1) of the ith particle are updated according to the following equation [1]:
(1)vij(t+1)=vij(t)+c1r1(pij(t)-xij(t))+c2r2(gj(t)-xij(t)),j=1,…,Dxij(t+1)=xij(t)+vij(t+1),j=1,…,D,
where c1 and c2 are the acceleration coefficients reflecting the weighting of stochastic acceleration terms that pull each particle toward Gbest and Pbest positions, respectively, and r1 and r2 are two random numbers in the range [0,1].


### 2.2. Some Variants PSO

Since PSO was introduced by Kennedy and Eberhart [1], many researchers have worked on improving its performance in various ways and deriving many interesting variants. One of the variant PSOs [2] introduces a parameter called inertia weight w into the original PSO as follows:
(2)vij(t+1)=wvij(t)+c1r1(pij(t)-xij(t))+c2r2(gj(t)-xij(t)),(3)xij(t+1)=xij(t)+vij(t+1),j=1,…,D,in which the inertia weightw plays the role of balancing the global and local search. A large inertia weight facilitates a global search, while a small inertia weight facilitates a local search. In (2), if its inertia weight w is a linearly decreasing weight over the course of search, then the variant PSO [2] is usually represented as PSO-W.Another variant PSO [16], called the comprehensive learning particle swarm optimizer (CLPSO), presents a new learning strategy. In the CLPSO, the velocity updating equation is changed to
(4)Vid⟵w*Vid+c*randid*(pbestfi(d)d-Xid),in whichfi=[fi(1),…,fi(M)] defines which particles’ pbests the ith particle should follow. pbestfi(d)d can be the corresponding dimension of any particle’s pbest (including its own pbest), and the decision depends on probability Pc, referred to as the learning probability, which can take different values for different particles. We first generate a random number for each dimension of the ith particle. If this random number is larger than Pci, then the corresponding dimension will learn from its own pbest; otherwise, it will learn from another particle’s pbest.


## 2.1. The Original PSO

Particle swarm optimization (PSO) was developed by Kennedy and Eberhart [1]. PSO emulates the swarm behavior and the individuals are treated as points in the D-dimensional search space. Each individual is named as a “particle” which represents a potential solution to a problem. The position and the velocity of the ith particle are represented as Xi(t)=(xi1(t),…,xiD(t)), and Vi(t)=(vi1(t),…,viD(t)) respectively. The best previous position (the position yielding the best fitness value) of the ith particle is represented as Pbesti(t)=(pi1(t),…,piD(t)). The best position discovered by the whole population is represented as Gbest(t)=(gb1(t),…,gbD(t)). Then the vector Vi(t+1) and the position Xi(t+1) of the ith particle are updated according to the following equation [1]:
(1)vij(t+1)=vij(t)+c1r1(pij(t)-xij(t))+c2r2(gj(t)-xij(t)),j=1,…,Dxij(t+1)=xij(t)+vij(t+1),j=1,…,D,
where c1 and c2 are the acceleration coefficients reflecting the weighting of stochastic acceleration terms that pull each particle toward Gbest and Pbest positions, respectively, and r1 and r2 are two random numbers in the range [0,1].


## 2.2. Some Variants PSO

Since PSO was introduced by Kennedy and Eberhart [1], many researchers have worked on improving its performance in various ways and deriving many interesting variants. One of the variant PSOs [2] introduces a parameter called inertia weight w into the original PSO as follows:
(2)vij(t+1)=wvij(t)+c1r1(pij(t)-xij(t))+c2r2(gj(t)-xij(t)),(3)xij(t+1)=xij(t)+vij(t+1),j=1,…,D,in which the inertia weightw plays the role of balancing the global and local search. A large inertia weight facilitates a global search, while a small inertia weight facilitates a local search. In (2), if its inertia weight w is a linearly decreasing weight over the course of search, then the variant PSO [2] is usually represented as PSO-W.Another variant PSO [16], called the comprehensive learning particle swarm optimizer (CLPSO), presents a new learning strategy. In the CLPSO, the velocity updating equation is changed to
(4)Vid⟵w*Vid+c*randid*(pbestfi(d)d-Xid),in whichfi=[fi(1),…,fi(M)] defines which particles’ pbests the ith particle should follow. pbestfi(d)d can be the corresponding dimension of any particle’s pbest (including its own pbest), and the decision depends on probability Pc, referred to as the learning probability, which can take different values for different particles. We first generate a random number for each dimension of the ith particle. If this random number is larger than Pci, then the corresponding dimension will learn from its own pbest; otherwise, it will learn from another particle’s pbest.


## 3. The Double Flight Modes Particle Swarm Optimization

### 3.1. The Flight Characteristics of Birds

Through careful observation, we have found that (1) most of birds have superb flight-skills. They can use various flight modes, such as rotational flight mode and non-rotational flight mode, to fly in their search space, can avoid the attack of their natural enemies and various obstacles and can avoid themselves being immersed into blind alley and (2) there is usually a King of birds (a flight commander) in most of the swarms of birds; the King controls or directs every bird’s flight mode and flying direction while the swarms of birds are searching for food in the search space. Therefore, we think if a bird uses only one flight mode to fly in its search space all the time, it will be unable to avoid the attack of its natural enemies and various obstacles and will easily be immersed into blind alley. If there is not a King of birds controlling the flying direction of the swarm, then the swarm will be fallen into being scattered and disunited. In most cases, if a bird has superb flight skills, it usually can find more food when it is foraging for food in its search space.For the sake of simplicity, we use the following idealized rules.(1)
Each bird only uses rotational flight mode and non-rotational flight mode to fly while it is searching for food in its search space.(2)
There is a King of birds among a swarm of birds. The King controls or directs every bird’s flight behavior in accordance with certain rules and directs each bird’s flight mode and flying direction while a swarm of birds is searching for food in its search space.(3)
The flight speed of a bird has something to do with the distance between the bird and its flying destination. We can say to a certain degree that the farther the distance between the bird and its flying destination, the faster the speed flying to the destination.If we idealize the flight characteristics of a swarm of birds according to the previous description, then we can redevelop a new particle swarm optimization inspired by the real birds in flight. In simulations, we use virtual birds (particles) naturally.


### 3.2. The Flight Modes of Birds

LetXi(t)=(xi1(t),…,xiD(t)) and Vi(t)=(vi1(t),…,viD(t)) be the position and the velocity of particle i, respectively, Pbesti(t)=(pi1(t),…,piD(t)) be the best previous position yielding the best fitness value for the ith particle, and Gbest=(gb1(t),…,gbD(t)) be the best position discovered by the whole population.We first give the conceptions of rotational flight mode and non-rotational flight mode, respectively.Definition 1.
LetXi(t)=(xi1(t),…,xiD(t)) be the position of particle i at the time instant t and Gbest(t)=(g1(t),…,gD(t)) be the best position discovered by the whole population.(1) We call that particlei uses rotational flight mode to fly to the position Gbest(t), if particle i flies to the position Gbest(t) according to the following equation:
(5)xij(t+1)=gk(t),j=1,…,D,
where the number k is a random integer in the set {1,…,D}.We can use a diagrammatic sketch to depict that a group of birds is usingrotational flight mode to fly to the Gbest as in Figure 1.Figure 1
A diagrammatic sketch of a swarm of birds usingrotational flight mode to fly to the Gbest.We can foresee if a group of birds is usingrotational flight mode to fly to the position Gbest(t) at the time instant t, then to some extent, the group will gather around the position Gbest(t) at the time instant t+1.(2) We call that particlei uses non-rotational flight mode to fly to the position Gbest(t), if particle i flies to the position Gbest(t) according to the following equation:
(6)vij(t+1)=vij(t)+c1r1(pij(t)-xij(t))+c2r2(gj(t)-xij(t));xij(t+1)=xij(t)+w(t)vij(t+1),j=1,…,D,
where w(t) is an increasing function about the variable |gj(t)-xij(t)| (the distance between the jth component of Xi(t) and the jth component of Gbest(t)), c1 and c2 are the acceleration coefficients, and both r1 and r2 are two uniformly distributed random numbers in the range [0,1].In our simulations of this paper, we selectw(t)=μ(1-exp(-ρ|gj(t)-xij(t)|)) as the increasing function in (6), where μ=0.8 and ρ=5, j=1,…,D.We can use a diagrammatic sketch to depict that particlei is using non-rotational flight mode to fly to the position Gbest as in Figure 2.Figure 2
A diagrammatic sketch of particlei using non-rotational flight mode to fly to the Gbest.


### 3.3. The Flight-Control Approach of the King

Since the King of birds controls each bird’s flight behavior in accordance with certain rules; therefore, we look at the King as a flight commander, and we think that every bird’s flight behavior is controlled by the King. Following that, we will set up a flight-command rule for the King, and the King uses this rule to control each bird’s flight behavior. We first give the conception offlight command as follows.Definition 2.
LetM be the population size of the swarm and si(t) be the order of particle i in the swarm according to the ascending sort of the fitness value at the time instant t. Then the conception of flight command is defined as the following equation:
(7)FCi(t)=si(t)-1M-1,
in which, if si(t)=1 (FCi(t)=0 accordingly), then the fitness value of particle i will be the best one in the swarm at the time instant t. Meanwhile, if si(t)=M (FCi(t)=1 accordingly), then the fitness value of particle i will be the worst one in the swarm at the time instant t.The King controls each particle’s flight mode according to the following approach.Step 1.
The King first gives an instructionδ randomly, where δ is a normal random distribution in the range [0,1].Step 2.
Each particle chooses its flight mode according the followingrule:
(8)particleichoosestheformula(6)tofly,ifFCi>δ,particleichoosestheformula(5)tofly,ifFCi≤δ.
That is to say, if si(t)>δ(M-1)+1, then particle i will choose non-rotational flight mode to fly to next step. Otherwise, particle i will choose rotational flight mode to fly to next step.The procedure of the DMPSO can be simply described as in Procedure1.Procedure 1:Double flight-modes particle swarm optimization.
Objective functionf(X), X=(x1,…,xD)
Initialize each particle’s positionXi and velocity Vi randomly and assign Xi to
thePbesti  at the same time (i=1,…,M)
while (The stop criterion is not satisfied) do {
For  i=1:M, do
{
if (i≤M)
calculate the fitness value of particle i  f(Xi(t))
end if
}
Rank the swarm according to the ascending sort of the fitness value and get
s
i
(
t
)
,
F
C
i
(
t
)
,
G
best
(
t
)
Assign each particle’s flight-mode according to the Rule (8)
Update Pbest(t)
}
t
=
t
+
1
end while
output Gbest,f(Gbest)


## 3.1. The Flight Characteristics of Birds

Through careful observation, we have found that (1) most of birds have superb flight-skills. They can use various flight modes, such as rotational flight mode and non-rotational flight mode, to fly in their search space, can avoid the attack of their natural enemies and various obstacles and can avoid themselves being immersed into blind alley and (2) there is usually a King of birds (a flight commander) in most of the swarms of birds; the King controls or directs every bird’s flight mode and flying direction while the swarms of birds are searching for food in the search space. Therefore, we think if a bird uses only one flight mode to fly in its search space all the time, it will be unable to avoid the attack of its natural enemies and various obstacles and will easily be immersed into blind alley. If there is not a King of birds controlling the flying direction of the swarm, then the swarm will be fallen into being scattered and disunited. In most cases, if a bird has superb flight skills, it usually can find more food when it is foraging for food in its search space.For the sake of simplicity, we use the following idealized rules.(1)
Each bird only uses rotational flight mode and non-rotational flight mode to fly while it is searching for food in its search space.(2)
There is a King of birds among a swarm of birds. The King controls or directs every bird’s flight behavior in accordance with certain rules and directs each bird’s flight mode and flying direction while a swarm of birds is searching for food in its search space.(3)
The flight speed of a bird has something to do with the distance between the bird and its flying destination. We can say to a certain degree that the farther the distance between the bird and its flying destination, the faster the speed flying to the destination.If we idealize the flight characteristics of a swarm of birds according to the previous description, then we can redevelop a new particle swarm optimization inspired by the real birds in flight. In simulations, we use virtual birds (particles) naturally.


## 3.2. The Flight Modes of Birds

LetXi(t)=(xi1(t),…,xiD(t)) and Vi(t)=(vi1(t),…,viD(t)) be the position and the velocity of particle i, respectively, Pbesti(t)=(pi1(t),…,piD(t)) be the best previous position yielding the best fitness value for the ith particle, and Gbest=(gb1(t),…,gbD(t)) be the best position discovered by the whole population.We first give the conceptions of rotational flight mode and non-rotational flight mode, respectively.Definition 1.
LetXi(t)=(xi1(t),…,xiD(t)) be the position of particle i at the time instant t and Gbest(t)=(g1(t),…,gD(t)) be the best position discovered by the whole population.(1) We call that particlei uses rotational flight mode to fly to the position Gbest(t), if particle i flies to the position Gbest(t) according to the following equation:
(5)xij(t+1)=gk(t),j=1,…,D,
where the number k is a random integer in the set {1,…,D}.We can use a diagrammatic sketch to depict that a group of birds is usingrotational flight mode to fly to the Gbest as in Figure 1.Figure 1
A diagrammatic sketch of a swarm of birds usingrotational flight mode to fly to the Gbest.We can foresee if a group of birds is usingrotational flight mode to fly to the position Gbest(t) at the time instant t, then to some extent, the group will gather around the position Gbest(t) at the time instant t+1.(2) We call that particlei uses non-rotational flight mode to fly to the position Gbest(t), if particle i flies to the position Gbest(t) according to the following equation:
(6)vij(t+1)=vij(t)+c1r1(pij(t)-xij(t))+c2r2(gj(t)-xij(t));xij(t+1)=xij(t)+w(t)vij(t+1),j=1,…,D,
where w(t) is an increasing function about the variable |gj(t)-xij(t)| (the distance between the jth component of Xi(t) and the jth component of Gbest(t)), c1 and c2 are the acceleration coefficients, and both r1 and r2 are two uniformly distributed random numbers in the range [0,1].In our simulations of this paper, we selectw(t)=μ(1-exp(-ρ|gj(t)-xij(t)|)) as the increasing function in (6), where μ=0.8 and ρ=5, j=1,…,D.We can use a diagrammatic sketch to depict that particlei is using non-rotational flight mode to fly to the position Gbest as in Figure 2.Figure 2
A diagrammatic sketch of particlei using non-rotational flight mode to fly to the Gbest.


## 3.3. The Flight-Control Approach of the King

Since the King of birds controls each bird’s flight behavior in accordance with certain rules; therefore, we look at the King as a flight commander, and we think that every bird’s flight behavior is controlled by the King. Following that, we will set up a flight-command rule for the King, and the King uses this rule to control each bird’s flight behavior. We first give the conception offlight command as follows.Definition 2.
LetM be the population size of the swarm and si(t) be the order of particle i in the swarm according to the ascending sort of the fitness value at the time instant t. Then the conception of flight command is defined as the following equation:
(7)FCi(t)=si(t)-1M-1,
in which, if si(t)=1 (FCi(t)=0 accordingly), then the fitness value of particle i will be the best one in the swarm at the time instant t. Meanwhile, if si(t)=M (FCi(t)=1 accordingly), then the fitness value of particle i will be the worst one in the swarm at the time instant t.The King controls each particle’s flight mode according to the following approach.Step 1.
The King first gives an instructionδ randomly, where δ is a normal random distribution in the range [0,1].Step 2.
Each particle chooses its flight mode according the followingrule:
(8)particleichoosestheformula(6)tofly,ifFCi>δ,particleichoosestheformula(5)tofly,ifFCi≤δ.
That is to say, if si(t)>δ(M-1)+1, then particle i will choose non-rotational flight mode to fly to next step. Otherwise, particle i will choose rotational flight mode to fly to next step.The procedure of the DMPSO can be simply described as in Procedure1.Procedure 1:Double flight-modes particle swarm optimization.
Objective functionf(X), X=(x1,…,xD)
Initialize each particle’s positionXi and velocity Vi randomly and assign Xi to
thePbesti  at the same time (i=1,…,M)
while (The stop criterion is not satisfied) do {
For  i=1:M, do
{
if (i≤M)
calculate the fitness value of particle i  f(Xi(t))
end if
}
Rank the swarm according to the ascending sort of the fitness value and get
s
i
(
t
)
,
F
C
i
(
t
)
,
G
best
(
t
)
Assign each particle’s flight-mode according to the Rule (8)
Update Pbest(t)
}
t
=
t
+
1
end while
output Gbest,f(Gbest)


## 4. Validation and Comparison

In order to test the performance of the DMPSO, we have tested it against the original PSO [1], the PSO-W [2], and the CLPSO [16]. For the ease of visualization, we have implemented our simulations using Matlab for various test functions.


### 4.1. Benchmark Functions

For the sake of having a fair and reasonable comparison between the DMPSO and the PSO, the DMPSO and the PSO-W, or the DMPSO and the CLPSO, we have chosen six well-known high-dimensional functions as our optimization simulation tests. All functions are tested on 50 dimensions. The properties and the formula of these functions are presented as follows.(a) Schwefel’s function:(9)f1(X)=∑i=1D|xi|+∏i=1D|xi|,-10≤xi≤10,D=50.
It is a unimodal function and has a global minimum fmin=0 at (0,…,0). The complexity of Schwefel’s function is due to its deep local optima being far from the global optimum. It will be hard to find the global optimum if many particles fall into one of the deep local optima.(b) Rastrigin’s function:(10)f2(X)=10D∑i=1D[xi2-10cos(2πxi)],2l-5.12≤xi≤5.12,D=50.
The function is a complex multimodal function, and the number of its local minima shows an exponential increase with the problem dimension and its peak shape is up and down in jumping. When attempting to solve Rastrigin’s function, algorithms may easily fall into a local optimum. Therefore, an algorithm capable of maintaining a larger diversity is likely to yield better results, so Rastrigin is viewed as a typical function used for testing global search performance of algorithm. It has a global minimum fmin=0 at (0,…,0).(c) Step function:(11)f3(X)=∑i=1D(⌊xi+0.5⌋)2,-100≤xi≤100,D=50,
in which y=⌊x⌋ is a bracket function (Gaussian function) and x-1<⌊x⌋≤x<⌊x⌋+1,  ∀x∈R. Step function is a discontinuous function and has a global minimum fmin=0 in the domain {X∈R50∣-0.5≤xi≤0.5}.(d) Ackley’s function:(12)f4(X)=-20exp[-151D∑i=1Dxi2]-exp[1D∑i=1Dcos(2πxi)]+20+e,-30≤xi≤30,D=50.
The function has one narrow global optimum basin and many minor local optima and has a global optimum fmin=0 at (0,…,0).(e) Sphere function:(13)f5(X)=∑i=1Dxi2,-5.12≤xi≤5.12,D=50.
It is a unimodal function, and its global minimum fmin=0 at (0,…,0).(f) Griewank’s function:(14)f6(X)=14000∑i=1Dxi2-∏i=1Dcos(xii)+1,000-600≤xi≤600,D=50.
The search space is relatively large in this optimization problem; Griewank’s function has a ∏i=1Dcos(xi/i) component causing linkages among variables, thereby making it difficult to reach the global optimum. Therefore, it is generally regarded as a complex multimodal problem that is hard to optimize. The function has a global minimum fmin=0 at (0,…,0).


### 4.2. Comparison of Experimental Results and Discussions

There are many means to carry out the comparison of algorithm performance. We can use such ways to compare the number of function evaluations (FEs) for a given accuracy or to compare their accuracies for a fixed number of function evaluations, and so forth. In our simulations, we use the two ways mentioned previously, and we set up a running-stopping condition for each algorithm. If a run has found the global optimal solution of optimization problem or has reached a fixed number of function evaluations, then it will stop running. We run each algorithm for 50 times so that we can do reasonable and meaningful analysis.In order to ensure the comparability of the experimental results, the parameters settings are set as consistent as possible for the DMPSO, the PSO, the PSO-W, and the CLPSO. The parameters settings are listed as follows: the population sizeM=30 and the acceleration coefficients c1=c2=2 for all simulations. As for the test functions f1(X), f2(X), and f3(X), we set the maximum iterations at 500 (or the maximum number of function evaluations at 15000 correspondingly). As for the test functions f4(X), f5(X), and f6(X), we set the maximum iterations at 2000 (or the maximum number of function evaluations at 60000 correspondingly). On the other hand, we set the inertia weight w=0.628 for the CLPSO, and set w(t)=0.8(1-exp(-5|gj(t)-xij(t)|)) for the DMPSO. As for the PSO-W, the linearly decreasing inertia weight w is used which starts at 0.9 and ends at 0.4, and vmax≤20%(ud-ld).In our experiments, we choose the best fitness value (BFV), the worst fitness value (WFV), the mean value (Mean), and the number of function evaluations in the form of mean (MFEs) ± standard deviation (STDEV) (success rate of algorithm in finding the global optima) as the evaluation indicators of optimization ability for the four algorithms mentioned previously. These evaluation indicators cannot only reflect the optimization ability but also indicate the computing cost. We have got the experimental results being listed in Table1.Table 1
Experimental results.
f
Algorithms
BFV
WFV
Mean
MFEs± STDEV
f
1
DMPSO
0
1.125465
e
-
009
6.434905
e
-
011
7350
±
1.771972
e
-
010 (56%)
PSO
9.736494
61.567035
31.111268
15000
±
11.065242 (0%)
CLPSO
8.174606
e
-
005
0.001325
5.125738
e
-
021
15000
±
2.702365
e
-
004 (0%)
PSO-w
4.429573
1.994109
e
+
002
80.584602
15000
±
58.997786 (0%)
f
2
DMPSO
0
49.747953
3.979836
11130
±
13.496281 (64%)
PSO
1.790843
e
+
002
4.039120
e
+
002
2.847614
e
+
002
15000
±
49.430124 (0%)
CLPSO
21.153304
47.516825
31.762105
15000
±
6.732226 (0%)
PSO-w
1.416269
e
+
002
2.890130
e
+
002
2.1243064
e
+
002
15000
±
43.169379 (0%)
f
3
DMPSO
0
0
0
1470
±
0 (100%)
PSO
351
11160
3.083180
e
+
003
15000
±
3.937885
e
+
003 (0%)
CLPSO
0
122
4.22
10440
±
17.105894 (48%)
PSO-w
45
311
1.405400
e
+
002
15000
±
54.365809 (0%)
f
4
DMPSO
8.881784
e
-
016
3.286260
e
-
014
1.140421
e
-
014
60000
±
8.672115
e
-
015 (0%)
PSO
0.079333
14.476725
3.692153
60000
±
4.976905 (0%)
CLPSO
1.289395
e
-
010
2.200675
1.082862
60000
±
0.692050 (0%)
PSO-w
0.003600
11.916679
0.656150
60000
±
1.737976 (0%)
f
5
DMPSO
0
1.729170
e
-
069
3.458545
e
-
071
11130
±
2.420835
e
-
070 (28%)
PSO
1.236611
e
-
004
26.217801
4.196887
60000
±
9.707225 (0%)
CLPSO
2.700980
e
-
038
9.595381
e
-
036
9.748685
e
-
037
60000
±
1.924460
e
-
036 (0%)
PSO-w
5.651214
e
-
021
1.681261
e
-
017
2.451834
e
-
018
60000
±
3.861006
e
-
018 (0%)
f
6
DMPSO
0
3.330669
e
-
016
3.774758
e
-
017
35430
±
9.049510
e
-
017 (82%)
PSO
0.04331
1.810140
e
+
002
20.290841
60000
±
41.921578 (0%)
CLPSO
1.787459
e
-
014
0.092264
0.014715
60000
±
0.021761 (0%)
PSO-w
1.760398
e
-
004
90.751966
1.826534
60000
±
12.832618 (0%)In order to more easily contrast the convergence characteristics of the four algorithms, Figure3 presents the convergence characteristics in term of the best fitness value of the mean run of each algorithm for each test function.The median convergence characteristics of 50D test functions: (a) Schwefel’s function, (b) Rastrigin’s function, (c) Step function, (d) Ackley’s function, (e) Sphere function, and (f) Griewank’s function.
(a)
(b)
(c)
(d)
(e)
(f)Discussions. From the results listed in Table 1 and the convergence curve simulation diagram in Figure 3, we can see that (1) the DMPSO performs much better than the original PSO, the CLPSO, and the PSO-W and (2) the DMPSO is much superior to the original PSO, the CLPSO, and the PSO-W in terms of global convergence property, accuracy, and efficiency. So, we conclude that the performance of the DMPSO is much better than that of the original PSO, that of the CLPSO, and that of the PSO-W.


## 4.1. Benchmark Functions

For the sake of having a fair and reasonable comparison between the DMPSO and the PSO, the DMPSO and the PSO-W, or the DMPSO and the CLPSO, we have chosen six well-known high-dimensional functions as our optimization simulation tests. All functions are tested on 50 dimensions. The properties and the formula of these functions are presented as follows.(a) Schwefel’s function:(9)f1(X)=∑i=1D|xi|+∏i=1D|xi|,-10≤xi≤10,D=50.
It is a unimodal function and has a global minimum fmin=0 at (0,…,0). The complexity of Schwefel’s function is due to its deep local optima being far from the global optimum. It will be hard to find the global optimum if many particles fall into one of the deep local optima.(b) Rastrigin’s function:(10)f2(X)=10D∑i=1D[xi2-10cos(2πxi)],2l-5.12≤xi≤5.12,D=50.
The function is a complex multimodal function, and the number of its local minima shows an exponential increase with the problem dimension and its peak shape is up and down in jumping. When attempting to solve Rastrigin’s function, algorithms may easily fall into a local optimum. Therefore, an algorithm capable of maintaining a larger diversity is likely to yield better results, so Rastrigin is viewed as a typical function used for testing global search performance of algorithm. It has a global minimum fmin=0 at (0,…,0).(c) Step function:(11)f3(X)=∑i=1D(⌊xi+0.5⌋)2,-100≤xi≤100,D=50,
in which y=⌊x⌋ is a bracket function (Gaussian function) and x-1<⌊x⌋≤x<⌊x⌋+1,  ∀x∈R. Step function is a discontinuous function and has a global minimum fmin=0 in the domain {X∈R50∣-0.5≤xi≤0.5}.(d) Ackley’s function:(12)f4(X)=-20exp[-151D∑i=1Dxi2]-exp[1D∑i=1Dcos(2πxi)]+20+e,-30≤xi≤30,D=50.
The function has one narrow global optimum basin and many minor local optima and has a global optimum fmin=0 at (0,…,0).(e) Sphere function:(13)f5(X)=∑i=1Dxi2,-5.12≤xi≤5.12,D=50.
It is a unimodal function, and its global minimum fmin=0 at (0,…,0).(f) Griewank’s function:(14)f6(X)=14000∑i=1Dxi2-∏i=1Dcos(xii)+1,000-600≤xi≤600,D=50.
The search space is relatively large in this optimization problem; Griewank’s function has a ∏i=1Dcos(xi/i) component causing linkages among variables, thereby making it difficult to reach the global optimum. Therefore, it is generally regarded as a complex multimodal problem that is hard to optimize. The function has a global minimum fmin=0 at (0,…,0).


## 4.2. Comparison of Experimental Results and Discussions

There are many means to carry out the comparison of algorithm performance. We can use such ways to compare the number of function evaluations (FEs) for a given accuracy or to compare their accuracies for a fixed number of function evaluations, and so forth. In our simulations, we use the two ways mentioned previously, and we set up a running-stopping condition for each algorithm. If a run has found the global optimal solution of optimization problem or has reached a fixed number of function evaluations, then it will stop running. We run each algorithm for 50 times so that we can do reasonable and meaningful analysis.In order to ensure the comparability of the experimental results, the parameters settings are set as consistent as possible for the DMPSO, the PSO, the PSO-W, and the CLPSO. The parameters settings are listed as follows: the population sizeM=30 and the acceleration coefficients c1=c2=2 for all simulations. As for the test functions f1(X), f2(X), and f3(X), we set the maximum iterations at 500 (or the maximum number of function evaluations at 15000 correspondingly). As for the test functions f4(X), f5(X), and f6(X), we set the maximum iterations at 2000 (or the maximum number of function evaluations at 60000 correspondingly). On the other hand, we set the inertia weight w=0.628 for the CLPSO, and set w(t)=0.8(1-exp(-5|gj(t)-xij(t)|)) for the DMPSO. As for the PSO-W, the linearly decreasing inertia weight w is used which starts at 0.9 and ends at 0.4, and vmax≤20%(ud-ld).In our experiments, we choose the best fitness value (BFV), the worst fitness value (WFV), the mean value (Mean), and the number of function evaluations in the form of mean (MFEs) ± standard deviation (STDEV) (success rate of algorithm in finding the global optima) as the evaluation indicators of optimization ability for the four algorithms mentioned previously. These evaluation indicators cannot only reflect the optimization ability but also indicate the computing cost. We have got the experimental results being listed in Table1.Table 1
Experimental results.
f
Algorithms
BFV
WFV
Mean
MFEs± STDEV
f
1
DMPSO
0
1.125465
e
-
009
6.434905
e
-
011
7350
±
1.771972
e
-
010 (56%)
PSO
9.736494
61.567035
31.111268
15000
±
11.065242 (0%)
CLPSO
8.174606
e
-
005
0.001325
5.125738
e
-
021
15000
±
2.702365
e
-
004 (0%)
PSO-w
4.429573
1.994109
e
+
002
80.584602
15000
±
58.997786 (0%)
f
2
DMPSO
0
49.747953
3.979836
11130
±
13.496281 (64%)
PSO
1.790843
e
+
002
4.039120
e
+
002
2.847614
e
+
002
15000
±
49.430124 (0%)
CLPSO
21.153304
47.516825
31.762105
15000
±
6.732226 (0%)
PSO-w
1.416269
e
+
002
2.890130
e
+
002
2.1243064
e
+
002
15000
±
43.169379 (0%)
f
3
DMPSO
0
0
0
1470
±
0 (100%)
PSO
351
11160
3.083180
e
+
003
15000
±
3.937885
e
+
003 (0%)
CLPSO
0
122
4.22
10440
±
17.105894 (48%)
PSO-w
45
311
1.405400
e
+
002
15000
±
54.365809 (0%)
f
4
DMPSO
8.881784
e
-
016
3.286260
e
-
014
1.140421
e
-
014
60000
±
8.672115
e
-
015 (0%)
PSO
0.079333
14.476725
3.692153
60000
±
4.976905 (0%)
CLPSO
1.289395
e
-
010
2.200675
1.082862
60000
±
0.692050 (0%)
PSO-w
0.003600
11.916679
0.656150
60000
±
1.737976 (0%)
f
5
DMPSO
0
1.729170
e
-
069
3.458545
e
-
071
11130
±
2.420835
e
-
070 (28%)
PSO
1.236611
e
-
004
26.217801
4.196887
60000
±
9.707225 (0%)
CLPSO
2.700980
e
-
038
9.595381
e
-
036
9.748685
e
-
037
60000
±
1.924460
e
-
036 (0%)
PSO-w
5.651214
e
-
021
1.681261
e
-
017
2.451834
e
-
018
60000
±
3.861006
e
-
018 (0%)
f
6
DMPSO
0
3.330669
e
-
016
3.774758
e
-
017
35430
±
9.049510
e
-
017 (82%)
PSO
0.04331
1.810140
e
+
002
20.290841
60000
±
41.921578 (0%)
CLPSO
1.787459
e
-
014
0.092264
0.014715
60000
±
0.021761 (0%)
PSO-w
1.760398
e
-
004
90.751966
1.826534
60000
±
12.832618 (0%)In order to more easily contrast the convergence characteristics of the four algorithms, Figure3 presents the convergence characteristics in term of the best fitness value of the mean run of each algorithm for each test function.The median convergence characteristics of 50D test functions: (a) Schwefel’s function, (b) Rastrigin’s function, (c) Step function, (d) Ackley’s function, (e) Sphere function, and (f) Griewank’s function.
(a)
(b)
(c)
(d)
(e)
(f)Discussions. From the results listed in Table 1 and the convergence curve simulation diagram in Figure 3, we can see that (1) the DMPSO performs much better than the original PSO, the CLPSO, and the PSO-W and (2) the DMPSO is much superior to the original PSO, the CLPSO, and the PSO-W in terms of global convergence property, accuracy, and efficiency. So, we conclude that the performance of the DMPSO is much better than that of the original PSO, that of the CLPSO, and that of the PSO-W.


## 5. Conclusions

This paper presents a novel particle swam optimization with double flight modes that we call the double flight modes particle swam optimization (DMPSO). In the optimization algorithm, each bird (particle) can use both rotational flight mode and non-rotational flight mode to fly while it is foraging for food in the search space. By using its superb flight skills, each bird (particle) has much improved its searching efficiency. From the experiments we conduct on some benchmark functions such as Schwefel, Rastrigin, Ackley, Step, Griewank, and Sphere, we can conclude that the DMPSO not only has marked advantage of global convergence property but also can effectively avoid the premature convergence problem to some extent and is one of good choices for use to solve the complex and high-dimensional optimization problems, although the DMPSO is not necessarily the best choice for solving various real-world optimization problems.



---

*Source: 356420-2013-12-16.xml*