# Parallel and Cooperative Particle Swarm Optimizer for Multimodal Problems

Authors: Geng Zhang; Yangmin Li
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

Although the original particle swarm optimizer (PSO) method and its related variant methods show some effectiveness for solving optimization problems, it may easily get trapped into local optimum especially when solving complex multimodal problems. Aiming to solve this issue, this paper puts forward a novel method called parallel and cooperative particle swarm optimizer (PCPSO). In case that the interacting of the elements inD-dimensional function vector
X=[x1,x2,…,xd,…,xD] is independent, cooperative particle swarm optimizer (CPSO) is used. Based on this, the PCPSO is presented to solve real problems. Since the dimension cannot be split into several lower dimensional search spaces in real problems because of the interacting of the elements, PCPSO exploits the cooperation of two parallel CPSO algorithms by orthogonal experimental design (OED) learning. Firstly, the CPSO algorithm is used to generate two locally optimal vectors separately; then the OED is used to learn the merits of these two vectors and creates a better combination of them to generate further search. Experimental studies on a set of test functions show that PCPSO exhibits better robustness and converges much closer to the global optimum than several other peer algorithms.


---

## Body

## 1. Introduction

Inspired from social behavior and cognitive behavior, Kennedy and Eberhart [1, 2] proposed particle swarm optimizer (PSO) algorithm to search for optimal value through population-based iterative learning algorithm. Due to the simple implementation and effective searching ability of the PSO algorithm, it has been widely used in feature selection [3], robot path planning [4], data processing [5], and other problems. However, many experiments have shown that the PSO algorithm may easily get trapped into local optimum especially when facing complex multimodal optimization problems. Better optimization algorithms are always needed for solving complex real-world engineering problems. In general, the unconstrained optimization problems that we are going to solve can be formulated as a D-dimensional minimization problem as follows:(1)Min⁡f(X),X=[x1,x2,…,xd,…,xD],where X=[x1,x2,…,xd,…,xD] is the vector to be optimized and D is the number of parameters [6].In PSO, a member in the swarm, called a particle, represents a potential solution which is a point in the search space. LetM denote the size of the swarm, the current state of each particle i is represented by its position vector Xi=[xi1,xi2,…,xid,…,xiD], and the movement of particle i is represented by velocity vector Vi=[vi1,vi2,…,vid,…,viD], where i=1,2,…,M is positive integer indexing particle in the swarm. Using t=1,2,…,T represents the iteration number; the velocity vid(t) and the position xid(t) can be updated as follows:(2)vidt+1=ωvidt+c1r1idpidt-xidt+c2r2idpndt-xidt,xid(t+1)=xid(t)+vid(t+1),where the inertia weight ω∈(0,1) determines how much the previous velocity can be preserved. A large inertia weight value tends to global exploration and a small value for local exploitation. c1 and c2 denote the acceleration constants which are usually set 2.0 or adaptively controlled [7]. r1id and r2id are random numbers generated between 0 and 1 for the dth dimension of ith particle. Pi(t)=[pi1t,pi2t,…,pidt,…,piD(t)] represents the best previous position of particle i which is defined by pbest from the previous t iterations and Pn(t)=[pn1t,pn2t,…,pndt,…,pnD(t)] represents the best position among particle i’s neighborhood which is defined by gbest or lbest from the previous t iterations. The gbest model, which is inclined to exploitation, has a faster convergence speed but has a higher probability of getting stuck into local optimum than the lbest model. On the contrary, the lbest model considered focusing more on exploration is less vulnerable to the attraction of local optima but has slower convergence speed than the gbest model [8]. The position vector Xi=[xi1,xi2,…,xid,…,xiD] and the velocity vector Vi=[vi1,vi2,…,vid,…,viD] are initialized randomly and are updated by (2) generation by generation until some criteria are met.There are many modified versions of PSO that have been reported in the literature. Most studies address the performance improvement of PSO from one of the four aspects: population topology [9–11], diversity maintenance [6, 12, 13], hybrid with auxiliary operations [14–16], and adaptive PSO [8, 17]. However, it is difficult to find a robust solution due to the complexity and large search space of high dimensional problems. Therefore, people try to split the large search space into smaller ones in order to simplify the problem. In [18], the search space implemented by genetic algorithm is divided by splitting the solution vector into several smaller vectors. Each smaller search space is optimized separately and the fitness function is evaluated by combining all the solutions found by the smaller search spaces. The same technique is used in PSO called cooperative PSO (CPSO_H), which uses several subswarms to search D-dimensional function vector X=[x1,x2,…,xd,…,xD] separately [19]. The searching results are integrated by a global swarm to improve the performance of the original PSO on high dimensional problems. Compared with traditional PSO, the CPSO_H has significant performance in terms of solution quality and robustness and performs better and better as the dimension of the problem increases. However, the efficiency of the algorithm is highly affected by the degree of interacting between function vectors X=[x1,x2,…,xd,…,xD].Inspired from previous studies, a novel algorithm called parallel and cooperative PSO (PCPSO) is proposed in this paper. PCPSO tries to overcome the influence of interacting between vector elements when taking similar split method as mentioned in [19]. Assuming that there is no interacting among the elements of the vector X=[x1,x2,…,xd,…,xD], the CPSO is used [19]. Although the application of CPSO is useless in solving real problems, it provides a framework for PCPSO. For high degree of interacting of vector elements, local optimum can be obtained by CPSO [20]. In order to jump out of local optimum, orthogonal experimental design (OED) is used to learn from two locally optimal vectors which are achieved by CPSO [21]. A better combination of these two vectors can be obtained to push the search further to get closer to global optimum.The rest of this paper is organized as follows. Section2 describes the PCPSO algorithm. In Section 3, the benchmark test functions are used to test PCPSO algorithm and their results are compared with some peer algorithms taken from literature to verify the effectiveness and robustness of PCPSO algorithm. Final conclusions are given in Section 4.


## 2. Parallel and Cooperative Particle Swarm Optimizer

### 2.1. Cooperative Particle Swarm Optimizer

Among PSO variants in the literature, premature convergence is still the main deficiency of the PSO especially when facing large dimensional high correlation problems. The difficulty for particles jump out of local optimum is that particles need to change several elements of vectorX=[x1,x2,…,xd,…,xD] simultaneously, but usually the PSO variants can only change one element at one time when facing local optimum. Although the OED method is used to combine the best elements of vector X=[x1,x2,…,xd,…,xD] as an exemplar [21]; the function value f(X) usually gets bigger because of the high correlation. Here, we first introduce the CPSO used in [19]; then the PCPSO is proposed based on CPSO. OED is used to learn and create a new vector from two locally optimal vectors which are achieved by CPSO and the new vector is treated as a start point for further search.In [19], there are two cooperative PSO algorithms, CPSO_Sk and CPSO_Hk. In CPSO_Sk, the D-dimensional search space is decomposed into k subcomponents, each corresponding to a swarm of s-dimensions (where n=k×s). CPSO_Hk is a hybrid method combining a standard PSO with the CPSO_Sk. The k subcomponents are used to place interacting elements together. However, the interacting elements are not known in real problems. We simplify the CPSO_Sk algorithm as CPSO which means the D-dimensional search space is decomposed into D subcomponents. Algorithm 1 shows the algorithm of CPSO. In order to evaluate the fitness of a particle in a swarm, the context vector is applied, which is a concatenation of all global best particles from all D swarms. The evaluation of the ith particle in the dth swarm is done by calling the function b(d,Sd·xi) which returns a D-dimensional vector with its dth component replaced by Sd·xi [22].Algorithm 1:Pseudocode for the generic CPSO algorithm.
(
1) Define  b(d,z)≡(S1·pn,  S2·pn,…,Sd-1·pn,z,Sd+1pn,…,SD·pn)
(2) Create and initialize D one-dimensional PSOs: Sd, d∈[1,…,D]
(3) Repeat:
(4) for each swarm d∈[1,…,D]:
(5) for each particle i∈[1,…,M]:
(6)  if  f(b(d,Sd·xi)) < f(b(d,Sd·pi))
(7)   Sd·pi = Sd·xi
(8)  if  f(b(d,Sd·pi)) < f(b(d,Sd·pn))
(9)   Sd·pn = Sd·pi
(10) end for
(11) Perform velocity and position updates using  (2) for each particle in Sd.
(12)  end for
(13)  Until: stopping condition is met
Thedth swarm is denoted as Sd,  Sd·xi denotes the current position of the ith particle in
thedth swarm, Sd·pn represents the best position in dth swarm. Similarly, Sd·pi
represents the best previous position of particlei in dth swarm. The context vector is the
concatenation ofS1·pn, S2·pn,…,SD·pn.


### 2.2. Parallel and Cooperative Particle Swarm Optimizer Algorithm

Since CPSO method uses one-dimensional swarm to search each dimension and interacting elements of the solution vector cannot be changed simultaneously, it is easier to fall into local optimum. If we try to use the CPSO algorithm (called First1D) to solve the high correlation problem, the vectorXL1=[xL11,xL12,…,xL1d,…,xL1D] started from random values will fall into a locally optimal vector. However, when another CPSO algorithm (called Second1D) which does not have any relationship with First1D is also used to solve the same problem, another locally optimal vector XL2=[xL21,xL22,…,xL2d,…,xL2D] will be obtained. Since the performance of CPSO depends on numerical distribution when X=[x1,x2,…,xd,…,xD] is initialized by random values, the independent vectors XL1 and XL2, containing good information for optimum search, will fall into different locally optimal vectors. It is desperately needed to find a method of extracting good information from XL1 and XL2 and form a new vector XLn which is closer to globally optimal value. If we exhaustively test all the combinations of XL1 and XL2 for the new vector XLn, there are 2D trials which are unrealistic to accomplish in practice. With the help of OED [15, 16, 23, 24], a relatively good vector XLn is obtained from XL1 and XL2, using only a few experimental tests.The OED with both the orthogonal array (OA) and the factor analysis makes it possible to discover the best combinations for different factors with only small number of experimental samples [23, 24]. Here we have a brief discussion about OED combined with an example of vectors XL1 and XL2. In order to simplify the problem, assuming D=3, the optimization problem is to minimize the Sphere function f(X)=x12+x22+x32, with XL1=[2,0,1], XL2=[3,-2,0] which are derived from First1D and Second1D (actually First1D and Second1D should obtain the same globally optimal vector because the test function is simple; here we just want to explain the OED theory). The whole analysis of using OED for Sphere function is shown in Table 1.Table 1
Best combination levels of Sphere function by using OED method.
x
1
x
2
x
3
Function value
C
1
(
X
L
1
)  2
(
X
L
1
)  0
(
X
L
1
)  1
F
1
=
5
C
2
(
X
L
1
)  2
(
X
L
2
)  −2
(
X
L
2
)  0
F
2
=
8
C
3
(
X
L
2
) 3
(
X
L
1
)  0
(
X
L
2
)  0
F
3
=
9
C
4
(
X
L
2
)  3
(
X
L
2
)  −2
(
X
L
1
)  1
F
4
=
14
Levels
Factor analysis
L
1
S
11 = (F1+F2)/2 = 6.5
S
21 = (F1+F3)/2 = 7
S
31 = (F1+F4)/2 = 9.5
L
2
S
12 = (F3+F4)/2 = 11.5
S
22 = (F2+F4)/2 = 11
S
32 = (F2+F3)/2 = 8.5
L
3
(XL1)  2
(XL1)  0
(XL2)  0
F
final
=
4OA is a predefined table which has numbers arranged in rows and columns. In Table1, we can see that there are three factors x1, x2, and x3 that affect function values and two levels XL1 and XL2 to choose. The rows represent the levels of factors in each combination, and the columns represent specific factors that can be changed from each combination [15, 24]. The programming of generating OA can be found in [24]. Factor analysis evaluates the effects of individual factors based on function values and determines the best level for each factor to form a new vector XLn. Assuming Ft denotes a function value of the combination t, where t=1,…,n stands for total number of combinations. The main effect of factor d with level l as Sdl is expressed as(3)sdl=∑t=1nFt·Cdlt∑t=1nCdlt,where d=1,…,D and l=1,2. If the factor is d and the level is l, Cdlt=1, otherwise 0. In Table 1, we first calculate the effect S11 of level 1 in factor x1; then the level 2 in factor x1 is S12. Since this is a minimal optimization problem, the better level 1 is chosen because the effect value of level 1 S11 is less than that of S12 in Table 1. As the example shown in Table 1, although this new vector XLn=[2,0,0] is not shown in the four combinations tested, the best combination can be obtained by OED method.OED works well when the interacting of vector elements is small. When the OED method is used to process some functions with high interacting vector elements, there exists a problem that the function value of new vectorXLn is bigger than the function values of vector XL1 and XL2. However, there are still two advantages by using OED in high correlated problems. One is the particle with new vector XLn jumping out of locally optimal vector; the other one is that each element of the new vector XLn chosen between XL1 and XL2 is probably closer to the globally optimal vector. Therefore, we try to use the new vector XLn as context vector to repeat the CPSO algorithm.Based on the analysis mentioned above, we propose the PCPSO algorithm to overcome the problem of falling into local optimum. Figure1 shows the flowchart of PCPSO. First we use two parallel CPSO algorithms (First1D and Second1D); then OED learning is applied to form the new vector XLn which is assigned to XL2; the better vector chosen from XL1 and XL2 is assigned to the new XL1. The new vectors XL1 and XL2, which are treated as context vectors in CPSO algorithm, keep the search going further. In order to illustrate the efficiency of PCPSO algorithm, Figure 2 shows the convergence characteristic of the PCPSO on Rotated Ackley’s function. The detailed description of Rotated Ackley’s function can be found in Section 3. The parameter settings for First1D and Second1D are the same as CPSO_H algorithm [19], where ω decreases linearly from 0.9 to 0.4, c1=c2=1.494, M=10 stands for particle number, and D=30 represents dimensions. Assuming First1D and Second1D need 80 CPSO iterations to accomplish locally optimal vectors followed with OED computation, there are 6 iterations of OED computation and therefore the total CPSO iteration is 6×80=480. From Figure 2, we can see that the function value drops sharply at first 80 iterations and that First1D and Second1D have similar performance. For the next 80 iterations, the better vector chosen from XL1 and XL2 is assigned to XL1 for another First1D implementation; the new vector XLn is assigned to XL2 for another Second1D implementation. The performance on First1D and Second1D are significantly different in iterations from 80 to 240. First1D algorithm gets trapped in locally optimal vector and Second1D algorithm still makes the function value drop sharply even though it is higher at start point at the 81st iteration. Since First1D algorithm has almost lost searching ability after 80 iterations, we can also adopt the strategy that Firt1D algorithm is only implemented at first 80 iterations in order to save computational cost. Vector XL1 is used for saving the best optimal vector ever found and is combined with vector XL2 for new vector XLn by OED method after 80 iterations. These operations iterate generation by generation until some criteria are met.Figure 1
Flowchart of the PCPSO algorithm.Figure 2
The convergence characteristic of the PCPSO on Rotated Ackley’s function in 30 dimensions. First1D and Second1D are CPSO algorithms.Compared with other PSO algorithms using OED [15, 16], the PCPSO needs some extra computations of updating vector XL1. However, the cooperation between local vector XL1 and XL2 makes the searching more effective. In [15], the OED result is set as an exemplar for the other particles to learn. From Figure 2, we can see that sometimes the function values after OED are even bigger. Another limitation of [15] is that the searching method gets trapped into local optimum easily because the two vectors for OED learning are similar. Based on the principle that each element of function vector moves closer to the globally optimal vector after OED, it can be seen that the new vector XLn jumps out of local optimum and keeps searching further even though the function value is higher than corresponding local optimum at start iteration.


## 2.1. Cooperative Particle Swarm Optimizer

Among PSO variants in the literature, premature convergence is still the main deficiency of the PSO especially when facing large dimensional high correlation problems. The difficulty for particles jump out of local optimum is that particles need to change several elements of vectorX=[x1,x2,…,xd,…,xD] simultaneously, but usually the PSO variants can only change one element at one time when facing local optimum. Although the OED method is used to combine the best elements of vector X=[x1,x2,…,xd,…,xD] as an exemplar [21]; the function value f(X) usually gets bigger because of the high correlation. Here, we first introduce the CPSO used in [19]; then the PCPSO is proposed based on CPSO. OED is used to learn and create a new vector from two locally optimal vectors which are achieved by CPSO and the new vector is treated as a start point for further search.In [19], there are two cooperative PSO algorithms, CPSO_Sk and CPSO_Hk. In CPSO_Sk, the D-dimensional search space is decomposed into k subcomponents, each corresponding to a swarm of s-dimensions (where n=k×s). CPSO_Hk is a hybrid method combining a standard PSO with the CPSO_Sk. The k subcomponents are used to place interacting elements together. However, the interacting elements are not known in real problems. We simplify the CPSO_Sk algorithm as CPSO which means the D-dimensional search space is decomposed into D subcomponents. Algorithm 1 shows the algorithm of CPSO. In order to evaluate the fitness of a particle in a swarm, the context vector is applied, which is a concatenation of all global best particles from all D swarms. The evaluation of the ith particle in the dth swarm is done by calling the function b(d,Sd·xi) which returns a D-dimensional vector with its dth component replaced by Sd·xi [22].Algorithm 1:Pseudocode for the generic CPSO algorithm.
(
1) Define  b(d,z)≡(S1·pn,  S2·pn,…,Sd-1·pn,z,Sd+1pn,…,SD·pn)
(2) Create and initialize D one-dimensional PSOs: Sd, d∈[1,…,D]
(3) Repeat:
(4) for each swarm d∈[1,…,D]:
(5) for each particle i∈[1,…,M]:
(6)  if  f(b(d,Sd·xi)) < f(b(d,Sd·pi))
(7)   Sd·pi = Sd·xi
(8)  if  f(b(d,Sd·pi)) < f(b(d,Sd·pn))
(9)   Sd·pn = Sd·pi
(10) end for
(11) Perform velocity and position updates using  (2) for each particle in Sd.
(12)  end for
(13)  Until: stopping condition is met
Thedth swarm is denoted as Sd,  Sd·xi denotes the current position of the ith particle in
thedth swarm, Sd·pn represents the best position in dth swarm. Similarly, Sd·pi
represents the best previous position of particlei in dth swarm. The context vector is the
concatenation ofS1·pn, S2·pn,…,SD·pn.


## 2.2. Parallel and Cooperative Particle Swarm Optimizer Algorithm

Since CPSO method uses one-dimensional swarm to search each dimension and interacting elements of the solution vector cannot be changed simultaneously, it is easier to fall into local optimum. If we try to use the CPSO algorithm (called First1D) to solve the high correlation problem, the vectorXL1=[xL11,xL12,…,xL1d,…,xL1D] started from random values will fall into a locally optimal vector. However, when another CPSO algorithm (called Second1D) which does not have any relationship with First1D is also used to solve the same problem, another locally optimal vector XL2=[xL21,xL22,…,xL2d,…,xL2D] will be obtained. Since the performance of CPSO depends on numerical distribution when X=[x1,x2,…,xd,…,xD] is initialized by random values, the independent vectors XL1 and XL2, containing good information for optimum search, will fall into different locally optimal vectors. It is desperately needed to find a method of extracting good information from XL1 and XL2 and form a new vector XLn which is closer to globally optimal value. If we exhaustively test all the combinations of XL1 and XL2 for the new vector XLn, there are 2D trials which are unrealistic to accomplish in practice. With the help of OED [15, 16, 23, 24], a relatively good vector XLn is obtained from XL1 and XL2, using only a few experimental tests.The OED with both the orthogonal array (OA) and the factor analysis makes it possible to discover the best combinations for different factors with only small number of experimental samples [23, 24]. Here we have a brief discussion about OED combined with an example of vectors XL1 and XL2. In order to simplify the problem, assuming D=3, the optimization problem is to minimize the Sphere function f(X)=x12+x22+x32, with XL1=[2,0,1], XL2=[3,-2,0] which are derived from First1D and Second1D (actually First1D and Second1D should obtain the same globally optimal vector because the test function is simple; here we just want to explain the OED theory). The whole analysis of using OED for Sphere function is shown in Table 1.Table 1
Best combination levels of Sphere function by using OED method.
x
1
x
2
x
3
Function value
C
1
(
X
L
1
)  2
(
X
L
1
)  0
(
X
L
1
)  1
F
1
=
5
C
2
(
X
L
1
)  2
(
X
L
2
)  −2
(
X
L
2
)  0
F
2
=
8
C
3
(
X
L
2
) 3
(
X
L
1
)  0
(
X
L
2
)  0
F
3
=
9
C
4
(
X
L
2
)  3
(
X
L
2
)  −2
(
X
L
1
)  1
F
4
=
14
Levels
Factor analysis
L
1
S
11 = (F1+F2)/2 = 6.5
S
21 = (F1+F3)/2 = 7
S
31 = (F1+F4)/2 = 9.5
L
2
S
12 = (F3+F4)/2 = 11.5
S
22 = (F2+F4)/2 = 11
S
32 = (F2+F3)/2 = 8.5
L
3
(XL1)  2
(XL1)  0
(XL2)  0
F
final
=
4OA is a predefined table which has numbers arranged in rows and columns. In Table1, we can see that there are three factors x1, x2, and x3 that affect function values and two levels XL1 and XL2 to choose. The rows represent the levels of factors in each combination, and the columns represent specific factors that can be changed from each combination [15, 24]. The programming of generating OA can be found in [24]. Factor analysis evaluates the effects of individual factors based on function values and determines the best level for each factor to form a new vector XLn. Assuming Ft denotes a function value of the combination t, where t=1,…,n stands for total number of combinations. The main effect of factor d with level l as Sdl is expressed as(3)sdl=∑t=1nFt·Cdlt∑t=1nCdlt,where d=1,…,D and l=1,2. If the factor is d and the level is l, Cdlt=1, otherwise 0. In Table 1, we first calculate the effect S11 of level 1 in factor x1; then the level 2 in factor x1 is S12. Since this is a minimal optimization problem, the better level 1 is chosen because the effect value of level 1 S11 is less than that of S12 in Table 1. As the example shown in Table 1, although this new vector XLn=[2,0,0] is not shown in the four combinations tested, the best combination can be obtained by OED method.OED works well when the interacting of vector elements is small. When the OED method is used to process some functions with high interacting vector elements, there exists a problem that the function value of new vectorXLn is bigger than the function values of vector XL1 and XL2. However, there are still two advantages by using OED in high correlated problems. One is the particle with new vector XLn jumping out of locally optimal vector; the other one is that each element of the new vector XLn chosen between XL1 and XL2 is probably closer to the globally optimal vector. Therefore, we try to use the new vector XLn as context vector to repeat the CPSO algorithm.Based on the analysis mentioned above, we propose the PCPSO algorithm to overcome the problem of falling into local optimum. Figure1 shows the flowchart of PCPSO. First we use two parallel CPSO algorithms (First1D and Second1D); then OED learning is applied to form the new vector XLn which is assigned to XL2; the better vector chosen from XL1 and XL2 is assigned to the new XL1. The new vectors XL1 and XL2, which are treated as context vectors in CPSO algorithm, keep the search going further. In order to illustrate the efficiency of PCPSO algorithm, Figure 2 shows the convergence characteristic of the PCPSO on Rotated Ackley’s function. The detailed description of Rotated Ackley’s function can be found in Section 3. The parameter settings for First1D and Second1D are the same as CPSO_H algorithm [19], where ω decreases linearly from 0.9 to 0.4, c1=c2=1.494, M=10 stands for particle number, and D=30 represents dimensions. Assuming First1D and Second1D need 80 CPSO iterations to accomplish locally optimal vectors followed with OED computation, there are 6 iterations of OED computation and therefore the total CPSO iteration is 6×80=480. From Figure 2, we can see that the function value drops sharply at first 80 iterations and that First1D and Second1D have similar performance. For the next 80 iterations, the better vector chosen from XL1 and XL2 is assigned to XL1 for another First1D implementation; the new vector XLn is assigned to XL2 for another Second1D implementation. The performance on First1D and Second1D are significantly different in iterations from 80 to 240. First1D algorithm gets trapped in locally optimal vector and Second1D algorithm still makes the function value drop sharply even though it is higher at start point at the 81st iteration. Since First1D algorithm has almost lost searching ability after 80 iterations, we can also adopt the strategy that Firt1D algorithm is only implemented at first 80 iterations in order to save computational cost. Vector XL1 is used for saving the best optimal vector ever found and is combined with vector XL2 for new vector XLn by OED method after 80 iterations. These operations iterate generation by generation until some criteria are met.Figure 1
Flowchart of the PCPSO algorithm.Figure 2
The convergence characteristic of the PCPSO on Rotated Ackley’s function in 30 dimensions. First1D and Second1D are CPSO algorithms.Compared with other PSO algorithms using OED [15, 16], the PCPSO needs some extra computations of updating vector XL1. However, the cooperation between local vector XL1 and XL2 makes the searching more effective. In [15], the OED result is set as an exemplar for the other particles to learn. From Figure 2, we can see that sometimes the function values after OED are even bigger. Another limitation of [15] is that the searching method gets trapped into local optimum easily because the two vectors for OED learning are similar. Based on the principle that each element of function vector moves closer to the globally optimal vector after OED, it can be seen that the new vector XLn jumps out of local optimum and keeps searching further even though the function value is higher than corresponding local optimum at start iteration.


## 3. Experimental Results and Discussions

### 3.1. Test Functions

Test functions including unimodal functions and multimodal functions are used to investigate the performance of PSO algorithms [25]. We choose 15 test functions on 30 dimensions. The formulas and the properties of these functions are presented below.Group A: unimodal and simple multimodal functions are as follows:(1) Sphere function(4)f1x=∑i=1Dxi2,(2) Rosenbrock’s function(5)f2x=∑i=1D-1100xi+1-xi22+xi-12,(3) Griewank’s function(6)f3(x)=∑i=1Dxi24000-∏i=1Dcos⁡xii+1,(4) Rastrigin’s function(7)f4(x)=xi2-10cos⁡(2πxi)+10,(5) Ackley’s function(8)f5x=-20exp⁡-0.21D∑i-1Dxi2-exp⁡1D∑i=1Dcos⁡⁡2πxi+20+e,(6) Weierstrass function(9)f6x=∑i=1D∑k=0kmax⁡akcos⁡⁡2πbkxi+0.5-D∑k=0kmax⁡akcos⁡πbka=0.5,b=3,kmax⁡=20,(7) Noncontinuous Rastrigin’s function(10)f7x=∑i=1Dyi2-10cos⁡⁡2πyi+10,yi=xixi<12round2xi2xi≥12aaaaaaaaaaafori=1,2,…,D,(8) Schwefel’s function(11)f8(x)=418.9829×D-∑i=1Dxisinxi1/2.Group B: Rotated and shifted multimodal functions are as follows.In Group A, the functions can be divided into smaller search spaces because of low interacting of the elements. Therefore, the CPSO algorithm is suitable to optimize these functions. In Group B, seven rotated multimodal functions are generated by an orthogonal matrixM to test the performance of the PCPSO algorithm. The new rotated vector y=M*x, which is obtained through the original vector x left multiplied by orthogonal matrix M, performs like the high interacting vector, because all elements in vector y will be changed once one element in vector x changes. Detailed illustration of generating the orthogonal matrix is introduced in [26]. Meanwhile, one shifted and rotated function is also listed in group B to test the performance of PSO algorithm. Table 2 shows the globally optimal vectors x*, the corresponding function values f(x*), the search range, and the initialization range of each function. Consider the following:(9) Rotated Ackley’s function(12)f9x=-20exp⁡-0.21D∑i=1Dyi2-exp⁡1D∑i=1Dcos⁡⁡2πyi+20+e,y=M*x,(10) Rotated Rastrigin’s function(13)f10x=∑i=1Dyi2-10cos⁡2πyi+10,y=M*x,(11) Rotated Griewank’s function(14)f11x=∑i=1Dyi24000-∏i=1Dcos⁡⁡yii+1,y=M*x,(12) Rotated Weierstrass function(15)f12x=∑i=1D∑k=0kmax⁡akcos⁡⁡2πbkyi+0.5aaaaaaaa-D∑k=0kmax⁡akcos⁡⁡πbka=0.5,b=3,kmax⁡=20,y=M*x,(13) Rotated noncontinuous Rastrigin’s function(16)f13(x)=∑i=1Dzi2-10cos⁡2πzi+10,zi=yiyi<12round2yi2yi≥12fori=1,2,…,D,y=M*x,(14) Rotated Schwefel’s function(17)f14x=418.9829×D-∑i=1Dzi,zi=yisin⁡yi1/2ifyi≤5000otherwiseaaaaaaaaaaaaaafori=1,2,…,D,y=y′+420.96,y′=M*x-420.96,(15) Rotated and Shifted Rastrigin’s function(18)f15x=∑i=1Dzi2-10cos⁡⁡2πzi+10-330,z=M*y,y=x-o,o=o1,o2,…,oD,where o=[o1,o2,…,oD] is the shifted globally optimal vector.Table 2
Globally optimal vectorx*, corresponding function value f(x*), search range, and initialization range of test functions.
Functions
x
*
f
(
x
*
)
Search range
Initialization range
f
1
[0, 0, …, 0]
0
[−100, 100]D
[−100, 50]D
f
2
[1, 1, …, 1]
0
[−2, 2]D
[−2, 2]D
f
3
[0, 0, …, 0]
0
[−600, 600]D
[−600, 200]D
f
4
[0, 0, …, 0]
0
[−5.12, 5.12]D
[−5.12, 2]D
f
5
[0, 0, …, 0]
0
[−32, 32]D
[−32, 16]D
f
6
[0, 0, …, 0]
0
[−0.5, 0.5]D
[−0.5, 0.2]D
f
7
[0, 0, …, 0]
0
[−5.12, 5.12]D
[−5.12, 2]D
f
8
[420.96, 420.96, …, 420.96]
0
[−500, 500]D
[−500, 500]D
f
9
[0, 0, …, 0]
0
[−32, 32]D
[−32, 16]D
f
10
[0, 0, …, 0]
0
[−5.12, 5.12]D
[−5.12, 2]D
f
11
[0, 0, …, 0]
0
[−600, 600]D
[−600, 200]D
f
12
[0, 0, …, 0]
0
[−0.5, 0.5]D
[−0.5, 0.2]D
f
13
[0, 0, …, 0]
0
[−5.12, 5.12]D
[−5.12, 2]D
f
14
[420.96, 420.96, …, 420.96]
0
[−500, 500]D
[−500, 500]D
f
15
[o1,o2,…,oD]
−330
[−5.12, 5.12]D
[−5.12, 2]D


### 3.2. PCPSO’s Performance on 15 Functions

The performance of PCPSO is tested by functionsf1–f15 which includes multimodal, rotated, and shifted functions in 30 dimensions. The parameter settings for the PCPSO algorithm are as follows: ω decreases linearly from 0.9 to 0.4, c1=c2=1.494, and particle number M=10. Since the vectors of f1–f8 have very low interacting elements, Figure 3 shows the convergence characteristics of PCPSO algorithm on test functions f1–f8 in 30 iterations, where the y-axis is expressed in log format. From Figure 3, we can see that all the function values drop significantly in 30 iterations except Rosenbrock’s function. In order to fully test PCPSO’s performance on Rosenbrock’s function, we make a comparison between PCPSO and CPSO algorithm in Figure 4. The upper plot of Figure 4 shows convergence characteristic of PCPSO which cooperates with the two independent algorithms (First1D and Second1D) by using OED learning; the lower plot is the convergence characteristic of CPSO algorithm. From Figure 4, we can see that the OED learning makes the function vector jump out of locally optimal vector and drops the function value finally. The convergence characteristics for the remaining six test functions f10–f15 are shown in Figure 5. All these six functions are rotated multimodal and nonseparable functions. Similar to Figure 2, in spite the fact that the function value of Second1D is increased in some cases at the start point caused by OED learning, it will drop sharply with the increase of iterations. This phenomenon is especially obvious in test function f12. To sum up, PCPSO works well in terms of preventing locally optimal vector and dropping function value sharply. The OED learning finds the best combination of two locally optimal vectors XL1 and XL2 in iterations 80, 160, and 240. However, there are still some drawbacks causing the vector falling into locally optimal vector like f14; the reason is that the vectors XL1 and XL2 are equal to each other, the OED learning ability is lost finally.Figure 3
The convergence characteristics of the PCPSO algorithm on test functionsf1–f8 in 30 iterations.Figure 4
The convergence characteristics of PCPSO and CPSO algorithms on Rosenbrock’s function. (a) PCPSO, (b) CPSO.
(a)
(b)Figure 5
The convergence progress of 30-dimensional test functions using PCPSO. (a) Rotated Rastrigin’s function, (b) Rotated Griewank’s function, (c) Rotated Weierstrass function, (d) Rotated noncontinuous Rastrigin’s function, (e) Rotated Schwefel’s function, and (f) Rotated and Shifted Rastrigin’s function.
(a)
(b)
(c)
(d)
(e)
(f)


### 3.3. Comparison with Other PSOs and Discussions

Seven PSO variants are taken from the literature for comparison on 15 test functions with 30 dimensions. In the following, we briefly describe the peer algorithms shown in Table3. The first algorithm is the standard PSO (SPSO), whose performance is improved by using a random topology. The second algorithm fully informed PSO (FIPS) uses all the neighbor particles to influence the flying velocity. The third orthogonal PSO (OPSO) aims to generate a better position by using OED. The fourth algorithm fitness-distance-ratio PSO (FDR_PSO) solves the premature convergence problem by using the fitness distance ratio. The fifth comprehensive learning PSO (CLPSO) is proposed for solving multimodal problems. The sixth cooperative PSO (CPSO_H) uses one-dimensional swarms to search each dimension separately. The seventh orthogonal learning PSO (OLPSO) can guide particles to fly towards an exemplar which is constructed by Pi and Pn using OED method.Table 3
Parameter settings for the seven PSO variants.
Algorithm
Parameter settings
SPSO [27]
ω: 0.4~0.9, c1=c2=1.193
FIPS [10]
χ
=
0.7298
,
∑
c
i
=
4.1
OPSO [16]
ω: 0.4~0.9, c1=c2=2.0
FDR_PSO [13]
ω: 0.4~0.9, ∑ci=4.0
CLPSO [6]
ω: 0.4~0.9, c1=c2=2.0
CPSO_H [19]
ω: 0.4~0.9, c=1.49,κ=6
OLPSO [15]
ω: 0.4~0.9, c=2.0,G=5The swarm size is set 40 for the seven PSO variants mentioned above and 10 for the PCPSO algorithm. The maximal FEs for the eight algorithms are set 200,000 in each run of each test function. All functions were run 25 times and the mean values and standard deviation of the results are presented in Table4. The best results are shown in boldface. From the results, we observe that CPSO_H, CLPSO, and OLPSO perform well on the first 8 functions. The reason is that CPSO-H suits well on 1-dimensional search, CLPSO and OLPSO can search further by using their own learning strategies. The learning strategy in CLPSO is to prevent falling into local optimum by random learning, while in OLPSO the strategy is to find a better combination of locally optimal vector. PCPSO method can keep searching by the cooperation of two locally optimal vectors XL1 and XL2. With the implementation of Second1D, the algorithm can always find a better vector XL2 compared with the previous best one XL1 by using OED method. The PCPSO method performs well on f2-f3, f9–f11, f13, and f15. However, the function value obtained by this method is not close enough to the globally optimal value, the premature convergence still happens in f12 and f14 due to lack of vector diversity. The advantages of this method are the rapid convergent speed and robustness; whatever test functions are, the algorithm can find an acceptable optimum especially when facing some complex functions. The nonparametric Wilcoxon rank sum tests are taken as t-test in Table 5 to determine whether the results obtained by PCPSO are statistically different from the best results achieved by the other algorithms. A value of one indicates that the performance of the two algorithms is statistically different with 95% certainty, whereas the value of zero implies that the performance is not statistically different. Among these 15 functions, there are 10 functions that are statistically different between the best results and the results of PCPSO. From Tables 4 and 5, we can see that the PCPSO obtains 5 best functions (f2,f3,f10,f11,f13) that are statistically different from the others. In addition, most of the best functions obtained by PCPSO are focused on nonseparable multimodal problems, which demonstrates the effectiveness and robustness of PCPSO algorithm.Table 4
Results for 30D functions.
F
SPSO
FIPS
OPSO
FDR_PSO
f
1
1.16
e
-
069
±
2.86
e
-
068
2.69
e
-
012
±
2.28
e
-
011
6.71
e
-
018
±
6.82
e
-
018
4.88
e
-
102
±
1.53
e
-
101
f
2
1.68
e
+
001
±
8.56
e
+
000
2.47
e
+
001
±
2.19
e
-
001
4.12
e
+
001
±
1.19
e
+
001
1.54
e
+
001
±
6.79
e
+
000
f
3
1.73
e
-
002
±
1.08
e
-
001
4.65
e
-
003
±
3.71
e
-
002
2.10
e
-
003
±
6.19
e
-
003
1.72
e
-
002
±
9.08
e
-
003
f
4
4.61
e
+
001
±
7.85
e
+
001
7.44
e
+
001
±
1.56
e
+
001
3.27
e
+
000
±
6.35
e
+
000
2.91
e
+
001
±
8.06
e
+
000
f
5
1.27
e
+
000
±
4.79
e
+
000
5.12
e
-
007
±
7.81
e
-
007
3.74
e
-
009
±
5.96
e
-
009
2.85
e
-
014
±
4.32
e
-
015
f
6
2.38
e
+
000
±
8.04
e
+
000
3.13
e
-
001
±
3.74
e
-
001
3.28
e
-
002
±
4.93
e
-
002
7.75
e
-
003
±
1.21
e
-
002
f
7
5.21
e
+
001
±
4.19
e
+
001
6.38
e
+
001
±
9.05
e
+
000
4.23
e
+
000
±
8.24
e
+
000
1.45
e
+
001
±
6.58
e
+
000
f
8
3.76
e
+
003
±
3.43
e
+
003
2.56
e
+
003
±
7.31
e
+
002
2.72
e
+
003
±
3.31
e
+
003
3.16
e
+
003
±
4.65
e
+
002
f
9
1.35
e
+
000
±
1.02
e
+
000
7.26
e
-
005
±
1.42
e
-
005
4.07
e
-
005
±
8.26
e
-
006
3.63
e
-
001
±
5.72
e
-
001
f
10
8.38
e
+
001
±
1.53
e
+
002
1.75
e
+
002
±
5.87
e
+
001
1.64
e
+
002
±
3.98
e
+
001
4.67
e
+
001
±
1.44
e
+
001
f
1
1
1.16
e
-
002
±
1.28
e
-
002
6.95
e
-
005
±
3.21
e
-
005
1.13
e
-
003
±
2.45
e
-
003
8.79
e
-
003
±
1.07
e
-
002
f
1
2
2.83
e
+
001
±
5.31
e
+
000
1.35
e
+
001
±
8.39
e
+
000
1.21
e
+
001
±
1.05
e
+
001
2.35
e
+
000
±
1.41
e
+
000
f
1
3
5.51
e
+
001
±
2.07
e
+
001
8.74
e
+
001
±
1.92
e
+
001
7.67
e
+
001
±
7.43
e
+
001
4.64
e
+
001
±
9.01
e
+
000
f
1
4
5.59
e
+
003
±
6.07
e
+
002
2.67
e
+
003
±
7.72
e
+
002
2.45
e
+
003
±
2.89
e
+
002
3.80
e
+
003
±
6.18
e
+
002
f
1
5
-
2.49
e
+
002
±
4.76
e
+
001
-
1.07
e
+
002
±
1.52
e
+
002
-
2.26
e
+
002
±
1.07
e
+
001
-
2.71
e
+
002
±
3.69
e
+
001
f
CLPSO
CPSO_H
OLPSO
PCPSO
f
1
4.46
e
-
014
±
1.51
e
-
014
1.16
e
-
033
±
2.26
e
-
033
1.16
e
-
038
±
1.56
e
-
038
1.12
e
-
030
±
5.14
e
-
31
f
2
2.10
e
+
001
±
2.71
e
+
000
2.06
e
+
001
±
1.13
e
+
001
1.74
e
+
001
±
1.05
e
+
001
1.12
e
+
001
±
8.19
e
+
000
f
3
3.14
e
-
010
±
4.64
e
-
010
3.63
e
-
002
±
7.11
e
-
002
3.17
e
-
010
±
6.23
e
-
011
3.36
e
-
012
±
7.51
e
-
013
f
4
4.76
e
-
010
±
2.17
e
-
010
0
±
0
0
±
0
2.21
e
-
012
±
6.14
e
-
013
f
5
4.97
e
-
014
±
2.12
e
-
014
4.56
e
-
014
±
1.25
e
-
014
4.76
e
-
015
±
1.57
e
-
016
4.83
e
-
008
±
1.38
e
-
008
f
6
4.34
e
-
007
±
2.09
e
-
007
8.15
e
-
015
±
8.92
e
-
016
1.27
e
-
002
±
2.79
e
-
002
2.41
e
-
011
±
1.09
e
-
011
f
7
4.36
e
-
010
±
2.44
e
-
010
1.21
e
-
001
±
3.19
e
-
001
4.51
e
-
008
±
1.17
e
-
008
2.65
e
-
007
±
2.27
e
-
007
f
8
1.26
e
-
012
±
8.62
e
-
013
1.08
e
+
003
±
2.27
e
+
002
3.88
e
-
004
±
6.21
e
-
005
3.82
e
-
004
±
7.31
e
-
006
f
9
3.47
e
-
004
±
2.11
e
-
004
2.13
e
+
000
±
3.55
e
-
001
2.28
e
-
005
±
1.14
e
-
005
2.25
e
-
006
±
1.08
e
-
006
f
10
3.35
e
+
001
±
5.58
e
+
000
8.04
e
+
001
±
2.23
e
+
001
5.77
e
+
001
±
1.27
e
+
001
1.63
e
+
000
±
5.18
e
+
000
f
1
1
7.07
e
-
005
±
1.18
e
-
005
5.54
e
-
002
±
4.79
e
-
002
2.73
e
-
005
±
3.08
e
-
005
1.22
e
-
005
±
5.15
e
-
006
f
1
2
3.09
e
+
000
±
1.51
e
+
000
1.53
e
+
001
±
3.62
e
+
000
7.03
e
+
000
±
1.41
e
+
000
9.37
e
+
000
±
3.58
e
+
000
f
1
3
3.53
e
+
001
±
5.52
e
+
000
8.60
e
+
001
±
2.61
e
+
001
4.03
e
+
001
±
1.34
e
+
001
1.07
e
+
001
±
2.37
e
+
000
f
1
4
2.46
e
+
003
±
1.88
e
+
003
3.82
e
+
003
±
6.62
e
+
002
3.15
e
+
003
±
1.27
e
+
003
3.26
e
+
003
±
1.23
e
+
003
f
1
5
-
2.97
e
+
002
±
2.13
e
+
001
-
2.38
e
+
002
±
7.42
e
+
001
-
2.76
e
+
002
±
3.47
e
+
001
-
3.08
e
+
002
±
1.02
e
+
001Table 5
t-test on 15 functions.
Functions
f
1
f
2
f
3
f
4
f
5
f
6
f
7
f
8
t-test
1
1
1
1
1
0
0
1
Functions
f
9
f
1
0
f
1
1
f
1
2
f
1
3
f
1
4
f
1
5
t-test
0
1
1
1
1
0
0


## 3.1. Test Functions

Test functions including unimodal functions and multimodal functions are used to investigate the performance of PSO algorithms [25]. We choose 15 test functions on 30 dimensions. The formulas and the properties of these functions are presented below.Group A: unimodal and simple multimodal functions are as follows:(1) Sphere function(4)f1x=∑i=1Dxi2,(2) Rosenbrock’s function(5)f2x=∑i=1D-1100xi+1-xi22+xi-12,(3) Griewank’s function(6)f3(x)=∑i=1Dxi24000-∏i=1Dcos⁡xii+1,(4) Rastrigin’s function(7)f4(x)=xi2-10cos⁡(2πxi)+10,(5) Ackley’s function(8)f5x=-20exp⁡-0.21D∑i-1Dxi2-exp⁡1D∑i=1Dcos⁡⁡2πxi+20+e,(6) Weierstrass function(9)f6x=∑i=1D∑k=0kmax⁡akcos⁡⁡2πbkxi+0.5-D∑k=0kmax⁡akcos⁡πbka=0.5,b=3,kmax⁡=20,(7) Noncontinuous Rastrigin’s function(10)f7x=∑i=1Dyi2-10cos⁡⁡2πyi+10,yi=xixi<12round2xi2xi≥12aaaaaaaaaaafori=1,2,…,D,(8) Schwefel’s function(11)f8(x)=418.9829×D-∑i=1Dxisinxi1/2.Group B: Rotated and shifted multimodal functions are as follows.In Group A, the functions can be divided into smaller search spaces because of low interacting of the elements. Therefore, the CPSO algorithm is suitable to optimize these functions. In Group B, seven rotated multimodal functions are generated by an orthogonal matrixM to test the performance of the PCPSO algorithm. The new rotated vector y=M*x, which is obtained through the original vector x left multiplied by orthogonal matrix M, performs like the high interacting vector, because all elements in vector y will be changed once one element in vector x changes. Detailed illustration of generating the orthogonal matrix is introduced in [26]. Meanwhile, one shifted and rotated function is also listed in group B to test the performance of PSO algorithm. Table 2 shows the globally optimal vectors x*, the corresponding function values f(x*), the search range, and the initialization range of each function. Consider the following:(9) Rotated Ackley’s function(12)f9x=-20exp⁡-0.21D∑i=1Dyi2-exp⁡1D∑i=1Dcos⁡⁡2πyi+20+e,y=M*x,(10) Rotated Rastrigin’s function(13)f10x=∑i=1Dyi2-10cos⁡2πyi+10,y=M*x,(11) Rotated Griewank’s function(14)f11x=∑i=1Dyi24000-∏i=1Dcos⁡⁡yii+1,y=M*x,(12) Rotated Weierstrass function(15)f12x=∑i=1D∑k=0kmax⁡akcos⁡⁡2πbkyi+0.5aaaaaaaa-D∑k=0kmax⁡akcos⁡⁡πbka=0.5,b=3,kmax⁡=20,y=M*x,(13) Rotated noncontinuous Rastrigin’s function(16)f13(x)=∑i=1Dzi2-10cos⁡2πzi+10,zi=yiyi<12round2yi2yi≥12fori=1,2,…,D,y=M*x,(14) Rotated Schwefel’s function(17)f14x=418.9829×D-∑i=1Dzi,zi=yisin⁡yi1/2ifyi≤5000otherwiseaaaaaaaaaaaaaafori=1,2,…,D,y=y′+420.96,y′=M*x-420.96,(15) Rotated and Shifted Rastrigin’s function(18)f15x=∑i=1Dzi2-10cos⁡⁡2πzi+10-330,z=M*y,y=x-o,o=o1,o2,…,oD,where o=[o1,o2,…,oD] is the shifted globally optimal vector.Table 2
Globally optimal vectorx*, corresponding function value f(x*), search range, and initialization range of test functions.
Functions
x
*
f
(
x
*
)
Search range
Initialization range
f
1
[0, 0, …, 0]
0
[−100, 100]D
[−100, 50]D
f
2
[1, 1, …, 1]
0
[−2, 2]D
[−2, 2]D
f
3
[0, 0, …, 0]
0
[−600, 600]D
[−600, 200]D
f
4
[0, 0, …, 0]
0
[−5.12, 5.12]D
[−5.12, 2]D
f
5
[0, 0, …, 0]
0
[−32, 32]D
[−32, 16]D
f
6
[0, 0, …, 0]
0
[−0.5, 0.5]D
[−0.5, 0.2]D
f
7
[0, 0, …, 0]
0
[−5.12, 5.12]D
[−5.12, 2]D
f
8
[420.96, 420.96, …, 420.96]
0
[−500, 500]D
[−500, 500]D
f
9
[0, 0, …, 0]
0
[−32, 32]D
[−32, 16]D
f
10
[0, 0, …, 0]
0
[−5.12, 5.12]D
[−5.12, 2]D
f
11
[0, 0, …, 0]
0
[−600, 600]D
[−600, 200]D
f
12
[0, 0, …, 0]
0
[−0.5, 0.5]D
[−0.5, 0.2]D
f
13
[0, 0, …, 0]
0
[−5.12, 5.12]D
[−5.12, 2]D
f
14
[420.96, 420.96, …, 420.96]
0
[−500, 500]D
[−500, 500]D
f
15
[o1,o2,…,oD]
−330
[−5.12, 5.12]D
[−5.12, 2]D


## 3.2. PCPSO’s Performance on 15 Functions

The performance of PCPSO is tested by functionsf1–f15 which includes multimodal, rotated, and shifted functions in 30 dimensions. The parameter settings for the PCPSO algorithm are as follows: ω decreases linearly from 0.9 to 0.4, c1=c2=1.494, and particle number M=10. Since the vectors of f1–f8 have very low interacting elements, Figure 3 shows the convergence characteristics of PCPSO algorithm on test functions f1–f8 in 30 iterations, where the y-axis is expressed in log format. From Figure 3, we can see that all the function values drop significantly in 30 iterations except Rosenbrock’s function. In order to fully test PCPSO’s performance on Rosenbrock’s function, we make a comparison between PCPSO and CPSO algorithm in Figure 4. The upper plot of Figure 4 shows convergence characteristic of PCPSO which cooperates with the two independent algorithms (First1D and Second1D) by using OED learning; the lower plot is the convergence characteristic of CPSO algorithm. From Figure 4, we can see that the OED learning makes the function vector jump out of locally optimal vector and drops the function value finally. The convergence characteristics for the remaining six test functions f10–f15 are shown in Figure 5. All these six functions are rotated multimodal and nonseparable functions. Similar to Figure 2, in spite the fact that the function value of Second1D is increased in some cases at the start point caused by OED learning, it will drop sharply with the increase of iterations. This phenomenon is especially obvious in test function f12. To sum up, PCPSO works well in terms of preventing locally optimal vector and dropping function value sharply. The OED learning finds the best combination of two locally optimal vectors XL1 and XL2 in iterations 80, 160, and 240. However, there are still some drawbacks causing the vector falling into locally optimal vector like f14; the reason is that the vectors XL1 and XL2 are equal to each other, the OED learning ability is lost finally.Figure 3
The convergence characteristics of the PCPSO algorithm on test functionsf1–f8 in 30 iterations.Figure 4
The convergence characteristics of PCPSO and CPSO algorithms on Rosenbrock’s function. (a) PCPSO, (b) CPSO.
(a)
(b)Figure 5
The convergence progress of 30-dimensional test functions using PCPSO. (a) Rotated Rastrigin’s function, (b) Rotated Griewank’s function, (c) Rotated Weierstrass function, (d) Rotated noncontinuous Rastrigin’s function, (e) Rotated Schwefel’s function, and (f) Rotated and Shifted Rastrigin’s function.
(a)
(b)
(c)
(d)
(e)
(f)


## 3.3. Comparison with Other PSOs and Discussions

Seven PSO variants are taken from the literature for comparison on 15 test functions with 30 dimensions. In the following, we briefly describe the peer algorithms shown in Table3. The first algorithm is the standard PSO (SPSO), whose performance is improved by using a random topology. The second algorithm fully informed PSO (FIPS) uses all the neighbor particles to influence the flying velocity. The third orthogonal PSO (OPSO) aims to generate a better position by using OED. The fourth algorithm fitness-distance-ratio PSO (FDR_PSO) solves the premature convergence problem by using the fitness distance ratio. The fifth comprehensive learning PSO (CLPSO) is proposed for solving multimodal problems. The sixth cooperative PSO (CPSO_H) uses one-dimensional swarms to search each dimension separately. The seventh orthogonal learning PSO (OLPSO) can guide particles to fly towards an exemplar which is constructed by Pi and Pn using OED method.Table 3
Parameter settings for the seven PSO variants.
Algorithm
Parameter settings
SPSO [27]
ω: 0.4~0.9, c1=c2=1.193
FIPS [10]
χ
=
0.7298
,
∑
c
i
=
4.1
OPSO [16]
ω: 0.4~0.9, c1=c2=2.0
FDR_PSO [13]
ω: 0.4~0.9, ∑ci=4.0
CLPSO [6]
ω: 0.4~0.9, c1=c2=2.0
CPSO_H [19]
ω: 0.4~0.9, c=1.49,κ=6
OLPSO [15]
ω: 0.4~0.9, c=2.0,G=5The swarm size is set 40 for the seven PSO variants mentioned above and 10 for the PCPSO algorithm. The maximal FEs for the eight algorithms are set 200,000 in each run of each test function. All functions were run 25 times and the mean values and standard deviation of the results are presented in Table4. The best results are shown in boldface. From the results, we observe that CPSO_H, CLPSO, and OLPSO perform well on the first 8 functions. The reason is that CPSO-H suits well on 1-dimensional search, CLPSO and OLPSO can search further by using their own learning strategies. The learning strategy in CLPSO is to prevent falling into local optimum by random learning, while in OLPSO the strategy is to find a better combination of locally optimal vector. PCPSO method can keep searching by the cooperation of two locally optimal vectors XL1 and XL2. With the implementation of Second1D, the algorithm can always find a better vector XL2 compared with the previous best one XL1 by using OED method. The PCPSO method performs well on f2-f3, f9–f11, f13, and f15. However, the function value obtained by this method is not close enough to the globally optimal value, the premature convergence still happens in f12 and f14 due to lack of vector diversity. The advantages of this method are the rapid convergent speed and robustness; whatever test functions are, the algorithm can find an acceptable optimum especially when facing some complex functions. The nonparametric Wilcoxon rank sum tests are taken as t-test in Table 5 to determine whether the results obtained by PCPSO are statistically different from the best results achieved by the other algorithms. A value of one indicates that the performance of the two algorithms is statistically different with 95% certainty, whereas the value of zero implies that the performance is not statistically different. Among these 15 functions, there are 10 functions that are statistically different between the best results and the results of PCPSO. From Tables 4 and 5, we can see that the PCPSO obtains 5 best functions (f2,f3,f10,f11,f13) that are statistically different from the others. In addition, most of the best functions obtained by PCPSO are focused on nonseparable multimodal problems, which demonstrates the effectiveness and robustness of PCPSO algorithm.Table 4
Results for 30D functions.
F
SPSO
FIPS
OPSO
FDR_PSO
f
1
1.16
e
-
069
±
2.86
e
-
068
2.69
e
-
012
±
2.28
e
-
011
6.71
e
-
018
±
6.82
e
-
018
4.88
e
-
102
±
1.53
e
-
101
f
2
1.68
e
+
001
±
8.56
e
+
000
2.47
e
+
001
±
2.19
e
-
001
4.12
e
+
001
±
1.19
e
+
001
1.54
e
+
001
±
6.79
e
+
000
f
3
1.73
e
-
002
±
1.08
e
-
001
4.65
e
-
003
±
3.71
e
-
002
2.10
e
-
003
±
6.19
e
-
003
1.72
e
-
002
±
9.08
e
-
003
f
4
4.61
e
+
001
±
7.85
e
+
001
7.44
e
+
001
±
1.56
e
+
001
3.27
e
+
000
±
6.35
e
+
000
2.91
e
+
001
±
8.06
e
+
000
f
5
1.27
e
+
000
±
4.79
e
+
000
5.12
e
-
007
±
7.81
e
-
007
3.74
e
-
009
±
5.96
e
-
009
2.85
e
-
014
±
4.32
e
-
015
f
6
2.38
e
+
000
±
8.04
e
+
000
3.13
e
-
001
±
3.74
e
-
001
3.28
e
-
002
±
4.93
e
-
002
7.75
e
-
003
±
1.21
e
-
002
f
7
5.21
e
+
001
±
4.19
e
+
001
6.38
e
+
001
±
9.05
e
+
000
4.23
e
+
000
±
8.24
e
+
000
1.45
e
+
001
±
6.58
e
+
000
f
8
3.76
e
+
003
±
3.43
e
+
003
2.56
e
+
003
±
7.31
e
+
002
2.72
e
+
003
±
3.31
e
+
003
3.16
e
+
003
±
4.65
e
+
002
f
9
1.35
e
+
000
±
1.02
e
+
000
7.26
e
-
005
±
1.42
e
-
005
4.07
e
-
005
±
8.26
e
-
006
3.63
e
-
001
±
5.72
e
-
001
f
10
8.38
e
+
001
±
1.53
e
+
002
1.75
e
+
002
±
5.87
e
+
001
1.64
e
+
002
±
3.98
e
+
001
4.67
e
+
001
±
1.44
e
+
001
f
1
1
1.16
e
-
002
±
1.28
e
-
002
6.95
e
-
005
±
3.21
e
-
005
1.13
e
-
003
±
2.45
e
-
003
8.79
e
-
003
±
1.07
e
-
002
f
1
2
2.83
e
+
001
±
5.31
e
+
000
1.35
e
+
001
±
8.39
e
+
000
1.21
e
+
001
±
1.05
e
+
001
2.35
e
+
000
±
1.41
e
+
000
f
1
3
5.51
e
+
001
±
2.07
e
+
001
8.74
e
+
001
±
1.92
e
+
001
7.67
e
+
001
±
7.43
e
+
001
4.64
e
+
001
±
9.01
e
+
000
f
1
4
5.59
e
+
003
±
6.07
e
+
002
2.67
e
+
003
±
7.72
e
+
002
2.45
e
+
003
±
2.89
e
+
002
3.80
e
+
003
±
6.18
e
+
002
f
1
5
-
2.49
e
+
002
±
4.76
e
+
001
-
1.07
e
+
002
±
1.52
e
+
002
-
2.26
e
+
002
±
1.07
e
+
001
-
2.71
e
+
002
±
3.69
e
+
001
f
CLPSO
CPSO_H
OLPSO
PCPSO
f
1
4.46
e
-
014
±
1.51
e
-
014
1.16
e
-
033
±
2.26
e
-
033
1.16
e
-
038
±
1.56
e
-
038
1.12
e
-
030
±
5.14
e
-
31
f
2
2.10
e
+
001
±
2.71
e
+
000
2.06
e
+
001
±
1.13
e
+
001
1.74
e
+
001
±
1.05
e
+
001
1.12
e
+
001
±
8.19
e
+
000
f
3
3.14
e
-
010
±
4.64
e
-
010
3.63
e
-
002
±
7.11
e
-
002
3.17
e
-
010
±
6.23
e
-
011
3.36
e
-
012
±
7.51
e
-
013
f
4
4.76
e
-
010
±
2.17
e
-
010
0
±
0
0
±
0
2.21
e
-
012
±
6.14
e
-
013
f
5
4.97
e
-
014
±
2.12
e
-
014
4.56
e
-
014
±
1.25
e
-
014
4.76
e
-
015
±
1.57
e
-
016
4.83
e
-
008
±
1.38
e
-
008
f
6
4.34
e
-
007
±
2.09
e
-
007
8.15
e
-
015
±
8.92
e
-
016
1.27
e
-
002
±
2.79
e
-
002
2.41
e
-
011
±
1.09
e
-
011
f
7
4.36
e
-
010
±
2.44
e
-
010
1.21
e
-
001
±
3.19
e
-
001
4.51
e
-
008
±
1.17
e
-
008
2.65
e
-
007
±
2.27
e
-
007
f
8
1.26
e
-
012
±
8.62
e
-
013
1.08
e
+
003
±
2.27
e
+
002
3.88
e
-
004
±
6.21
e
-
005
3.82
e
-
004
±
7.31
e
-
006
f
9
3.47
e
-
004
±
2.11
e
-
004
2.13
e
+
000
±
3.55
e
-
001
2.28
e
-
005
±
1.14
e
-
005
2.25
e
-
006
±
1.08
e
-
006
f
10
3.35
e
+
001
±
5.58
e
+
000
8.04
e
+
001
±
2.23
e
+
001
5.77
e
+
001
±
1.27
e
+
001
1.63
e
+
000
±
5.18
e
+
000
f
1
1
7.07
e
-
005
±
1.18
e
-
005
5.54
e
-
002
±
4.79
e
-
002
2.73
e
-
005
±
3.08
e
-
005
1.22
e
-
005
±
5.15
e
-
006
f
1
2
3.09
e
+
000
±
1.51
e
+
000
1.53
e
+
001
±
3.62
e
+
000
7.03
e
+
000
±
1.41
e
+
000
9.37
e
+
000
±
3.58
e
+
000
f
1
3
3.53
e
+
001
±
5.52
e
+
000
8.60
e
+
001
±
2.61
e
+
001
4.03
e
+
001
±
1.34
e
+
001
1.07
e
+
001
±
2.37
e
+
000
f
1
4
2.46
e
+
003
±
1.88
e
+
003
3.82
e
+
003
±
6.62
e
+
002
3.15
e
+
003
±
1.27
e
+
003
3.26
e
+
003
±
1.23
e
+
003
f
1
5
-
2.97
e
+
002
±
2.13
e
+
001
-
2.38
e
+
002
±
7.42
e
+
001
-
2.76
e
+
002
±
3.47
e
+
001
-
3.08
e
+
002
±
1.02
e
+
001Table 5
t-test on 15 functions.
Functions
f
1
f
2
f
3
f
4
f
5
f
6
f
7
f
8
t-test
1
1
1
1
1
0
0
1
Functions
f
9
f
1
0
f
1
1
f
1
2
f
1
3
f
1
4
f
1
5
t-test
0
1
1
1
1
0
0


## 4. Conclusion

In this paper, a novel PCPSO algorithm is presented in order to overcome the problem of falling into local optimum. In PCPSO, considering the complex interacting of the elements of the vector, the CPSO (First1D and Second1D) algorithm is used twice to create two locally optimal vectors. These two vectors are combined together by OED learning so that the best elements can be chosen to jump out of local optimum and search further.Although OED operation may cause the function value higher than local optima in some cases, the function value will always drop sharply after OED because the OED operation chooses the elements that are closer to globally optimal vector. Experimental tests have been conducted on 15 benchmark functions including unimodal, multimodal, rotated, and shifted functions. From the comparative results, it can be concluded that PCPSO significantly improves the performance of PSO and provides a better solution on most rotated multimodal problems.



---

*Source: 743671-2015-04-07.xml*