2009 International Workshop on Nonlinear Maps and their Applications
NOMA'09, Urbino, Italy, September 10-11, 2009 (paper id: 011)

Particle Swarm Optimization Containing
Characteristic Swarms
Masaki SUGIMOTO, Taku HARAGUCHI, Haruna MATSUSHITA and Yoshifumi NISHIO
Dept. Electrical and Electronic Eng.,
Tokushima University
Email: { sugimoto, taku, haruna, nishio}@ee.tokushima-u.ac.jp

I. I NTRODUCTION
Particle Swarm Optimization (PSO) [1] is a popular optimization technique for solving objective functions and PSO
is an evolutionary algorithm to simulate the movement offlocks of birds toward foods. Due to its simple concept, easy
implementation and quick convergence, PSO has attracted
attentions and has been widely applied to different fields
in recent years. Furthermore, PSO has demonstrated great
performances for many problems. However, quick convergenceoften leads to local optimum problem. It is important for
multimodal functions with a lot of local optima to compromise
between the quick convergence and being trapped in a local
optimum. In order to escape from such local optima andto avoid the premature convergence, the search for a global
optimum should be diverse. In particular, some researchers
have proposed the PSO method using multiple swarms whose
particles exchange their information among them .
On the other hand, in the real world, a company performs
its business in an effective way. Each member of a company
belongs to a department which has a determined role. Some
departments cooperate each other to achieve a common goal.
Assignment to a department is decided by a member’s ability
and/or outcome.
In this study, we propose an improved PSO algorithm using
plural swarms; PSO containing plural swarms and the particles
have different features (called PPSO). The proposed algorithm
reflects the concept of a company in the real world. The
important features of PPSO are that each particle of PPSO
belongs to one of plural swarms, each particle have different
characteristics, and are periodically reconstituted. At every
generation, each particle is ranked by its cost among all theparticles in all the swarms. Furthermore, the particles periodically change the swarm and the characteristics, according to
its total ranking.
We explain the algorithm of PPSO in detail in Section II.
In Section III, we perform basic numerical experiments byusing four algorithm methods of PSOs including PPSO. Furthermore, we confirm the efficiency of PPSO for multimodal
functions.
II. PSO CONTAINING PLURAL SWARMS AND THE
PARTICLES HAVE DIFFERENT FEATURES (PPSO)
Each particle of PSO has two information; position andvelocity. The position vector of each particle i and its velocity
vector are represented by X i = (xi1 , xi2 , . . . , xiD ) and V i =(vi1 , vi2 , . . . , viD ), respectively, where (d = 1, 2, . . . , D),(i = 1, 2, ..., N ) and xid ∈ [xmin , xmax ].
The important features of PPSO are that each particle of
PPSO belongs to one of plural swarms, which have different
characteristics, and each particle periodically changes theswarm that it belongs to. Each swarm is denoted as Sk(k = 1, 2, . . . , K) and has N/K particles. The position vectorof each particle i and its velocity vector are updated depending
on its personal best position, the best position among all theparticles and the best position among the particles belonging
to the swarm. At every generation, each particle is ranked byits cost f (X i ) among all the particles in all the swarms. At
every Tc generation, the particle changes the swarm, that it
belongs to, according to its total ranking Ri .
[PPSO1] (Initialization) Let a generation step t = 0 and tc =0, i.e., tc is the time step for the reconstitution of the swarms.
Randomly initialize the particle position X i and its velocity
V i for all particles i and initialize P i = (pi1 , pi2 , ..., piD )with a copy of X i . Evaluate the objective function f (X i )for each particle i, and find P g with the best function value
among the all particles. Attach each particle i to any swarm
Sk at random.
[PPSO2] Evaluate the current cost f (X i ). Update the personal
best position pbest P i for each particle i and the global best
position gbest P g among the all particles in all the swarms
so far.
[PPSO3] Let P sk = (psk 1 , psk 2 , · · · , psk D ) represents theswarm best position with the best cost among the particles
belonging to the swarm Sk so far (called sbest). Update P sk
for each swarm Sk , if needed.
sk = arg min{f (X i )},i

i ∈ Sk .

(1)[PPSO4] Assign each particle i a rank r = 1, · · · , N depending on its cost f (X i ). The best rank of 1 indicates the particle
with the smallest function value, and the worst rank of N
indicates the particle with the largest function value. ri denotes
the sum of the rank of particle i so far; rinew = riold + ri .
[PPSO5] Updated V i and X i of each particle i depending
on its pbest, its swarm best sbest and gbest;vid (t + 1) =wRi vid + c1 r1 {pid − xid (t)}+ c2 r2 {psk d − xid (t)} + c3 r3 {pgd − xid (t)},xid (t + 1) =xid (t) + vid (t + 1),(2)- 87 -TABLE IC OMPARISON RESULTS PSO1, PSO2, PPSO-R AND PPSO ON TESTFUNCTIONS WITH D = 30.

where r1 , r2 and r3 are random variables distributed uniformly
on [0, 1], and c1 , c2 and c3 are positive acceleration coefficients. wRi is an inertia weight of the each particle i. In otherwords, each particle has different features depending on wRi ,and the particles belonging to different total ranking behave
differently.
[PPSO6] If tc = Tc , perform [PPSO7]. If not, perform[PPSO10]. Thus, perform [PPSO7] every Tc generation. Tc
is a fixed parameter and is the number of generations toreconstitute the swarms.
[PPSO7] Rank each particle i as R = 1, · · · , N according to
its ri . Ri denotes the total rank of the particle i. Ri = 1 is thebest total rank and indicates the particle i with the smallest
ri . Therefore, the inertia weight wRi[PPSO8] Reconstitute the respective swarms according to theNtotal rank Ri of each particle i. Each swarm k contains K
particles, and the best swarm S1 consists of the particles withNhigh total rank Ri from 1 to K
. In other words, the eachswarm
Sincludestheparticleswiththe total rank Ri fromk()(k−1)NkN+ 1 to K .
K[PPSO9] Initialize the past total rank for all the particles,
namely reset Ri = 0 for all the particles, and reset tc = 0.
[PPSO10] Let t = t + 1 and tc = tc + 1. Go back to [PPSO2],
and repeat until t = T .

f
f1
f2
f3
f4

Avg.
Min.
Max.
Avg.
Min.
Max.
Avg.
Min.
Max.
Avg.
Min.
Max.

PSO1
3.96e-13
5.54e-93
1.19e-11
33.42
1.40e-02
76.25
35.04
13.86
51.48
7.96e-0302.46e-02

PSO2
2.48e-28
9.02e-46
7.41e-27
30.15
2.36
85.68
22.31
13.86
33.66
1.01e-02
03.70e-02

PPSO-R
4.08e-32
1.21e-60
1.17e-30
27.87
12.20
75.85
15.74
3.96
26.73
3.29e-04
09.86e-03

PPSO
2.03e-33
2.80e-53
6.01e-32
22.77
8.42
72.93
14.50
6.93
26.73
8.83e-1002.65e-08

TABLE IIC OMPARISON R ESULTS PSO1, PSO2, PPSO-R AND PPSO ON TESTFUNCTIONS WITH D = 100.

f
f1
f2
f3

III. N UMERICAL E XPERIMENTS
f4

In order to confirm the performance of PPSO algorithm,
we have performed basic numerical experiments. The problem
is finding the optimum (minimum) value of f (x) in thealgorithm. We use the following four bench marks [2].

Avg.
Min.
Max.
Avg.
Min.
Max.
Avg.
Min.
Max.
Avg.
Min.
Max.

PSO1
4.97e-01
5.60e-08
8.01
253.80131.25672.93317.56228.68394.57
4.18e-03
7.77e-11
3.98e-02

PSO2
2.03e-01
7.96e-03
2.18
704.94288.492124.28
319.37243.58427.409.08e-03
8.59e-05
3.03e-02

PPSO-R
5.10e-01
1.48e-02
2.72
1206.34546.212584.70
174.73109.89284.851.41e-02
1.80e-04
6.21e-02

PPSO
7.39e-01
1.90e-02
4.72
1006.64556.76
1993.77167.49115.84233.533.94e-03
8.88e-16
1.97e-02−5.12 ≤ xi ≤ 5.12.
1. Sphere function:
f1 (x) =D−1∑

4. Griewank’s function:
x2d ,

(3)
f4 (x) =d=1

where x ∈ [−2.048, 2.047]D and the optimum solution x∗ are
all [0, 0, . . . , 0].
2. Rosenbrock’s function:

D
D
∑
∏
xd
x2d
+
cos( √ ) + 1,
4000
d
d=1

(6)

d=1

where x ∈ [−600, 600]D and the optimum solution x∗ are all[0, 0, . . . , 0].

(5)The optimum function values f (x∗ ) of all functions are
0. f1 and f2 are unimodal functions, and f3 and f4 aremultimodal functions with numerous local minima. All thefunctions have D variables. In this study, D is set to 30 and
100 to investigate the performances in various dimensions.
In order to evaluate the efficiency of PPSO and to investigate
behaviors of PPSO, we compare the four algorithms;PSO1,
PSO2, PPSO-R and PPSO. PSO1 is the standard PSO andPPSO is the proposed algorithm explained in Section II. In
PSO2, the inertia weight w is not the fixed value unlike the
PSO1 and monotonically decreases with the generations as

where x ∈ [−5.12, 5.12] and the optimum solution x are
all [0, 0, . . . , 0]. We consider that an almost optimum value isobtained if the algorithm attains the criterion f1 (x) = 100.
This criterion is based on [2]. The range of initialization is

wmax − wmin
× t,
(7)
T
where wmax and wmin are the maximum and minimum valueof w(t), respectively. In order to investigate the effect of omitting [PPSO6] to [PPSO9] in Section. II, we compare PPSO

f2 (x) =D−1∑(100(x2d − xd+1 )2 + (1 − xd )2 ),

(4)

d=1

where x ∈ [−2.048, 2.047]D and the optimum solution x∗ are
all [1, 1, . . . , 1].
3. Rastrigin’s function and its optimum (minimum):f3 (x) =D∑(x2d − 5 cos(2πxd ) + 5),

d=1
D

∗

- 88 -

w(t) = wmax −

6

10

5

10

PSO1
PSO2
PPSO-R
PPSO

0

10

-5

10

Function Value log(f(x))

Function Value log(f(x))

10

PSO1
PSO2
PPSO-R
PPSO

5

-10

10

-15

10

-20

10

4

10

3

10

-25

10
2

10
-30

10

1

-35

10

10
0

500

1000

1500

2000

2500

0

3000

500

1000

1500

2000

2500

3000

Number of Generation t

Number of Generation t

(a)

(b)

3

10

2

10

PSO1
PSO2
PPSO-RPPSO

010Function Value log(f(x))

Function Value log(f(x))

PSO1
PSO2
PPSO-R
PPSO

2

10

-2

10

-4

10

-6

10

-8

10

1

-10

10

10

0

500

1000

1500

2000

2500

3000

0

Number of Generation t

500

1000

1500

2000

2500

3000

Number of Generation t

(c)

(d)

Fig. 1. Mean gbest value of every generation for 30-dimensional four functions. (a) Sphere function. (b) Rosenbrock’s function. (c) Rastrigin’s function.
(d) Griewank’s function.

with PPSO-R algorithm which does not have the ranking
evaluation and the reconstitute process of the swarms. The
inertia weight w of PPSO-R is the same as PPSO, which hasdifferent value by each swarm; however, all the particles keepon the constitution of the initial swarm.
The population size N is set to 60 in PSO1 and PSO2.
PPSO-R and PPSO have K = 6 swarms, and each swarm
contains 10 particles, i.e. N = 60. For PSO1, the inertia
weight is fixed as w = 0.5 which is a mean value of all
the inertia weights used in PPSO-R and PPSO. For PSO2, w
is decreased with time according to Eq. (8) where wmax = 0.9
and wmin = 0.4. For PSO1 and PSO2, the acceleration coefficients are set as c1 = c2 = 1.8. Because PPSO-R and PPSO
consist of the plural swarms whose features are different, we
use difference inertia weight on each swarm. For PPSO-R and
PPSO, the parameters are set as c1 = 1.8, c2 = 1.4, c3 = 0.4,
w is
wmax − wmin× (Ri − 1).
(8)wRi = wmax −(N − 1)
The timing of reconstruction is set as Tc = 100.
We carry out the simulation 30 times for all the optimization
functions with 3000 generations, namely T = 3000. The
performances with minimum, maximum and mean function

values on four functions with 30-dimension are listed inTable I. We can see that the mean values of PPSO are the bestin only f3 and f4 which are the two multimodal functions.
However, in case of the performances on the test functions
with D = 100 shown in Table II, PPSO can obtain the best
mean values for the two multimodal functions. In particular,
PPSO greatly improves the performance from PSO1 on f1 ,f3 and f4 with D = 30, and f3 with D = 100. From theseresults, we can say that PPSO is more effective for multimodal
functions.
Figure 1 shows the mean gbest values of every generation
over 30 runs for four test functions with 30 variables. For
the multimodal functions f3 and f4 as Figs. 1(c) and (d),PSO1, which is the standard PSO, are bad values. Instead,the proposed PPSO can obtain the best results. Meanwhile,
the mean gbest values for 100 dimensional test functions
are shown in Fig. 2. The performances of PPSO are thebest values in both the multimodal functions. Because it
is difficult for PSO1 to find the optimum solution on themultimodal functions, PSO1 are easily trapped in the local
optima and prematurely converge. Let us consider PPSO-R
and PPSO which have plural swarms and search with sharing
the information of the swarm best position sbest. The particles- 89 -

7

3

10

10

PSO1
PSO2
PPSO-R
PPSO

2

6

10

Function Value log(f(x))

Function Value log(f(x))

10

1

10

PSO1
PSO2
PPSO-R
PPSO

5

10

4

10

0

10
3

10

2

-1

10

10
0

500

1000

1500

2000

2500

0

3000

500

1000

(a)

2000

2500

3000

(b)

4

1

10

10

PSO1
PSO2
PPSO-R
PPSO

PSO1
PSO2
PPSO-RPPSO

010Function Value log(f(x))

Function Value log(f(x))

1500
Number of Generation t

Number of Generation t

3

10

-1

10

-2

10

2

-3

10
0

500

1000

1500

2000

2500

10

3000

0

500

1000

1500

2000

2500

3000

Number of Generation t

Number of Generation t

(c)

(d)

Fig. 2. Mean gbest value of every generation for 100-dimensional four functions. (a) Sphere function. (b) Rosenbrock’s function. (c) Rastrigin’s function.
(d) Griewank’s function.

belonging to the different particles have the different inertia
weight wRi , namely, their behaviors are different. The searchrange of PPSO-R and PPSO are wider than PSO1 because
there are some kinds of particles in PPSO-R and PPSO andthey are updated depending on its pbest, sbest and gbest.
However, we can see that PPSO-R cannot find the optimum
solution. Then, because PPSO-R does not have the reconstituteprocess of swarms, the attracting force of PPSO-R to theglobal best position gbest is weak even if the particles belong
to the swarm whose sbest is far from the optimum solution.
On the other hand, in PPSO, the particles moving around thearea, where is very far from the optimum solution, are attracted
toward gbest by the reconstitute process which is to redefine
the role of respective particles. In other words, the particles of
PPSO can search the solution at the wide region and in depth.
From these results, we can say that PPSO is the effective
algorithm, which has the plural kinds of particles and whose
particle situation is periodically reconstituted, for the themultimodal functions.
IV. C ONCLUSIONSIn this study, we have proposed a new Particle Swarm
Optimization algorithm (PPSO) containing the plural swarms
and each particle had different features. We have performed

the basic numerical experiments for various dimensions byusing four algorithm methods of PSOs including PPSO. We
have confirmed that PPSO could obtain the effective results
especially for the complex problems such as the multimodal
function.
R EFERENCES[1] J. Kennedy and R. Eberhart, “Particle swarm optimization,” Proc. IEEE Int. Conf. on Neural Networks, vol. 4,
pp. 1942–1948, 1995.
[2] J. Kennedy and R. Medes, “Population structure andparticle swarm performance,” Proc. IEEE Congress on
Evolutionary Computation, vol. 2, pp. 1671–1676, 2002.
[3] E. Miyagawa and T. Saito, “Particle swarm optimizers
with grow-and-reduce structure,” 4Proc. IEEE Congress on
Evolutionary Computation, pp. 3974–3979, 2008.
[4] G. Yen and M. Daneshyari, “Diversity-based informationexchange among multiple swarm in particle swarm optimization,” Proc. IEEE Congress on Evolutionary Computation, pp. 1686–1693, 2006.
[5] M. Iwamatsu, “Multi-species particle swarm optimizer
for multimodal function optimization,” IEICE Trans. on
Information and Systems, vol. E89-D, no. 3, pp. 1181–
1187, 2006.

- 90 -