2009 International Symposium on Nonlinear Theory and its Applications
NOLTA'09, Sapporo, Japan, October 18-21, 2009

Particle Swarm Optimization Containing Plural Kinds of Swarms
Masaki Sugimoto, Taku Haraguchi, Haruna Matsushita and Yoshifumi Nishio
Department of Electrical and Electronic Engineering, Tokushima University,
2-1 Minami-Josanjima, Tokushima 770-8506, Japan
Email: {sugimoto, taku, haruna, nishio}@ee.tokushima-u.ac.jp
Abstract— This study proposes a new Particle Swarm
Optimization (PSO) algorithm containing plural swarms
whose particles have diﬀerent features. Each particle ofthe proposed PSO belongs to one of plural swarms, whichhave diﬀerent characteristics, and the particle periodicallychanges the swarm that it belongs to. We confirm the eﬀectiveness of the proposed PSO for both unimodal and multimodal functions with various dimensions.
1. Introduction
Particle Swarm Optimization (PSO) [1] is a popular optimization technique for solving objective functions and PSO
is an evolutionary algorithm to simulate the movement offlocks of birds toward foods. Due to its simple concept,
easy implementation and quick convergence, PSO has attracted attentions and has been widely applied to diﬀerent
fields in recent years. Furthermore, PSO has demonstrated
great performances for many problems. However, quick
convergence often leads to local optimum problem. It is
important for multimodal functions with a lot of local optima to compromise between the quick convergence andbeing trapped in a local optimum. In order to escape fromsuch local optima and to avoid the premature convergence,the search for a global optimum should be diverse. Many
researchers have improved the performance of PSO by enhancing its ability with more diverse search [2]-[6]. In particular, some researchers have proposed the PSO method
using multiple swarms whose particles exchange their information among them [7][8].
On the other hand, in real world, a company performs
its business in an eﬀective way. Each member of a company belongs to a department which has a determined role.
Some departments cooperate each other to achieve a common goal. Assignment to a department is decided by amember’s ability and/or outcome.
In our past study, we have proposed a modified PSO
algorithm using plural swarms [9]. This algorithm contained plural swarms whose features were the same, and theswarms shared the information of the best position among
them. Furthermore, all the particles were repositioned periodically. However, the eﬀectiveness of using plural swarms
has not been clearly understood.
In this study, we propose an improved PSO algorithm
using plural swarms; PSO containing plural swarms whose
particles have diﬀerent features (called PPSO). The pro-

posed algorithm reflects the concept of a company in thereal world. The proposed PPSO does not have the reposition process and each swarm has diﬀerent features, unlike
our past algorithm. The important features of PPSO arethat each particle of PPSO belongs to one of plural swarms,
which have diﬀerent characteristics, and are periodicallyreconstituted. At every generation, each particle is ranked
by its cost among all the particles in all the swarms. Furthermore, the particles periodically change the swarm thatit belongs to, according to its total ranking.
We explain the algorithm of PPSO in detail in Section 2.
In Section 3, we perform basic numerical experiments byusing four algorithm methods of PSOs including PPSO.
Furthermore, we confirm the eﬃciency of PPSO for highdimensional functions.
2. PSO containing plural swarms whose particles havediﬀerent features (PPSO)
Each particle of PSO has two information; position andvelocity. The position vector of each particle i and its
velocity vector are represented by Xi = (xi1 , xi2 , . . . , xiD )and V i = (vi1 , vi2 , . . . , viD ), respectively, where (d =1, 2, . . . , D), (i = 1, 2, ..., N) and xid ∈ [xmin , xmax ].
The important features of PPSO are that each particle of
PPSO belongs to one of plural swarms, which have diﬀerent characteristics, and each particle periodically changes
the swarm that it belongs to. Each swarm is denoted as
S k (k = 1, 2, . . . , K) and has N/K particles. The position
vector of each particle i and its velocity vector are updated
depending on its personal best position, the best position
among all the particles and the best position among theparticles belonging to the swarm. At every generation, eachparticle is ranked by its cost f (Xi ) among all the particles inall the swarms. At every T c generation, the particle changes
the swarm, that it belongs to, according to its total ranking
Ri .
[PPSO1] (Initialization) Let a generation step t = 0 and
tc = 0, i.e., tc is the time step for the reconstitution of
the swarms. Randomly initialize the particle position Xi
and its velocity V i for all particles i and initialize Pi =(pi1 , pi2 , ..., piD ) with a copy of Xi . Evaluate the objectivefunction f (Xi ) for each particle i, and find Pg with the best
function value among the all particles. Attach each particle
i to any swarm S k at random.
[PPSO2] Evaluate the current cost f (Xi ). Update the per-- 584 -

sonal best position pbest Pi for each particle i and theglobal best position gbest Pg among the all particles in all
the swarms so far.
[PPSO3] Let Psk = (p sk 1 , p sk 2 , · · · , p sk D ) represents theswarm best position with the best cost among the particles
belonging to the swarm S k so far (called sbest). Update
Psk for each swarm S k , if needed.
sk = arg min{ f (Xi )},i

i ∈ S k.

f1 (x) =

xd2 ,

(3)

d=1

where x ∈ [−2.048, 2.047]D and the optimum solution x∗
are all [0, 0, . . . , 0].
2. Rosenbrock’s function:(1)[PPSO4] Assign each particle i a rank r = 1, · · · , N depending on its cost f (Xi ). The best rank of 1 indicatesthe particle with the smallest function value, and the worst
rank of N indicates the particle with the largest function
value. ri denotes the sum of the rank of particle i so far;rinew = riold + ri .
[PPSO5] Updated V i and Xi of each particle i depending
on its pbest, its swarm best sbest and gbest;vid (t + 1) =wk vid + c1 r1 {pid − xid (t)}{}+ c2 r2 p sk d − xid (t) + c3 r3 {pgd − xid (t)}, (2)xid (t + 1) =xid (t) + vid (t + 1),
where r1 , r2 and r3 are random variables distributed uniformly on [0, 1], and c1 , c2 and c3 are positive accelerationcoeﬃcients. wk is an inertia weight of the each swarm S k .
In other words, each swarms has diﬀerent features depending on wk , and the particles belonging to diﬀerent swarms
behave diﬀerently.
[PPSO6] If tc = T c , perform [PPSO7]. If not, perform[PPSO10]. Thus, perform [PPSO7] every T c generation.
T c is a fixed parameter and is the number of generations toreconstitute the swarms.
[PPSO7] Rank each particle i as R = 1, · · · , N according
to its ri . Ri denotes the total rank of the particle i. Ri = 1
is the best total rank and indicates the particle i with thesmallest ri .
[PPSO8] Reconstitute the respective swarms according to
the total rank Ri of each particle i. Each swarm k contains
N
K particles, and the best swarm S 1 consists of the particles
with high total rank Ri from 1 to NK . In other words, theeach swarm
S k includes
the particles with the total rank Ri()
kNfrom (k−1)N
+
1
to
.
K
K
[PPSO9] Initialize the past total rank for all the particles,
namely reset Ri = 0 for all the particles, and reset tc = 0.
[PPSO10] Let t = t+1 and tc = tc +1. Go back to [PPSO2],
and repeat until t = T .

f2 (x) =D−1∑(100(xd2 − xd+1 )2 + (1 − xd )2 ),where x ∈ [−2.048, 2.047]D and the optimum solution x∗
are all [1, 1, . . . , 1].
3. Rastrigin’s function and its optimum (minimum):f3 (x) =D∑(xd2 − 5 cos(2πxd ) + 5),

(5)

d=1

where x ∈ [−5.12, 5.12]D and the optimum solution x∗ are
all [0, 0, . . . , 0]. We consider that an almost optimum value
is obtained if the algorithm attains the criterion f1 (x) =100. This criterion is based on [2]. The range of initialization is −5.12 ≤ xi ≤ 5.12.
4. Griewank’s function:
f4 (x) =

D
D
∑
∏
xd2
xd
+
cos( √ ) + 1,
4000
d
d=1
d=1

(6)

where x ∈ [−600, 600]D and the optimum solution x∗ are
all [0, 0, . . . , 0].
The optimum function values f (x∗ ) of all functions are 0.
f1 and f2 are unimodal functions, and f3 and f4 are multimodal functions with numerous local minima. All the functions have D variables. In this study, D is set to 30 and 100
to investigate the performances in various dimensions.
In order to evaluate the eﬃciency of PPSO and to investigate behaviors of PPSO, we compare the four algorithms;PSO1, PSO2, PPSO-R and PPSO. PSO1 is the standard PSO and PPSO is the proposed algorithm explainedin Section 2. In PSO2, the inertia weight w is not the fixedvalue unlike the PSO1 and monotonically decreases withthe generations as
w(t) = wmax −In order to confirm the performance of PPSO algorithm,
we have performed basic numerical experiments. The
problem is finding the optimum (minimum) value of f (x) in
the algorithm. We use the following four bench marks [2].

(4)

d=1

3. Numerical Experiments

1. Sphere function:

D−1∑wmax − wmin
× t,
T

(7)

where wmax and wmin are the maximum and minimum valueof w(t), respectively. In order to investigate the eﬀectof omitting [PPSO6] to [PPSO9] in Section. 2, we compare PPSO with PPSO-R algorithm which does not havethe ranking evaluation and the reconstitute process of theswarms. The inertia weight w of PPSO-R is the same

- 585 -

Table 1: Comparison results PSO1, PSO2, PPSO-R and
PPSO on test functions with D = 30.
f

f1

f2

f3

f4

Avg.
Min.
Max.
Avg.
Min.
Max.
Avg.
Min.
Max.
Avg.
Min.
Max.

PSO1
9.04e-61
7.38e-66
1.45e-59
20.34
0.06
73.78
31.05
13.86
46.53
8.60e-0305.62e-02

PSO2
2.19e-33
1.16e-46
4.42e-32
30.16
2.13
87.73
22.44
15.84
30.69
1.14e-02
06.87e-02

PPSO-R
8.24e-25
6.37e-39
2.24e-23
29.00
15.08
76.94
15.75
6.93
28.81
2.37e-0302.16e-02

PPSO
1.13e-29
8.56e-35
6.19e-29
21.57
0.06
76.51
14.95
5.94
22.77
1.31e-03
0
1.48e-02

Table 2: Comparison Results PSO1, PSO2, PPSO-R and
PPSO on test functions with D = 100.
f

f1

f2

f3

f4

Avg.
Min.
Max.
Avg.
Min.
Max.
Avg.
Min.
Max.
Avg.
Min.
Max.

PSO1
1.08e-06
2.18e-11
3.00e-05
17035.82
98.53
503344.7
254.68197.00324.707.47e-03
6.92e-12
2.22e-02

PSO2
2.20e-01
4.76e-04
1.01
747.08367.54
1958.46307.93200.48401.125.29e-03
1.15e-04
2.76e-02

PPSO-R
3.48
2.08e-01
18.41
5478.69
514.8914010.7
208.83152.66314.865.78e-02
1.67e-02
1.81e-01

PPSO
9.92e-10
1.88e-15
1.28e-08
179.8377.57
311.71138.4895.26
190.072.71e-03
6.66e-16
2.22e-02

as PPSO, which has diﬀerent value by each swarm; however, all the particles keep on the constitution of the initialswarm.
The population size N is set to 60 in PSO1 and PSO2.
PPSO-R and PPSO have K = 6 swarms, and each swarm
contains 10 particles, i.e. N = 60. For PSO1, the inertia weight is fixed as w = 0.6 which is a mean valueof all the inertia weights used in PPSO-R and PPSO. For
PSO2, w is decreased with time according to Eq. (7) where
wmax = 0.9 and wmin = 0.4. For PSO1 and PSO2, theacceleration coeﬃcients are set as c1 = c2 = 1.8. Because
PPSO-R and PPSO consist of the plural swarms whose features are diﬀerent, we use diﬀerence inertia weight on eachswarm. For PPSO-R and PPSO, the parameters are set as
c1 = 1.8, c2 = 1.4, c3 = 0.4, w1 = 0.9, w2 = 0.8, w3 =0.7, w4 = 0.6, w5 = 0.5 and w6 = 0.4. The timing of reconstruction is set as T c = 100.
We carry out the simulation 30 times for all the optimization functions with 3000 generations, namely T = 3000.
The performances with minimum, maximum and meanfunction values on four functions with 30-dimension arelisted in Table 1. We can see that the mean values of PPSO

are the best in only f3 and f4 which are the two multimodal
functions. However, in case of the performances on the test
functions with D = 100 shown in Table 2, PPSO can obtain
the best mean values for all the test functions. In particular,
PPSO greatly improves the performance from PSO1 on f1 ,f2 and f3 with D = 100. From these results, we can say that
PPSO is more eﬀective for higher dimensional functions.
Figure 1 shows the mean gbest values of every generation over 30 runs for four test functions with 30 variables.
For the unimodal functions f1 and f2 as Figs. 1(a) and (b),PSO1, which is the standard PSO, can obtain the best results. However, the performances of PSO1 for the multimodal functions f1 and f2 as Figs. 1(c) and (d) are bad values, instead, the proposed PPSO can obtain the best results.
Meanwhile, the mean gbest values for 100 dimensional test
functions are shown in Fig. 2. The performances of PPSO
are the best values in all the four test functions, and PSO1
cannot obtain good results for both the unimodal functions
and the multimodal functions. Because it is diﬃcult for
PSO1 to find the optimum solution on the high-dimensional
functions, PSO1 are easily trapped in the local optima andprematurely converge.
Let us consider PPSO-R and PPSO which have plural swarms and search with sharing the information of
the swarm best position sbest. The particles belonging to
the diﬀerent swarms have the diﬀerent inertia weight wk ,namely, their behaviors are diﬀerent. The search range of
PPSO-R and PPSO are wider than PSO1 because there aresome kinds of particles in PPSO-R and PPSO and they are
updated depending on its pbest, sbest and gbest. However,
we can see that PPSO-R cannot find the optimum solution. Then, because PPSO-R does not have the reconstitute process of swarms, the attracting force of PPSO-R to
the global best position gbest is weak even if the particles
belong to the swarm whose sbest is far from the optimum
solution. On the other hand, in PPSO, the particles movingaround the area, where is very far from the optimum solution, are attracted toward gbest by the reconstitute process
which is to redefine the role of respective particles. In otherwords, the particles of PPSO can search the solution at thewide region and in depth.
Meanwhile, PSO2 also uses the diﬀerent inertia weightin the simulation. However, the inertia weights of all
the particles simultaneously decrease with generation, andPSO2 cannot achieve the good results.
From these results, we can say that PPSO is the eﬀective algorithm, which has the plural kinds of particles andwhose swarm situation is periodically reconstituted, for thecomplex problems such as the multimodal and the highdimension.
4. Conclusions
In this study, we have proposed a new Particle Swarm
Optimization algorithm (PPSO) containing the pluralswarms whose particles had diﬀerent features. The im-- 586 -

10
10
10

0

−20

−40

PSO1
PSO2
PPSO−R
PPSO

−60

−80

0

500

1000 1500 2000
Number of Generaton t

2500

10
10
10
10
10

3000

6

10
PSO1
PSO2
PPSO−R
PPSO

5

4

3

2

1

0

500

(a)

1000 1500 2000
Number of Generation t

2500

10
PSO1
PSO2
PPSO−R
PPSO

10

10

3000

3

Function Value log(f(x))

1010Function Value log(f(x))

1020Function Value log(f(x))

Function Value log(f(x))

10

2

1

0

500

(b)

1000 1500 2000
Number of Generation t

2500

10

10

10

10

3000

1

PSO1
PSO2
PPSO−R
PPSO

0

−1

−2

−3

0

500

(c)

1000 1500 2000
Number of Generation t

2500

3000

(d)

Figure 1: Mean gbest value of every generation for 30-dimensional four functions. (a) Sphere function. (b) Rosenbrock’s
function. (c) Rastrigin’s function. (d) Griewank’s function.

10

10

10

10

Function Value log(f(x))

Function Value log(f(x))

10
0

−5

−10

0

PSO1
PSO2
PPSO−R
PPSO
500

10

10

PSO1
PSO2
PPSO−R
PPSO

8

6

4

4

PSO1
PSO2
PPSO−RPPSO
10Function Value log(f(x))

5Function Value log(f(x))

10

3

10

10

10

1000 1500 2000
Number of Generation t

2500

3000

10

2

0

(a)

500

1000 1500 2000
Number of Generation t

2500

3000

10

(b)

PSO1
PSO2
PPSO−R
PPSO

2

0

−2

2

0

500

1000 1500 2000
Number of Generation t

(c)

2500

3000

0

500

1000 1500 2000
Number of Generation t

2500

3000

(d)

Figure 2: Mean gbest value of every generation for 100-dimensional four functions. (a) Sphere function. (b) Rosenbrock’s
function. (c) Rastrigin’s function. (d) Griewank’s function.

portant features of PPSO were that each particle of PPSO
belonged to one of plural swarms, which had diﬀerent
characteristics, and periodically changed the swarm that it
belongs to. We have performed the basic numerical experiments for various dimensions by using four algorithm
methods of PSOs including PPSO. We have confirmed that
PPSO could obtain the eﬀective results especially for thecomplex problems such as the multimodal function andhigh-dimension.
References[1] J. Kennedy and R. Eberhart, “Particle swarm optimization,” Proc. IEEE Int. Conf. on Neural Networks,
vol. 4, pp. 1942–1948, 1995.
[2] J. Kennedy and R. Medes, “Population structure andparticle swarm performance,” Proc. IEEE Congress
on Evolutionary Computation, vol. 2, pp. 1671–1676,
2002.
[3] A. Ratnaweera, S.K. Halgamuge, and H.C. Watson,“Self-organizing hierarchical particle swarm optimizer
with time-varying acceleration coeﬃcients,” IEEE
Trans. on Evolutionary Computation, vol. 8, no. 3,
pp. 240–255, 2004.

on Evolutionary Computation, vol. 8, no. 3, pp. 225–
239, 2004.
[5] T.J. Richer and T.M. Blackwell, “The Levy particle
swarm,” Proc. IEEE Congress on Evolutionary Computation, pp. 808–815, 2006.
[6] E. Miyagawa and T. Saito, “Particle swarm optimizers
with grow-and-reduce structure,” Proc. IEEE Congress
on Evolutionary Computation, pp. 3974–3979, 2008.
[7] G. Yen and M. Daneshyari, “Diversity-based information exchange among multiple swarm in particle swarm
optimization,” Proc. IEEE Congress on Evolutionary
Computation, pp. 1686–1693, 2006.
[8] M. Iwamatsu, “Multi-species particle swarm optimizer for multimodal function optimization,” IEICE
Trans. on Information and Systems, vol. E89-D, no. 3,
pp. 1181–1187, 2006.
[9] M. Sugimoto, T. Haraguchi, H. Matsushita andY. Nishio, “Particle swarm optimization containing
plural swarms,” Proc. RISP Int. Workshop on Nonlinear Circuits and Signal Processing, pp. 419–
412, 2009.

[4] F.V.d. Bergh and A.P. Engelbrecht, “A cooperative approach to particle swarm optimization,” IEEE Trans.

- 587 -