2011 International Workshop on Nonlinear Circuits,Communication and Signal Processing
NCSP'11, Tianjin, China, March 1-3, 2011

Behavior of Independent-Minded Particle Swarm Optimization
Haruna Matsushita† , Toshimichi Saito‡ and Yoshifumi Nishio§†Kagawa University
2217–20 Hayashi-cho, Takamatsu,
Kagawa 761–0396, Japan
Phone:+81–87–864–2237
haruna@eng.kagawa-u.ac.jp‡Hosei University
3–7–2, Kajino-cho, Koganei,
Tokyo 184–8584, Japan
Phone:+81–423–87–6191
tsaito@hosei.ac.jp

Abstract
This study proposes an improved Independent-mined Particle
Swarm Optimization (IIPSO) algorithm. Particles of IIPSO
have independence, and it is decided stochastically that eachdimension of each particle is affected by gbest or not. IIPSO
is applied to various optimization problem. The effectivenessof IIPSO for multimodal functions is confirmed in terms ofrobustness, accuracy based on evaluation criteria and parameter dependence.
1. Introduction
Particle Swarm Optimization (PSO) [1] is an algorithm tosimulate the movement of flocks of birds. Due to the simple
concept, easy implementation and quick convergence, PSO
has attracted much attention and is used to wide applications
in different fields in recent years. Since each particle flies
toward its personal best position pbest and the best position
among the whole swarm gbest, all the particles of the standard
PSO are fully-connected and always influence each other.
On the other hand, various topological neighborhoods for
PSO have been considered [2]–[5]. In these papers, each particle shares its best position among neighboring particles on
the network. It is an application of the network topology tothe particle swarm, and investigations of the suitable network
for PSO have attracted attention in these years [6], [7].
Our previous study has proposed a novel application of
the complex network to PSO; an Independent-minded Particle Swarm Optimization (IPSO) [8]. The most importantfeature of IPSO is that it is decided stochastically that eachparticle depends on gbest or becomes independent from theswarm and moves depending only on pbest. In other words,
the particles are not always connected each other, and they
act with self-reliance. IPSO was applied to some benchmarks
used widely in the literature, and it has been confirmed thatIPSO is effective for multimodal problems with numerous local optima.
However, the information of all the dimensions of one
benchmark are obtained by same equation. In fact, all the§Tokushima University
2–1 Minami-Josanjima,
Tokushima 770–8506, Japan
Phone:+81–88–656–7470
nishio@ee.tokushima-u.ac.jp

dimensions of the benchmark function have a same feature,and the shape of functions are symmetric in all the dimensions. Meanwhile, many real-world optimization problems
are generally asymmetric and the feature of objective function is different in each dimension. In IPSO, the information of all the dimensions of each particle are simultaneously
updated depending on gbest if the particle is decided to beconnected to the swarm. Although IPSO has improved theperformances on the benchmark problems, we consider that
this updating feature is not effective for the real-world optimization problem.
In this study, we modify the updating rule of IPSO and propose an improved IPSO (IIPSO). The particles of IIPSO have
independence as the conventional IPSO. The difference between IPSO and IIPSO is the number of dimensions of theparticle affected by the swarm. In IIPSO, whether the particle is affected by the swarm is judged for every dimension.
This effect helps to easy escape from the local optima and toapply PSO to the complicated optimization problem whose
each dimension has different feature. We apply IIPSO tovarious benchmark optimization functions and investigate thebehavior of IIPSO by carrying out the simulation using different probability of independence. IIPSO is compared withthe standard PSO and the conventional IPSO. we confirm thatIIPSO can effectively enhance the searching efficiency for themultimodal functions.
2. Improved Independent-minded Particle Swarm Optimizer (IIPSO)
In the algorithm of the PSO, multiple potential solutions called “particles” coexist. Each particle i (i =1, 2, · · · , M ) has two information; position and velocity,represented by X i = (xi1 , · · · , xid , · · · , xiD ) and V i =(vi1 , · · · , vid , · · · , viD ) (d = 1, 2, · · · , D), respectively. At
each time step, each particle flies toward its own past best position (pbest) and the best position among all particles (gbest).
In other words, they always influence each other.
In this study, we propose the improved IPSO (IIPSO) for- 103 -Table 1: Four Test Functions.
Function nameSphere;Rastrigin;
Ackley;Test FunctionDXx2d ,f1 (X) =f2 (X) =f3 (X) =

d=1
DX

¡ 2
¢
xd − 10 cos (2πxd ) + 10 ,

d=1
D−1
X³
d=1

Stretched V;

f4 (X) =D−1
X

20 + e − 20e−0.2√Initialization SpaceCriterionx ∈ [−5.12, 5.12]D ,0.01

x ∈ [−5.12, 5.12]D ,50

x ∈ [−30, 30]D ,1.0x ∈ [−10, 10]D

100.5(x2d +x2d+1 )´−e0.5(cos(2πxd )+cos(2πxd+1 )) ,¡¢(x2d + x2d+1 )0.25 1 + sin2 (50(x2d + x2d+1 )0.1 ) ,d=1

the application to the complicated optimization problem. The
particles of both the conventional IPSO and IIPSO have independence, thus, it is decided stochastically whether they areconnected to others at every step. In other words, they are
not always affected by gbest and their pbest does not alwaysaffect the swarm. The difference between IPSO and IIPSO
is the number of dimensions of the particle affected by theswarm. If the particle of IPSO is decided to be connectedto the swarm, the particle information of all the dimensions
are simultaneously updated depending on gbest. On the other
hand, in IIPSO, whether the particle is affected by the swarm
is judged for every dimension.
(Step1) Let a generation step t = 0. Randomly initialize
the particle position X i (xid ∈ [xmin , xmax ]), initialize its
velocity V i to zero, and initialize P i = (pi1 , pi2 , · · · , piD )with a copy of X i .
(Step2) Decide whether each particle i is connected to theothers according to r3i = (r3i1 , r3i2 , · · · , r3iD ) which is a
D-dimensional random vector (∈ (0, 1)) for i. If any of eachdimension of r3 i satisfies r3 id ≤ C, the information of i istransmitted to other particles. If not, the particle i is isolated
from the swarm, then, it and others does not affect each other.
C is a constant cooperativeness coefficient which is the independence probability of the particles.
(Step3) Evaluate the fitness f (X i ) for each particle i. Update
the personal best position (pbest) as P i = X i if needed.
(Step4) Let P g represents the best position with the best
pbest among particles being connected to others, and it isdenoted by gbest. Update gbest P g = (pg1 , pg2 , . . . , pgD )according to(Step5) Update V i and X i of each particle i according tovid (t + 1) =wvid (t) + c1 r1 (pid − xid (t)) + c2 r2 (pigwvid (t) + c1 r1 (pid − xid (t)),−xid (t)),
r3 id ≤ C
r3 id > C(2)xid (t + 1) = xid (t) + vid (t + 1),(3)

where w is the inertia weight determining how much of theprevious velocity of the particle is preserved. c1 and c2 are
two positive acceleration coefficients, generally c1 = c2 .
r1 and r2 are uniform random number values from U (0, 1).
These equations mean that whether each particle is affectedby gbest is decided at random with the cooperativeness C.
When C = 0, all the dimensions of all the particles move depending only on own pbest, and when C = 1, the algorithm
is completely the same as the standard PSO.
(Step6) Let t = t + 1 and go back to (Step2).
3. Computer Simulation

We apply IIPSO to 4 benchmark optimization problems
summarized in Table 1. f1 is unimodal functions, and f2 –f4
are multimodal functions with numerous local minima. The
optimum solution X ∗ of all the functions are [0, 0, . . . , 0],
and its optimum value f (X ∗ ) is 0. All the functions have Dvariables, in this study, D = 30.
IIPSO is compared with the standard PSO and IPSO. For
all PSOs in all the simulations, the population size M is set to36, and the parameters are set as w = 0.7 and c1 = c2 = 1.6.
g = arg min f (P i ), min{r3 id , d} ≤ C.
(1) In order to investigate the behavior of IPSO and IIPSO, wei
carry out simulations using different cooperativeness C fromIn other words, even if f (P i ) is the minimum pbest among 0 to 1.0. The maximum generation are set to 3000, and theall the particles, gbest is not affected by it if the information results are evaluated in an achievement rate of the criterionof i is not transmitted to others according to (Step2).
attainment over 100 trials.

- 104 -3100PSO
IPSO
IIPSO

f (Pg)f (Pg)

10PSO
IPSO
IIPSO

210−50101Achievement rate [%]10−2−110 −3
1001010Achievement rate [%]−310100PSO
IPSO
IIPSO

500 −310−2−101010
Cooperativeness C

10−2−11010010100PSO
IPSO
IIPSO

500 −310−2−11010
Cooperativeness C(a)010(b)

5210100gf (P )f (Pg)

10PSO
IPSO
IIPSO

−510−10PSO
IPSO
IIPSO

−2−31010−2−11001010100500 −310PSO
IPSO
IIPSO

−2−101010
Cooperativeness C−3−210Achievement rate [%]10Achievement rate [%]01010(c)−11010010100500 −310PSO
IPSO
IIPSO

−210−110
CooperativenessC010(d)Figure 1: Simulation results of three PSOs using different C to four benchmarks. (a) Sphere function. (b) Rastrigin’s function.
(c) Ackley’s function. (d) Stretched V sine wave function.

Table 2: Comparison results of 3 PSOs on 4 test functions.
ff1

f2

Best avg.
CAchievement
Best avg.
CAchievement
Best avg.
CAchievement
Best avg.
CAchievement

PSO
3.37e-50
100%
56.45
37%
57.34
0%
20.32
4%

IPSO
3.37e-50
1.0
100%
37.56
0.6
89%
6.13
0.4
53%
8.44
0.06
71%

IIPSO
3.37e-50
1.0
100%
11.41
0.005
100%
2.85e-08
0.04
100%7.20e-02
0.08
100%

ment rate [%] are listed in Table 2. For the unimodal function
f1 , the fully-connected PSOs with C = 1.0, namely the standard PSO, obtained the best result. In fact, the results became
better as the value of C increased. This is because quick communication is more important for the simple problem than thediversity.

On the other hand, for the multimodal functions as f2 –f3 ,IPSO and IIPSO obtained more effective results than when it
was fully-connected (C = 1.0). In addition, IIPSO signifif3
cantly improved performances of not only the standard PSO
but also IPSO although IPSO improved the performance ofthe standard PSO. These results mean that the particle diverf4
sity is more important for the multimodal functions than thequick communication, and the particles of IIPSO is more diverse than IPSO. At the same time, IIPSO obtained perfectachievement rate for all the multimodal problems althoughFigure 1 shows respective mean results and their achieve- not only the standard PSO but also IPSO was not able to thement rate over 100 runs in different cooperativeness C. Note 100% achievement rate for f2 –f4 . It is the significant imthat the standard PSO used C = 1.0 for all the simulations. proved performances from the standard PSO and IPSO. FurThe best mean result among all C, its value of C and achieve- thermore, IIPSO kept 100% for long range of C. It means that- 105 -runs by changing the cooperativeness C are shown in Fig. 2.
The best mean result among all C are listed in Table 3. Whenthe particles were little affected by gbest, IPSO and IIPSO obtained more effective results than when it was fully-connected(C = 1.0). This result means that we can obtain better performance that the particles value own information and aresometimes affected by others, instead of connecting to all theparticles. Furthermore, it is clear that IIPSO significantly improved the performances from IPSO. In fact, IIPSO obtained
the perfect achievement rate at many values of C although
IPSO has never obtained it even if C was changed.
From these results, we can say that IIPSO easy escape fromthe local optima and provides significantly improved performance in terms of robustness, accuracy based on evaluation
criteria and parameter dependence. Therefore, IIPSO is effective algorithm for the multimodal functions.

5gf (P )10010PSOIPSO
IIPSO−3Achievement rate [%]1010−2−101010100
PSO
IPSO
IPSOM

500 −310−2−1

1010
Cooperativeness C

010Figure 2: Simulation results of combined problem.

Table 3: Comparison results of 3 PSOs on combined problem. 4. Conclusions
ff5

Best avg.
CAchievement

PSO
51.91
14%

IPSO
4.43
0.04
91%

IIPSO
8.57e-03
0.02
100%

the parameter-dependence of IIPSO is weaker than IPSO, thatis, IIPSO is easier to set the parameters than IPSO.
Next, we consider a combined optimization benchmarkfunction. In the standard benchmark function, the feature of
all the dimensions is same because the same equation is used
for all the dimensions, even if it is a high dimension problem.
To avoid this issue, we consider a following function which
is a combination of the plural benchmarks.

This study has proposed the improved Independent-minedParticle Swarm Optimization (IIPSO) algorithm. The particles of IIPSO have independence, and it is decided stochastically that each particle depends on gbest or not. Furthermore, whether the particle is affected by the swarm is judged
for every dimension. We have applied IIPSO to various optimization problem. From the results, we have confirmed thatIIPSO easy escape from the local optima, and for the multimodal functions, IIPSO provides significantly improved performance in terms of robustness, accuracy based on evaluation criteria and parameter dependence.
References[1] J. Kennedy and R. C. Eberhart, “Particle swarm optimization,” in Proc.
of IEEE. Int. Conf. on Neural Netw., pp. 1942–1948, 1995.

Pd1 2 Pd2
Pd3[2] J. Kennedy and R. Mendes, “Population structure and particle swarm
f5 (X) = d=1
xd + d=d1 +1 |xd | + d=ddx4d
performance,” in Proc. of Cong. on Evolut. Comput., pp. 1671–1676,
2 +1
¡ 2¢
Pd42002.
+ d=d3 +1 xd − 10 cos (2πxd ) + 10[3]J.
Lane, A. Engelbrecht and J. Gain, “Particle Swarm Optimization with√
Pd5 −1 ³
Spatially Meaningful Neighbors,” in Proc. of IEEE Swarm Intelligence−0.2 0.5(x2d +x2d+1 )+ d=d4 +1 20 + e − 20eSymposium, pp. 1–8, 2008.
¢[4] S. B. Akat and V. Gazi, “Particle Swarm Optimization with Dynamic−e0.5(cos(2πxd )+cos(2πxd+1 ))Neighborhood Topology: Three Neighborhood Strategies and Prelimi¢¡
Pd6 −120.1
20.25
222nary Results,” in Proc. of IEEE Swarm Intelligence Symposium, pp. 1–8,)
)
.
+x)
1+sin
(50(x+x+ d=d(xd+1dd+1d5 +12008.
(4)[5] H. Matsushita and Y. Nishio, “Network-Structured Particle Swarm OpEq. (4) consists of 6 benchmarks, and different benchmark
timizer with Various Topology and its Behaviors,” in Lecture Notes in
Computer Science, vol. 5629, pp. 163–171, 2009.
is used for each dimension. In particular, Sphere function,rdth
3 De Jong’s function, 4 De Jong’s function, Rastrigin’s [6] J. Kennedy, “Small worlds and mega-minds: effects of neighborhood
topology on particle swarm performance,” in Proc. of Cong. on Evolut.
function, Ackley’s function and Stretched V sine wave funcComput., pp. 1931–1938, 1999.
stthththththtion are used for 1 –d1 , (d1 + 1) –d2 , (d2 + 1) –d3 ,[7]
H.
Matsushita and Y. Nishio, “Network-Structured Particle Swarm Optith
ththth(d3 + 1)th –dthmizer with Small-World Topology,” in Proc. of Int. Symposium on Non4 , (d4 + 1) –d5 and (d5 + 1) –d6 dimenlinear Theory and its Applications, pp. 372–375, 2009.
sions, respectively. The optimum solution X ∗ and its opti∗[8]
H.
Matsushita, Y. Nishio and T. Saito, ”Particle Swarm Optimizationmum value f (X ) are [0, 0, · · · , 0] and 0. In this simulation,with Novel Concept of Complex Network,” in Proc. of Int. Symposium
we set each dimension as d1 = 4, d2 = 8, d3 = 12, d4 = 16,
on Nonlinear Theory and its Applications, pp. 197–200, Sep. 2010.
d5 = 20 and d6 = 24, and the criterion for f5 is set as 10.
The mean results and achievement rate of 3 PSOs over 100- 106 -