# Grey Wolf Optimizer Based on Powell Local Optimization Method for Clustering Analysis

Authors: Sen Zhang; Yongquan Zhou
Journal: Discrete Dynamics in Nature and Society
 (2015)

---

## Abstract

One heuristic evolutionary algorithm recently proposed is the grey wolf optimizer (GWO), inspired by the leadership hierarchy and hunting mechanism of grey wolves in nature. This paper presents an extended GWO algorithm based on Powell local optimization method, and we call it PGWO. PGWO algorithm significantly improves the original GWO in solving complex optimization problems. Clustering is a popular data analysis and data mining technique. Hence, the PGWO could be applied in solving clustering problems. In this study, first the PGWO algorithm is tested on seven benchmark functions. Second, the PGWO algorithm is used for data clustering on nine data sets. Compared to other state-of-the-art evolutionary algorithms, the results of benchmark and data clustering demonstrate the superior performance of PGWO algorithm.


---

## Body

## 1. Introduction

Swarm Intelligence (SI) has received much attention. Many SI algorithms have been proposed, such as genetic algorithm (GA) [1], particle swarm optimization (PSO) [2], Differential Evolution (DE) [3, 4], and evolutionary programming (EP) [5]. Although these algorithms are capable of solving complex optimization problems, it has been proven by the well-known No Free Lunch (NFL) theorem that there is no optimization solving all optimization problems [6]. So the theorem allows researchers to propose some new algorithms. Some of the recent algorithms are grey wolf optimizer (GWO) [7], Multiverse Optimizer (MVO) [8], Ant Lion Optimizer (ALO) [9], artificial bee colony (ABC) algorithm [10], Firefly algorithm (FA) [11], Cuckoo search (CS) algorithm [12], Cuckoo Optimization Algorithm (COA) [13], gravitational search algorithm (GSA) [14], and Adaptive Gbest-guided Gravitational Search Algorithm (GGSA) [15].In this paper, we concentrate on GWO, developed by Mirjalili et al. [7] in 2014 based on simulating hunting behavior and social leadership of grey wolves in nature. Numerical comparisons showed that the superior performance of GWO is competitive to that of other population-based algorithms. Because it is simple and easy to implement and has fewer control parameters, GWO has caused much attention and has been used to solve a number of practical optimization problems [16–18].However, like other stochastic optimization algorithms, such as PSO and GA, as the growth of the search space dimension, GWO algorithm provides a poor convergence behavior at exploitation [19, 20]. Therefore, it is necessary to emphasize that our work falls in increasing the local search ability of GWO algorithm. According to [21], we can know that many direct search methods are fairly fast optimizers and have a strong ability of local search. Powell’s method [22] is one of the direct search methods. In order to use good convergence property of Powell’s method, we propose a hybrid metaheuristic that is grey wolf optimizer based on Powell local optimization method (PGWO). Compared to other state-of-the-art evolutionary algorithms, PGWO performs significantly better.Cluster analysis or clustering is the task of grouping similar objects or multidimensional data vectors into a number of clusters or groups. Clustering analysis is an unsupervised learning process, without a priori knowledge of clustering, and the clustering algorithm is automatic classification based on the distance or similarity between samples. As a result of this characteristic, clustering techniques have been applied to a wide range of problems, such as data mining [23], data analysis, pattern recognition [24], and image segmentation [25]. The traditional clustering methods can be simply classified as partitioning methods, hierarchical methods, density-based methods, and grid-based methods [26]. In this paper, we concentrate on partitioning methods. Partitioning clustering method divides data vectors into a predefined number of clusters through optimizing some certain criterion. K-means is the most popular partitioning clustering method [27, 28]. However, K-means highly depends on initial states and always falls into local optimum. In order to overcome this problem, many methods have been proposed. For example, K-harmonic mean algorithm has been proposed for clustering instead of K-means in [29]. A simulated annealing- (SA-) based approach has been developed in [30]. A tabu search- (TS-) based method was introduced in [31, 32]. Genetic algorithm- (GA-) based methods were presented in [33–36]. Fathian et al. [37] have developed a clustering algorithm based on honey-bee mating optimization (HBMO). The particle swarm optimization (PSO) is applied for clustering in [38]. Hatamlou et al. employed a big bang-big crunch algorithm for data clustering in [39]. Karaboga and Ozturk presented a novel clustering approach based on artificial bee colony (ABC) algorithm in [40]. Data clustering based on gravitational search algorithm was presented in [41, 42]. In 1991, Colorni et al. have presented ant colony optimization (ACO) algorithm based on the behavior of ants seeking a path between their colony and a source of food. Then, Shelokar and Kao solved the clustering problem using the ACO algorithm [43, 44]. Kao et al. have presented a hybrid approach according to combination of the K-means algorithm, Nelder-Mead simplex search, and PSO for clustering analysis [45]. Niknam et al. have presented a hybrid evolutionary algorithm based on PSO and SA to solve the clustering problem [46]. But every algorithm has some drawbacks; for example, K-means algorithm sucks in local optima, and convergence is highly dependent on initial positions in case of genetic algorithm; in ACO, the solution vector has been affected as the number of iterations increased; Kao et al. [45] and Niknam et al. [46] stated that PSO gives better clustering results, when it is applied in one-dimensional data set and for small data set, but when it is applied to large data set, it does not give the good results, and so forth. In this paper, a PGWO algorithm is used to solve the cluster problem, tested on nine data sets. This algorithm takes the advantage of GWO and Powell method. Initial process is started by GWO, which allows searching all the space for a global solution. When global solution is found, the clustering is switched to Powell for faster convergence to finish the clustering process. As can be seen from the simulation results, this proposed algorithm not only has higher convergence speed but also can find out the optimal solution compared to the other algorithms across the majority of data sets whether in small data set or in large data set.The rest of the paper is organized as follows: Section2 presents a brief introduction to GWO. Section 3 discusses the basic principles of GWO based on Powell local optimization method. The experimental results of test functions and data clustering are shown in Sections 4 and 5, respectively. Finally, Section 6 concludes the work and suggests some directions for future studies.


## 2. Grey Wolf Optimizer (GWO)

The GWO algorithm, proposed by Mirjalili et al. (2014) [7], is inspired by the hunting behavior and social leadership of grey wolves in nature. It is similar to other metaheuristics, and in GWO algorithm the search begins by a population of randomly generated wolves (candidate solutions). In order to formulate the social hierarchy of wolves when designing GWO, in this algorithm the population is split into four groups: alpha (α), beta (β), delta (δ), and omega (ω). Over the course of iterations, the first three best solutions are called α, β, and δ, respectively. The rest of the candidate solutions are named as ω. In this algorithm, the hunting (optimization) is guided by α, β, and δ. The ω wolves are required to encircle α, β, and δ so as to find better solutions. The encircle process could be molded as follows [7]:(1)
D
→
=
C
→
·
X
→
p
t
-
X
→
t
,
X
→
t
+
1
=
X
→
p
t
-
A
→
·
D
→
,where t indicates the current iteration, C
→
=
2
·
r
→
2, A
→
=
2
a
→
·
r
→
1
-
a
→, X
→
p is the position vector of the prey, X
→ indicates the position vector of a grey wolf, α
→ is gradually decreased from 2 to 0, and r
1 and r
2 are random numbers over range [
0,1
].In order to mathematically simulate the hunting behavior of grey wolves, in the GWO algorithm we always assume thatα, β, and δ have better knowledge about the position of the prey (optimum). Therefore, the positions of the first three best solutions (
α
,
β
,
δ
) obtained so far are saved and other wolves (ω) are obliged to reposition with respect to α, β, and δ. The mathematical model of readjusting the positions of ω wolves is presented as follows [7]:(2)
D
→
α
=
C
→
1
·
X
→
α
-
X
→
,
D
→
β
=
C
→
2
·
X
→
β
-
X
→
,
D
→
δ
=
C
→
3
·
X
→
δ
-
X
→
,
(3)
X
→
1
=
X
→
α
-
A
→
1
·
D
→
α
,
X
→
2
=
X
→
β
-
A
→
2
·
D
→
β
,
X
→
3
=
X
→
δ
-
A
→
3
·
D
→
δ
,
(4)
X
→
t
+
1
=
X
→
1
+
X
→
2
+
X
→
3
3
,where X
→
α is the position of the alpha, X
→
β is the position of the beta, X
→
δ is the position of the delta, C
→
1, C
→
2, and C
→
3 and A
→
1, A
→
2, and A
→
3 are all random vectors, X
→ is the position of the current solution, and t indicates the number of iterations.In these formulas, it may also be observed that there are two vectorsA
→ and C
→ obliging the GWO algorithm to explore and exploit the search space. With decreasing A, half of the iterations are devoted to exploration (A
≥
1) and the other half are dedicated to exploitation (A
<
1). The range of C is 2
≤
C
≤
0, and the vector C also improves exploration when C
>
1 and the exploitation is emphasized when C
<
1. Note here that A is decreased linearly over the course of the iterations. In contrast, C is generated randomly whose aim is to emphasize exploration/exploitation at any stage avoiding local optima. The main steps of grey wolf optimizer are given in Algorithm 1.Algorithm 1:GWO algorithm.
Initialize the grey wolf populationX
i
=
(
i
=
1,2
,
…
,
n
) and parameters
Calculate the fitness of population
Find the first three agentsX
α, X
β, X
δ
While(t < Max  number  of  iterations)
Update the position of the current search agent by (4)
Calculate the fitness of population
UpdateX
α, X
β, X
δ
t
=
t
+
1
End while
ReturnX
α


## 3. Grey Wolf Optimizer Based on Powell Local Optimization Method (PGWO)

### 3.1. Local Search Based on Powell’s Method

Powell’s method, strictly Powell’s conjugate direction method, is an algorithm proposed by Powell [22] for finding a local minimum of a function. The function need not be differentiable, and no derivatives are taken. The method successively utilized a bidirectional search approach along each search vector to pursue the minimum of a function. The new position is represented as a linear combination of the search vectors. The new displacement vector as a new search vector is added to the search vector list. Correspondingly, the most successful vector which contributed most to the new direction is deleted from the search vector list. The algorithm is iterated by some run times until no significant improvement is made. The detailed steps of Powell’s method procedure are presented in Algorithm 2 [47].Algorithm 2:Powell’s method.
Initialize the starting pointX
1, independent vectors d
i
=
e
i
(
i
=
1
,
…
,
D
), the tolerance for stop criteria ε, set
f
(
1
)
=
f
(
X
1
), X
c
=
X
1, K
=
1
While(stopping criterion is not met, namely Δ
f
>
ε) do
For
i
=
1toD
do
If
(
K
≥
2
)
then
d
i
=
d
i
+
1
End If
X
i
+
1
=
X
i
+
λ
i
d
i, λ
i is determined by Minimizing f
(
X
i
+
1
)
End For
d
i
+
1
=
∑
1
D
λ
i
∗
d
i
=
X
D
+
1
-
X
D,  X
c
k
+
1
=
X
D
+
1
+
λ
k
d
i
+
1,  f
k
+
1
=
f
X
c
k
+
1
k
=
k
+
1,  X
1
=
X
c
k,  Δ
f
=
f
k
-
f
k
-
1
End While


### 3.2. The Proposed Algorithm

Over the last years, many research results have been published in the field of evolutionary algorithms [7]. And the results show that the hybridizations metaheuristics and local search techniques are exceptionally successful. Successful hybridizations have been proposed, especially in combinatorial and discrete solution spaces [48, 49]. Interestingly, for real-valued solution spaces, few results have been introduced yet—an astonishing fact as many direct search methods are fairly fast optimizers [21]. Therefore, in order to overcome the shortcomings of GWO algorithm including slow convergence speed, easily falling into local optimum value, low computation accuracy, and low success rate of convergence [19, 20], GWO algorithm based on Powell local optimization method is proposed. It adopts the powerful local optimization ability of Powell’s method and embeds it into GWO as a local search operator. In this case, the proposed method has potential to provide superior results compared to other state-of-the-art evolutionary algorithms. The general steps of PGWO are presented as follows, among which p indicates the performing probability of Powell’s method and is set to 0.5. We also have tried to vary the number of grey wolves (population size n) and the performing probability of Powell’s method. From our simulations, we found that n
=
30 and p
=
0.5 are sufficient for most optimization problems. So we will use fixed n
=
30 and p
=
0.5 in the rest of the simulations. In the following section, various benchmark functions are employed to investigate the efficiency of PGWO algorithm (see Algorithm 3).Algorithm 3:PGWO algorithm.
Initialize the grey wolf populationX
i
=
(
i
=
1,2
,
…
,
n
) and parameters
Calculate the fitness of population
Find the first three agentsX
α, X
β, X
δ
While(t < Max  number  of  iterations)
Update the position of the current search agent by (4)
Ifrand
>
p, choose current best solution X
α as a starting point and generate a new solution X
α
′ by Powell’s
method as illustrated Algorithm2. If X
α
′
<
X
α, replace X
α with X
α
′, otherwise, go to Step 6
Calculate the fitness of population
UpdateX
α, X
β, X
δ
t
=
t
+
1
End While
Return X
α


## 3.1. Local Search Based on Powell’s Method

Powell’s method, strictly Powell’s conjugate direction method, is an algorithm proposed by Powell [22] for finding a local minimum of a function. The function need not be differentiable, and no derivatives are taken. The method successively utilized a bidirectional search approach along each search vector to pursue the minimum of a function. The new position is represented as a linear combination of the search vectors. The new displacement vector as a new search vector is added to the search vector list. Correspondingly, the most successful vector which contributed most to the new direction is deleted from the search vector list. The algorithm is iterated by some run times until no significant improvement is made. The detailed steps of Powell’s method procedure are presented in Algorithm 2 [47].Algorithm 2:Powell’s method.
Initialize the starting pointX
1, independent vectors d
i
=
e
i
(
i
=
1
,
…
,
D
), the tolerance for stop criteria ε, set
f
(
1
)
=
f
(
X
1
), X
c
=
X
1, K
=
1
While(stopping criterion is not met, namely Δ
f
>
ε) do
For
i
=
1toD
do
If
(
K
≥
2
)
then
d
i
=
d
i
+
1
End If
X
i
+
1
=
X
i
+
λ
i
d
i, λ
i is determined by Minimizing f
(
X
i
+
1
)
End For
d
i
+
1
=
∑
1
D
λ
i
∗
d
i
=
X
D
+
1
-
X
D,  X
c
k
+
1
=
X
D
+
1
+
λ
k
d
i
+
1,  f
k
+
1
=
f
X
c
k
+
1
k
=
k
+
1,  X
1
=
X
c
k,  Δ
f
=
f
k
-
f
k
-
1
End While


## 3.2. The Proposed Algorithm

Over the last years, many research results have been published in the field of evolutionary algorithms [7]. And the results show that the hybridizations metaheuristics and local search techniques are exceptionally successful. Successful hybridizations have been proposed, especially in combinatorial and discrete solution spaces [48, 49]. Interestingly, for real-valued solution spaces, few results have been introduced yet—an astonishing fact as many direct search methods are fairly fast optimizers [21]. Therefore, in order to overcome the shortcomings of GWO algorithm including slow convergence speed, easily falling into local optimum value, low computation accuracy, and low success rate of convergence [19, 20], GWO algorithm based on Powell local optimization method is proposed. It adopts the powerful local optimization ability of Powell’s method and embeds it into GWO as a local search operator. In this case, the proposed method has potential to provide superior results compared to other state-of-the-art evolutionary algorithms. The general steps of PGWO are presented as follows, among which p indicates the performing probability of Powell’s method and is set to 0.5. We also have tried to vary the number of grey wolves (population size n) and the performing probability of Powell’s method. From our simulations, we found that n
=
30 and p
=
0.5 are sufficient for most optimization problems. So we will use fixed n
=
30 and p
=
0.5 in the rest of the simulations. In the following section, various benchmark functions are employed to investigate the efficiency of PGWO algorithm (see Algorithm 3).Algorithm 3:PGWO algorithm.
Initialize the grey wolf populationX
i
=
(
i
=
1,2
,
…
,
n
) and parameters
Calculate the fitness of population
Find the first three agentsX
α, X
β, X
δ
While(t < Max  number  of  iterations)
Update the position of the current search agent by (4)
Ifrand
>
p, choose current best solution X
α as a starting point and generate a new solution X
α
′ by Powell’s
method as illustrated Algorithm2. If X
α
′
<
X
α, replace X
α with X
α
′, otherwise, go to Step 6
Calculate the fitness of population
UpdateX
α, X
β, X
δ
t
=
t
+
1
End While
Return X
α


## 4. Experimental Results and Discussion

### 4.1. Simulation Platform

All algorithms are tested in Matlab R2012a (7.14) and experiments are executed on AMD Athlon (tm) II X4 640 Processor 3.00 GHz PC with 3 G RAM. Operating system is Windows 7.


### 4.2. Benchmark Functions

In order to evaluate the property of PGWO algorithm, seven standard benchmark functions are employed [50, 51]. These benchmark functions can be divided into two different groups: unimodal and multimodal functions. Table 1 lists the functions, where Dim indicates dimension of the function, Range is the boundary of the function’s search space, and f
m
i
n is the minimum return value of the function, among which the former two functions (
F
1
-
F
2
) are unimodal, (
F
3
–
F
5
) are multimodal functions, and the last two functions (
F
6
-
F
7
) are fixed-dimension multimodal benchmark functions. There are some parameters that should be initialized before running. Table 2 is the initial value of these algorithms.Table 1
Benchmark functions.
Functions
Name
Dim
Range
f
min
⁡
F
1
=
∑
i
=
1
n
x
i
2
Sphere
30
[
-100, 100]
0
F
2
=
∑
i
=
1
n
-
1
100
x
i
+
1
-
x
i
2
2
+
x
i
-
1
2
Rosenbrock
30
[
-30, 30]
0
F
3
=
-
20
exp
⁡
-
0.2
1
n
∑
i
=
1
n
x
i
2
-
exp
⁡
1
n
∑
i
=
1
n
cos
⁡
2
π
x
i
+
20
+
e
Ackley
30
[
-32, 32]
0
F
4
=
π
n
10
sin
⁡
π
y
1
+
∑
i
=
1
n
-
1
y
1
-
1
2
1
+
10
sin
2
⁡
π
y
i
+
1
+
y
n
-
1
2
Penalized
30
[
-50, 50]
0
+
∑
i
=
1
n
u
x
i
,
10,100,4
y
i
=
1
+
x
i
+
1
4
u
(
x
i
,
a
,
k
,
m
)
=
k
x
i
-
a
m
x
i
>
a
0
-
a
<
x
i
<
a
k
-
x
i
-
a
m
x
i
<
-
a
F
5
=
0.1
s
i
n
2
3
π
x
1
+
∑
i
=
1
n
x
i
-
1
2
1
+
s
i
n
2
3
π
x
i
+
1
+
x
n
-
1
2
1
+
s
i
n
2
2
π
x
n
Penalized 2
30
[
-50, 50]
0
+
∑
i
=
1
n
u
(
x
i
,
5,100,4
)
F
6
=
∑
i
=
1
11
a
i
-
x
1
b
i
2
+
b
i
x
2
b
i
2
+
b
i
x
3
+
x
4
2
Kowalik
4
[
-5, 5]
0.0003075
F
7
=
1
+
x
1
+
x
2
+
1
2
19
-
14
x
1
+
3
x
1
2
-
14
x
2
+
6
x
1
x
2
+
3
x
2
2
Goldstein-Price
2
[
-2, 2]
3
×
30
+
2
x
1
-
3
x
2
2
×
18
-
32
x
1
+
12
x
1
2
+
48
x
2
-
36
x
1
x
2
+
27
x
2
2Table 2
Initial parameters of algorithms.
Algorithm
Parameter
Value
PGWO
Population size
30
α
→
Linearly decreased from 2 to 0
Performing probability of Powell  (P)
0.5
Max iteration
500  (benchmark functions test), 200  (data sets test)
Stopping criteria
Max iteration
GWO
Population size
30
α
→
Linearly decreased from 2 to 0
Max iteration
500  (functions test), 200  (data sets test)
Stopping criteria
Max iteration
PSO
Population size
30
C
1
,
C
2
1.4962, 1.4962
W
0.7298
Max iteration
500  (benchmark functions test), 200  (data sets test)
Stopping criteria
Max iteration
ABC
Population size
30
Limit
10
Max iteration
500  (benchmark functions test), 200  (data sets test)
Stopping criteria
Max iteration
CS
Population size
30
p
a
0.25
Max iteration
500  (benchmark functions test), 200  (data sets test)
Stopping criteria
Max iteration
GGSA
Population size
30
c
1
′
(
-
2
t
3
/
T
3
)
+
2
c
2
′
(
2
t
3
/
T
3
)
G
0
1
α
20
Max iteration
500  (benchmark functions test), 200  (data sets test)
Stopping criteria
Max iteration
T indicates the maximum number of interactions; t is the current iteration.The experimental results are presented in Table3. The results are averaged over 20 independent runs, and bold results mean that PGWO is better, while underlined results mean that the other algorithm is better. The Best, Worst, Mean, and Std. represent the optimal fitness value, worst fitness value, mean fitness value, and standard deviation, respectively. Note that the Matlab code of the GGSA algorithm is given in http://www.alimirjalili.com/Projects.html.Table 3
Results comparison of different optimal algorithms for 20 independent runs.
Function
Criteria
GGSA
CS
ABC
PSO
GWO
PGWO
F
1
Best
1.0133E − 17
3.7450E + 00
2.1820E + 01
1.7964E − 06
4.7447E − 29
2.5422
E
-
54
Worst
1.0181E − 16
2.4776E + 01
1.8094E + 02
2.2907E − 04
2.2227E − 26
7.9888
E
-
53
Mean
5.1858E − 17
8.8292E + 00
8.1908E + 01
6.5181E − 05
1.8269E − 27
1.6276
E
-
53
Std.
2.6953E − 17
4.7633E + 00
4.3834E + 01
6.9891E − 05
4.8884E − 27
1.7834
E
-
53
F
2
Best
2.3974E + 01
3.0920E + 02
4.4296E + 03
2.4432E + 01
2.6182E + 01
4.9374
E
+
00
Worst
3.4411E + 02
2.2219E + 03
4.5866E + 04
4.1217E + 02
2.8769E + 01
1.5466
E
+
01
Mean
7.7481E + 01
5.5445E + 02
1.5193E + 04
1.2465E + 02
2.7507E + 01
1.1692
E
+
01
Std.
7.4751E + 01
4.1835E + 02
9.7239E + 03
1.4004E + 02
6.7488
E
-
01
_
2.0844
E
+
00
F
3
Best
2.3591E − 09
4.0632E + 00
9.2131E + 00
6.3889E − 03
7.9048E − 14
4.4409
E
-
15
Worst
9.8704E − 09
1.0603E + 01
1.6954E + 01
4.9535E + 00
1.5010E − 13
4.4409
E
-
15
Mean
5.0646E − 09
6.0677E + 00
1.3090E + 01
2.0423E + 00
1.0498E − 13
4.4409
E
-
15
Std.
1.9095E − 09
1.5421E + 00
2.2882E + 00
1.1464E + 00
1.7485E − 14
0
F
4
Best
5.4733E − 01
2.0863E + 00
7.5357E − 01
2.4473E − 03
1.8626E − 02
1.6432
E
-
32
Worst
4.6556E + 00
4.1099E + 00
5.5219E + 00
3.6426E + 00
1.2149E − 01
1.6432
E
-
32
Mean
1.9728E + 00
3.2711E + 00
2.0544E + 00
8.5739E − 01
5.4300E − 02
1.6432
E
-
32
Std.
1.0488E + 00
5.9739E − 01
1.3358E + 00
1.0326E + 00
2.9934E − 02
2.8080
E
-
48
F
5
Best
2.1024E − 02
2.7015E + 00
2.2507E + 00
6.3080E − 06
3.9444E − 01
1.3498
E
-
32
Worst
2.4387E + 01
1.2516E + 01
1.1090E + 02
3.6461E + 00
8.5465E − 01
1.9625
E
-
01
Mean
7.7881E + 00
6.2644E + 00
1.1124E + 01
3.7285E − 01
6.7323E − 01
4.5918
E
-
02
Std.
6.6733E + 00
2.5624E + 00
2.3792E + 01
9.3480E − 01
1.4330E − 01
6.0835
E
-
02
F
6
Best
1.2358E − 03
3.1550E − 04
8.3768E − 04
3.0749
E
-
04
_
3.0750E − 04
3.0752
E
-
04
Worst
1.2219E − 02
6.0087
E
-
04
_
3.4810E − 03
1.9361E − 03
2.0363E − 02
1.5943
E
-
03
Mean
4.3558E − 03
4.3266
E
-
04
_
1.6421E − 03
9.3309E − 04
6.4067E − 03
4.5499
E
-
04
Std.
2.6046E − 03
8.5397
E
-
05
_
7.2141E − 04
4.1733E − 04
9.3762E − 03
3.4265
E
-
04
F
7
Best
3.0000E + 00
3.0000E + 00
3.0000E + 00
3.0000E + 00
3.0000E + 00
3.0000
E
+
00
Worst
3.0000E + 00
3.0000E + 00
3.0620E + 00
3.0000E + 00
8.4000E + 01
3.0000
E
+
00
Mean
3.0000E + 00
3.0000E + 00
3.0136E + 00
3.0000E + 00
1.1100E + 01
3.0000
E
+
00
Std.
2.5794E − 15
1.5717E − 15
1.9505E − 02
1.2310
E
-
15
_
2.4931E + 01
9.2458
E
-
07To improve the performance evaluation of evolutionary algorithms, statistical tests should be conducted [52]. In order to determine whether the results of PGWO differ from the best results of the other algorithms in a statistical method, a nonparametric test which is known as Wilcoxon’s rank-sum test [53, 54] is performed at 5% significance level. The p values calculated in Wilcoxon’s rank-sum test comparing PGWO and the other algorithms over all the benchmark functions are given in Table 4. According to [52], p values < 0.05 can be considered as sufficient evidence against the null hypothesis.Table 4
p values produced by Wilcoxon’s rank-sum test comparing PGWO versus GWO, PSO, ABC, CS, and GGSA   over benchmark functions (p
≥
0.05 have been underlined).
PGWO vs
GWO
PSO
ABC
CS
GGSA
F
1
6.80E − 08
6.80E − 08
6.80E − 08
6.80E − 08
6.80E − 08
F
2
6.80E − 08
6.80E − 08
6.80E − 08
6.80E − 08
6.80E − 08
F
3
7.66E − 09
8.01E − 09
8.01E − 09
8.01E − 09
8.01E − 09
F
4
8.01E − 09
8.01E − 09
8.01E − 09
8.01E − 09
8.01E − 09
F
5
5.97E − 08
0.295606
5.97E − 08
5.97E − 08
4.67E − 07
F
6
0.003057
0.000375
1.38E − 06
0.001782
1.06E − 07
F
7
0.0002
4.93E − 08
6.80E − 08
6.19E − 08
6.49E − 08


### 4.3. Comparison of Experiment Results

As shown in Table3, PGWO has the best result for F
1. For F
2, PGWO provided better results than the other algorithms across Best, Worst, and Mean, while the Std. of GWO is better than PGWO. The unimodal benchmark functions have only one global solution without any local optima so they are very suitable to examine exploitation. Hence, these results indicate that the proposed method provides greatly improved exploitation compared to the other algorithms.However, it should be noticed that multimodal benchmark functions have many local minima. The final results are more important because these functions can reflect the ability of the algorithm escaping from poor local optima and obtaining the global optimum. We have tested the experiments onF
3
–
F
5. Seen from Table 3, the PGWO provides the best results.ForF
6
-
F
7, these are fixed-dimension multimodal benchmark functions with only a few local minima; the dimensions of the functions are also small. In this case, it is hard to judge the property of the algorithms. The major difference compared with functions F
3
–
F
5 is that functions F
6
-
F
7 appear to be simpler than F
3
–
F
5 due to their low dimension and a smaller number of local minima. For F
6, CS provided better results than the other algorithms across Mean, Std., and Worst. For F
7, the majority of the algorithms can find the optimal solution, but the PSO is more stable than the other algorithms in terms of Std.Thep values of Wilcoxon’s rank-sum in Table 4 show that the results of PGWO in F
5 are not significantly better than PSO. However, the PGWO achieves significant improvement in all remaining benchmark functions compared to the other algorithms. Therefore, this is evidence that the proposed algorithm has high performance in dealing with unimodal, multimodal, and fixed-dimension multimodal benchmark functions.The convergence curves of six algorithms are illustrated in Figure1. As can be seen in these figures, PGWO has a much better convergence rate than the other algorithms on all benchmark functions.Figure 1
The convergence curves of the average fitness value over 20 independent runs.According to this comprehensive comparative study and discussions, these results show that the proposed algorithm is able to significantly improve the performance of GWO and overcome its major shortcomings. For that reason, in the next section, we apply PGWO algorithm to solve clustering problem.


## 4.1. Simulation Platform

All algorithms are tested in Matlab R2012a (7.14) and experiments are executed on AMD Athlon (tm) II X4 640 Processor 3.00 GHz PC with 3 G RAM. Operating system is Windows 7.


## 4.2. Benchmark Functions

In order to evaluate the property of PGWO algorithm, seven standard benchmark functions are employed [50, 51]. These benchmark functions can be divided into two different groups: unimodal and multimodal functions. Table 1 lists the functions, where Dim indicates dimension of the function, Range is the boundary of the function’s search space, and f
m
i
n is the minimum return value of the function, among which the former two functions (
F
1
-
F
2
) are unimodal, (
F
3
–
F
5
) are multimodal functions, and the last two functions (
F
6
-
F
7
) are fixed-dimension multimodal benchmark functions. There are some parameters that should be initialized before running. Table 2 is the initial value of these algorithms.Table 1
Benchmark functions.
Functions
Name
Dim
Range
f
min
⁡
F
1
=
∑
i
=
1
n
x
i
2
Sphere
30
[
-100, 100]
0
F
2
=
∑
i
=
1
n
-
1
100
x
i
+
1
-
x
i
2
2
+
x
i
-
1
2
Rosenbrock
30
[
-30, 30]
0
F
3
=
-
20
exp
⁡
-
0.2
1
n
∑
i
=
1
n
x
i
2
-
exp
⁡
1
n
∑
i
=
1
n
cos
⁡
2
π
x
i
+
20
+
e
Ackley
30
[
-32, 32]
0
F
4
=
π
n
10
sin
⁡
π
y
1
+
∑
i
=
1
n
-
1
y
1
-
1
2
1
+
10
sin
2
⁡
π
y
i
+
1
+
y
n
-
1
2
Penalized
30
[
-50, 50]
0
+
∑
i
=
1
n
u
x
i
,
10,100,4
y
i
=
1
+
x
i
+
1
4
u
(
x
i
,
a
,
k
,
m
)
=
k
x
i
-
a
m
x
i
>
a
0
-
a
<
x
i
<
a
k
-
x
i
-
a
m
x
i
<
-
a
F
5
=
0.1
s
i
n
2
3
π
x
1
+
∑
i
=
1
n
x
i
-
1
2
1
+
s
i
n
2
3
π
x
i
+
1
+
x
n
-
1
2
1
+
s
i
n
2
2
π
x
n
Penalized 2
30
[
-50, 50]
0
+
∑
i
=
1
n
u
(
x
i
,
5,100,4
)
F
6
=
∑
i
=
1
11
a
i
-
x
1
b
i
2
+
b
i
x
2
b
i
2
+
b
i
x
3
+
x
4
2
Kowalik
4
[
-5, 5]
0.0003075
F
7
=
1
+
x
1
+
x
2
+
1
2
19
-
14
x
1
+
3
x
1
2
-
14
x
2
+
6
x
1
x
2
+
3
x
2
2
Goldstein-Price
2
[
-2, 2]
3
×
30
+
2
x
1
-
3
x
2
2
×
18
-
32
x
1
+
12
x
1
2
+
48
x
2
-
36
x
1
x
2
+
27
x
2
2Table 2
Initial parameters of algorithms.
Algorithm
Parameter
Value
PGWO
Population size
30
α
→
Linearly decreased from 2 to 0
Performing probability of Powell  (P)
0.5
Max iteration
500  (benchmark functions test), 200  (data sets test)
Stopping criteria
Max iteration
GWO
Population size
30
α
→
Linearly decreased from 2 to 0
Max iteration
500  (functions test), 200  (data sets test)
Stopping criteria
Max iteration
PSO
Population size
30
C
1
,
C
2
1.4962, 1.4962
W
0.7298
Max iteration
500  (benchmark functions test), 200  (data sets test)
Stopping criteria
Max iteration
ABC
Population size
30
Limit
10
Max iteration
500  (benchmark functions test), 200  (data sets test)
Stopping criteria
Max iteration
CS
Population size
30
p
a
0.25
Max iteration
500  (benchmark functions test), 200  (data sets test)
Stopping criteria
Max iteration
GGSA
Population size
30
c
1
′
(
-
2
t
3
/
T
3
)
+
2
c
2
′
(
2
t
3
/
T
3
)
G
0
1
α
20
Max iteration
500  (benchmark functions test), 200  (data sets test)
Stopping criteria
Max iteration
T indicates the maximum number of interactions; t is the current iteration.The experimental results are presented in Table3. The results are averaged over 20 independent runs, and bold results mean that PGWO is better, while underlined results mean that the other algorithm is better. The Best, Worst, Mean, and Std. represent the optimal fitness value, worst fitness value, mean fitness value, and standard deviation, respectively. Note that the Matlab code of the GGSA algorithm is given in http://www.alimirjalili.com/Projects.html.Table 3
Results comparison of different optimal algorithms for 20 independent runs.
Function
Criteria
GGSA
CS
ABC
PSO
GWO
PGWO
F
1
Best
1.0133E − 17
3.7450E + 00
2.1820E + 01
1.7964E − 06
4.7447E − 29
2.5422
E
-
54
Worst
1.0181E − 16
2.4776E + 01
1.8094E + 02
2.2907E − 04
2.2227E − 26
7.9888
E
-
53
Mean
5.1858E − 17
8.8292E + 00
8.1908E + 01
6.5181E − 05
1.8269E − 27
1.6276
E
-
53
Std.
2.6953E − 17
4.7633E + 00
4.3834E + 01
6.9891E − 05
4.8884E − 27
1.7834
E
-
53
F
2
Best
2.3974E + 01
3.0920E + 02
4.4296E + 03
2.4432E + 01
2.6182E + 01
4.9374
E
+
00
Worst
3.4411E + 02
2.2219E + 03
4.5866E + 04
4.1217E + 02
2.8769E + 01
1.5466
E
+
01
Mean
7.7481E + 01
5.5445E + 02
1.5193E + 04
1.2465E + 02
2.7507E + 01
1.1692
E
+
01
Std.
7.4751E + 01
4.1835E + 02
9.7239E + 03
1.4004E + 02
6.7488
E
-
01
_
2.0844
E
+
00
F
3
Best
2.3591E − 09
4.0632E + 00
9.2131E + 00
6.3889E − 03
7.9048E − 14
4.4409
E
-
15
Worst
9.8704E − 09
1.0603E + 01
1.6954E + 01
4.9535E + 00
1.5010E − 13
4.4409
E
-
15
Mean
5.0646E − 09
6.0677E + 00
1.3090E + 01
2.0423E + 00
1.0498E − 13
4.4409
E
-
15
Std.
1.9095E − 09
1.5421E + 00
2.2882E + 00
1.1464E + 00
1.7485E − 14
0
F
4
Best
5.4733E − 01
2.0863E + 00
7.5357E − 01
2.4473E − 03
1.8626E − 02
1.6432
E
-
32
Worst
4.6556E + 00
4.1099E + 00
5.5219E + 00
3.6426E + 00
1.2149E − 01
1.6432
E
-
32
Mean
1.9728E + 00
3.2711E + 00
2.0544E + 00
8.5739E − 01
5.4300E − 02
1.6432
E
-
32
Std.
1.0488E + 00
5.9739E − 01
1.3358E + 00
1.0326E + 00
2.9934E − 02
2.8080
E
-
48
F
5
Best
2.1024E − 02
2.7015E + 00
2.2507E + 00
6.3080E − 06
3.9444E − 01
1.3498
E
-
32
Worst
2.4387E + 01
1.2516E + 01
1.1090E + 02
3.6461E + 00
8.5465E − 01
1.9625
E
-
01
Mean
7.7881E + 00
6.2644E + 00
1.1124E + 01
3.7285E − 01
6.7323E − 01
4.5918
E
-
02
Std.
6.6733E + 00
2.5624E + 00
2.3792E + 01
9.3480E − 01
1.4330E − 01
6.0835
E
-
02
F
6
Best
1.2358E − 03
3.1550E − 04
8.3768E − 04
3.0749
E
-
04
_
3.0750E − 04
3.0752
E
-
04
Worst
1.2219E − 02
6.0087
E
-
04
_
3.4810E − 03
1.9361E − 03
2.0363E − 02
1.5943
E
-
03
Mean
4.3558E − 03
4.3266
E
-
04
_
1.6421E − 03
9.3309E − 04
6.4067E − 03
4.5499
E
-
04
Std.
2.6046E − 03
8.5397
E
-
05
_
7.2141E − 04
4.1733E − 04
9.3762E − 03
3.4265
E
-
04
F
7
Best
3.0000E + 00
3.0000E + 00
3.0000E + 00
3.0000E + 00
3.0000E + 00
3.0000
E
+
00
Worst
3.0000E + 00
3.0000E + 00
3.0620E + 00
3.0000E + 00
8.4000E + 01
3.0000
E
+
00
Mean
3.0000E + 00
3.0000E + 00
3.0136E + 00
3.0000E + 00
1.1100E + 01
3.0000
E
+
00
Std.
2.5794E − 15
1.5717E − 15
1.9505E − 02
1.2310
E
-
15
_
2.4931E + 01
9.2458
E
-
07To improve the performance evaluation of evolutionary algorithms, statistical tests should be conducted [52]. In order to determine whether the results of PGWO differ from the best results of the other algorithms in a statistical method, a nonparametric test which is known as Wilcoxon’s rank-sum test [53, 54] is performed at 5% significance level. The p values calculated in Wilcoxon’s rank-sum test comparing PGWO and the other algorithms over all the benchmark functions are given in Table 4. According to [52], p values < 0.05 can be considered as sufficient evidence against the null hypothesis.Table 4
p values produced by Wilcoxon’s rank-sum test comparing PGWO versus GWO, PSO, ABC, CS, and GGSA   over benchmark functions (p
≥
0.05 have been underlined).
PGWO vs
GWO
PSO
ABC
CS
GGSA
F
1
6.80E − 08
6.80E − 08
6.80E − 08
6.80E − 08
6.80E − 08
F
2
6.80E − 08
6.80E − 08
6.80E − 08
6.80E − 08
6.80E − 08
F
3
7.66E − 09
8.01E − 09
8.01E − 09
8.01E − 09
8.01E − 09
F
4
8.01E − 09
8.01E − 09
8.01E − 09
8.01E − 09
8.01E − 09
F
5
5.97E − 08
0.295606
5.97E − 08
5.97E − 08
4.67E − 07
F
6
0.003057
0.000375
1.38E − 06
0.001782
1.06E − 07
F
7
0.0002
4.93E − 08
6.80E − 08
6.19E − 08
6.49E − 08


## 4.3. Comparison of Experiment Results

As shown in Table3, PGWO has the best result for F
1. For F
2, PGWO provided better results than the other algorithms across Best, Worst, and Mean, while the Std. of GWO is better than PGWO. The unimodal benchmark functions have only one global solution without any local optima so they are very suitable to examine exploitation. Hence, these results indicate that the proposed method provides greatly improved exploitation compared to the other algorithms.However, it should be noticed that multimodal benchmark functions have many local minima. The final results are more important because these functions can reflect the ability of the algorithm escaping from poor local optima and obtaining the global optimum. We have tested the experiments onF
3
–
F
5. Seen from Table 3, the PGWO provides the best results.ForF
6
-
F
7, these are fixed-dimension multimodal benchmark functions with only a few local minima; the dimensions of the functions are also small. In this case, it is hard to judge the property of the algorithms. The major difference compared with functions F
3
–
F
5 is that functions F
6
-
F
7 appear to be simpler than F
3
–
F
5 due to their low dimension and a smaller number of local minima. For F
6, CS provided better results than the other algorithms across Mean, Std., and Worst. For F
7, the majority of the algorithms can find the optimal solution, but the PSO is more stable than the other algorithms in terms of Std.Thep values of Wilcoxon’s rank-sum in Table 4 show that the results of PGWO in F
5 are not significantly better than PSO. However, the PGWO achieves significant improvement in all remaining benchmark functions compared to the other algorithms. Therefore, this is evidence that the proposed algorithm has high performance in dealing with unimodal, multimodal, and fixed-dimension multimodal benchmark functions.The convergence curves of six algorithms are illustrated in Figure1. As can be seen in these figures, PGWO has a much better convergence rate than the other algorithms on all benchmark functions.Figure 1
The convergence curves of the average fitness value over 20 independent runs.According to this comprehensive comparative study and discussions, these results show that the proposed algorithm is able to significantly improve the performance of GWO and overcome its major shortcomings. For that reason, in the next section, we apply PGWO algorithm to solve clustering problem.


## 5. Data Clustering

### 5.1. The Mathematical Description of Data Clustering

Clustering sample is set toX
=
X
i
,
i
=
1,2
,
…
,
n, where X
i is a p dimensional vector. Clustering problem is to find a partition C
=
C
1
,
C
2
,
…
,
C
m, satisfying the following conditions [26]:(1)
X
=
⋃
i
=
1
m
C
i.
(2)
C
i
≠
ϕ
,
i
=
1,2
,
…
,
m.
(3)
C
i
∩
C
j
=
ϕ
,
i
,
j
=
1,2
,
…
,
m
,
i
≠
j.


### 5.2. The Clustering Criterion

Clustering is the process of grouping a set of data objects into a number of clusters or groups. The aim of clustering is to make the data within a cluster have a high similarity while being very dissimilar to objects in other clusters. Dissimilarities and similarities are evaluated based on the attribute of data sets, containing distance metric. The most popular distance metric is Euclidean distance [55]. Let i
=
(
x
i
1
,
x
i
2
,
…
,
x
i
p
) and j
=
(
x
j
1
,
x
j
2
,
…
,
x
j
p
) be two objects described by p numeric attributes; the Euclidean distance between object i and j is presented as follows:(5)
d
i
,
j
=
x
i
1
-
x
j
1
2
+
x
i
2
-
x
j
2
2
+
⋯
+
x
i
p
-
x
j
p
2
.For given N objects, the clustering problem is to minimize the sum squared Euclidean distance between objects and allocate each object to one of k cluster centers [55]. The clustering aims at finding clusters center through minimizing the objective function. The objective function is defined as follows [26]:(6)
J
c
=
∑
k
=
1
m
∑
X
i
∈
C
k
d
X
i
,
Z
k
,where m indicates the number of clusters, C
k indicates the kth cluster, Z
k indicates thekth center of the clustering, and d
(
X
i
,
Z
k
) indicates the distance of the sample to the corresponding center of clustering; namely, d
(
X
i
,
Z
k
)
=
X
i
-
Z
k.


### 5.3. PGWO Algorithm on Clustering

In clustering analysis, each element in the data set is ap dimensional vector. Moreover, the actual position of a grey wolf represents the k cluster centers, so each grey wolf indicates a k
∗
p dimensional vector. For each grey wolf i, its position is denoted as a vector X
i
=
(
x
i
,
1
,
x
i
,
2
,
…
,
x
i
,
k
∗
p
). In the initialization phase, we utilize maximum and minimum values of each component of the data set (which is to be grouped) as PGWO algorithm the initialization search scope of the grey wolves, and the initialization solution is randomly generated in this range. We use (6) to calculate the fitness function of grey wolves’ individuals and the main steps of the fitness function are shown in Algorithm 4.Algorithm 4:Main steps of the fitness function.
(1) For data vector x
i
(2) Calculate the Euclidean distance by (5)
(3) Assign x
i to the closest cluster center
(4) Calculate the measure function by (6)
(5) End For
(6) Return value of the fitness function


### 5.4. Data Clustering Experimental Results and Discussion

In order to verify performance of the proposed PGWO approach for clustering, we compare the results of theK-means, GGSA, CS, ABC, PSO, GWO, and PGWO clustering algorithms using nine different data sets that are selected from the UCI machine learning repository [56].Artificial data set (N
=
600, d
=
2, and k
=
4) is a two-featured problem with four unique classes. A total of 600 patterns were drawn from four independent bivariate normal distributions, where classes were distributed according to(7)
N
2
μ
=
m
i
0
,
Σ
=
0.5
0.05
0.05
0.5
,where i
=
1,2
,
3,4    m
1
=
-
3, m
2
=
0, m
3
=
3, and m
=
6. μ and Σ are mean vector and covariance matrix, respectively [45, 57].Iris data (N
=
150, d
=
4, and K
=
3) is a data set with 150 random samples of flowers from theIris speciessetosa,versicolor, andvirginica collected by Anderson [58]. From each species there are 50 observations for sepal length, sepal width, petal length, and petal width in cm. This data set was used by Fisher [59] in his initiation of the linear-discriminant-function technique [28, 56, 57].Wisconsin breast cancer (N
=
683, d
=
9, and K
=
2) consists of 683 objects characterized by nine features: clump thickness, cell size uniformity, cell shape uniformity, marginal adhesion, single epithelial cell size, bare nuclei, bland chromatin, normal nucleoli, and mitoses. There are two categories in the data: malignant (444 objects) and benign (239 objects) [28, 56, 57].Contraceptive method choice (N
=
1473, d
=
10, K
=
3), CMC for short, is a data set that is a subset of the 1987 National Indonesia Contraceptive Prevalence Survey. The samples are married women who either were not pregnant or did not know if they were at the time of interview. The problem is to predict the current contraceptive method choice (no use, long-term methods, or short-term methods) of a woman based on her demographic and socioeconomic characteristics [28, 56, 57].Seeds data (N
=
210, d
=
7, and K
=
3) is a data set that consists of 210 patterns belonging to three different varieties of wheat: Kama, Rosa, and Canadian. From each species, there are 70 observations for area A, perimeter P, compactness C (C
=
4
∗
p
i
∗
A
/
p
∧
2), length of kernel, width of kernel, asymmetry coefficient, and length of kernel groove [56].Statlog (Heart) data (N
=
270, d
=
13, K
=
2) is a data set that is a heart disease database similar to a database already present in the repository (heart disease databases) but in a slightly different form [56].Wine data (N
=
178, d
=
13, K
=
3) is also taken from MCI laboratory. These data are the results of a chemical analysis of wines grown in the same region in Italy but derived from three different cultivars. The analysis determined the quantities of 13 constituents found in each of the three types of wines. There are 178 instances with 13 numeric attributes in wine data set. All attributes are continuous. There is no missing attribute value [28, 56, 57].Balance scale data (N
=
625, d
=
4, and K
=
3) is a data set that was generated to model psychological experimental results. Each example is classified as having the balance scale tip to the right, or left being balanced. The attributes are the left weight, the left distance, the right weight, and the right distance. The correct way to find the class is the greater of (left-distance ∗ left-weight) and (right-distance ∗ right-weight). If they are equal, it is balanced [56].Haberman’s Survival (N
=
306, d
=
3, and K
=
2) is a data set that contains cases from a study that was conducted between 1958 and 1970 at the University of Chicago’s Billings Hospital on the survival of patients who had undergone surgery for breast cancer. It records two survival status patients with the age of patient at time of operation, patient’s year of operation, and number of positive axillary nodes detected [56].The results of comparison of intracluster distances for the seven clustering algorithms over 20 independent runs are shown in Table5. Table 6 reports the p values produced by Wilcoxon’s rank-sum test comparing PGWO and the other algorithms over all the data sets.Table 5
Comparison of intracluster distances for the seven clustering algorithms over 20 independent runs.
Data set
Criteria
K-means
GGSA
CS
ABC
PSO
GWO
PGWO
Art
Best
5.1454E + 02
5.1390E + 02
5.1391E + 02
5.1752E + 02
5.1390E + 02
5.1455E + 02
5.1390
E
+
02
Worst
9.0474E + 02
5.1390E + 02
5.3213E + 02
5.5298E + 02
8.9248E + 02
5.2058E + 02
5.1390
E
+
02
Mean
6.0974E + 02
5.1390E + 02
5.1597E + 02
5.2620E + 02
5.3283E + 02
5.1687E + 02
5.1390
E
+
02
Std.
1.6903E + 02
1.5430
E
-
13
_
4.8536E + 00
9.7988E + 00
8.4652E + 01
1.8768E + 00
6.1243
E
-
08
Iris
Best
9.7190E + 01
9.6655E + 01
9.6659E + 01
9.7616E + 01
9.6655E + 01
9.6835E + 01
9.6655
E
+
01
Worst
1.2318E + 02
1.1839E + 02
9.7684E + 01
1.0401E + 02
1.2767E + 02
1.2108E + 02
9.6655
E
+
01
Mean
1.0226E + 02
9.8865E + 01
9.6862E + 01
9.9710E + 01
1.0596E + 02
9.9903E + 01
9.6655
E
+
01
Std.
1.0290E + 01
5.5454E + 00
3.1103E − 01
1.8460E + 00
1.4578E + 01
6.7482E + 00
8.5869
E
-
10
Cancer
Best
2.9763E + 03
2.9644E + 03
2.9645E + 03
2.9887E + 03
2.9644E + 03
2.9645E + 03
2.9644
E
+
03
Worst
2.9884E + 03
3.0611E + 03
2.9677E + 03
3.2650E + 03
4.7288E + 03
2.9650E + 03
2.9644
E
+
03
Mean
2.9830E + 03
2.9715E + 03
2.9651E + 03
3.0760E + 03
3.4055E + 03
2.9647E + 03
2.9644
E
+
03
Std.
4.8278E + 00
2.1284E + 01
7.3222E − 01
6.5283E + 01
7.8385E + 02
1.1452E − 01
1.6118
E
-
08
CMC
Best
5.7016E + 03
5.6938E + 03
5.7054E + 03
5.8219E + 03
5.6942E + 03
5.8236E + 03
5.6937
E
+
03
Worst
5.7053E + 03
5.7111E + 03
5.7762E + 03
6.3743E + 03
7.1580E + 03
6.0911E + 03
5.6937
E
+
03
Mean
5.7040E + 03
5.6971E + 03
5.7263E + 03
6.0868E + 03
5.7690E + 03
5.9239E + 03
5.6937
E
+
03
Std.
1.1022E + 00
3.7872E + 00
2.1119E + 01
1.7292E + 02
3.2694E + 02
9.0635E + 01
9.8777
E
-
04
Seeds
Best
3.1322E + 02
3.1180E + 02
3.1187E + 02
3.1573E + 02
3.1180E + 02
3.1323E + 02
3.1180
E
+
02
Worst
3.1373E + 02
3.1354E + 02
3.1746E + 02
3.4997E + 02
4.2035E + 02
3.1782E + 02
3.1180
E
+
02
Mean
3.1337E + 02
3.1189E + 02
3.1347E + 02
3.3161E + 02
3.2813E + 02
3.1475E + 02
3.1180
E
+
02
Std.
2.3686E − 01
3.8862E − 01
1.6383E + 00
1.0943E + 01
3.9697E + 01
1.5716E + 00
1.7815
E
-
09
Heart
Best
1.0682E + 04
1.0749E + 04
1.0623E + 04
1.0683E + 04
1.0623E + 04
1.0637E + 04
1.0623
E
+
04
Worst
1.0701E + 04
1.2684E + 04
1.0625E + 04
1.1622E + 04
1.0626E + 04
1.0683E + 04
1.0624
E
+
04
Mean
1.0693E + 04
1.1542E + 04
1.0624E + 04
1.0920E + 04
1.0624E + 04
1.0657E + 04
1.0623
E
+
04
Std.
8.1672E + 00
5.7044E + 02
4.6239E − 01
2.5921E + 02
7.6866E − 01
1.3817E + 01
2.3235
E
-
01
Wine
Best
1.6385E + 04
1.6493E + 04
1.6296E + 04
1.6566E + 04
1.6294E + 04
1.6316E + 04
1.6292
E
+
04
Worst
1.8437E + 04
2.0245E + 04
1.6311E + 04
1.7668E + 04
1.6312E + 04
1.6371E + 04
1.6294
E
+
04
Mean
1.6974E + 04
1.7999E + 04
1.6301E + 04
1.7010E + 04
1.6297E + 04
1.6345E + 04
1.6293
E
+
04
Std.
8.6757E + 02
1.1562E + 03
4.5677E + 00
3.6824E + 02
4.0149E + 00
1.4836E + 01
5.2338
E
-
01
Balance scale
Best
1.4239E + 03
1.4238E + 03
1.4256E + 03
1.4265E + 03
1.4238E + 03
1.4239E + 03
1.4238
E
+
03
Worst
1.4337E + 03
1.4291E + 03
1.4285E + 03
1.4310E + 03
1.4262E + 03
1.4260E + 03
1.4257
E
+
03
Mean
1.4275E + 03
1.4259E + 03
1.4268E + 03
1.4282E + 03
1.4248E + 03
1.4243
E
+
03
_
1.4245
E
+
03
Std.
3.6565E + 00
1.6015E + 00
7.3239E − 01
1.1831E + 00
9.7396E − 01
6.8756
E
-
01
_
8.6619
E
-
01
Haberman-Survival
Best
2.6251E + 03
2.5670E + 03
2.5670E + 03
2.5671E + 03
2.5670E + 03
2.5673E + 03
2.5670
E
+
03
Worst
3.1966E + 03
2.6226E + 03
2.5670
E
+
03
_
2.5709E + 03
2.5678E + 03
2.6686E + 03
2.5678
E
+
03
Mean
2.6554E + 03
2.5702E + 03
2.5670
E
+
03
_
2.5679E + 03
2.5671E + 03
2.5898E + 03
2.5673
E
+
03
Std.
1.2740E + 02
1.2354E + 01
1.7590
E
-
06
_
8.3107E − 01
3.0624E − 01
2.3346E + 01
4.0711
E
-
01Table 6
p values produced by Wilcoxon’s rank-sum test comparing PGWO versus GWO, PSO, ABC, CS, GGSA, and K-means over data sets (p
≥
0.05 have been underlined).
PGWO vs
GWO
PSO
ABC
CS
GGSA
K-means
Art
6.80E − 08
1.60E − 05
6.80E − 08
6.80E − 08
6.11E − 08
4.49E − 08
Iris
6.75E − 08
1.19E − 06
6.75E − 08
6.75E − 08
0.107346
5.17E − 08
Cancer
6.76E − 08
6.76E − 08
6.76E − 08
6.76E − 08
1.2E − 06
5.98E − 08
CMC
6.69E − 08
6.69E − 08
6.69E − 08
6.69E − 08
6.69E − 08
3.39E − 08
Seeds
6.80E − 08
6.80E − 08
6.80E − 08
6.80E − 08
1.23E − 03
4.81E − 08
Heart
6.61E − 08
2.02E − 06
6.61E − 08
5.87E − 07
6.61E − 08
6.07E − 08
Wine
6.80E − 08
1.23E − 07
6.80E − 08
6.80E − 08
6.80E − 08
6.63E − 08
Balance scale
0.067868
0.007712
6.8E − 08
1.06E − 07
0.003966
6.01E − 07
Haberman-Survival
6.90E − 07
0.524948
1.78E − 03
0.860418
0.090604
6.26E − 08For Art data set, the optimum value, the worst value, and the average value of PGWO and GGSA are all5.1390
E
+
02, while the standard deviation of GGSA is better than PGWO. For PSO, it only gets the optimum solution 5.1390
E
+
02.For Iris data set, Cancer data set, and Seeds data set, PGWO, PSO, and GGSA provide the optimum value in comparison to those obtained by other methods. However, the worst value, the average value, and the standard deviation value of PGWO are superior to those of the other methods.For Heart data set, PGWO, PSO, and CS all find the optimum solution1.0623
E
+
04. That means they all can find the global solution. However, PGWO is slightly better for the worst value, the average value, and the standard deviation.For CMC data set and Wine data set, Table5 shows that the average, best, worst, and standard deviation values of the fitness function for PGWO algorithm are much smaller than those of the other six methods. The PGWO clustering algorithm is capable of providing the same partition of the data points in all runs.For balance scale data set, the optimum values of the fitness function for PGWO, PSO, and GGSA are1.4238
E
+
03. That means they all can find the global solution. And the optimum values of the fitness function for GWO and K-means are 1.4239
E
+
03; the result is close to PGWO, PSO, and GGSA. However, the standard deviation values of GWO, K-means, PGWO, PSO, GGSA, and CS are 6.8756
E
-
01, 3.6565
E
+
00, 8.6619
E
-
01, 9.7396
E
-
01, 1.6015
E
+
00, and 7.3239
E
-
01, respectively. From the standard deviation, we can see that the GWO is more stable than the other methods.For Haberman’s Survival data set, the optimum value, the worst value, and the average value of the fitness function for CS and PSO are almost the same. The results of CS and PSO algorithms are better than those of the other methods.Thep values in Table 6 show that the results of PGWO are significantly better in Art data set, Cancer data set, CMC data set, Seeds data set, Heart data set, and Wine data set. From Table 6, we conclude that for Iris data set PGWO is performing superior to the other algorithms except for GGSA. Since the p value for balance scale data set of PGWO versus GWO is more than 0.05, there is no statistical difference between them both. For Haberman Survival data set, while comparing PGWO and the other algorithms, we can conclude that PGWO is significantly performing better in three out of six groups compared to algorithms. So, it can be claimed that PGWO provides better results than the other algorithms across the majority of the data sets.Figure2 shows convergence curves of clustering on data sets over 20 independent runs. As can be seen from the figure, the convergence speed of PGWO is the fastest.Figure 2
Convergence curves of clustering on data sets over 20 independent runs.Figure3 indicates ANOVA tests of clustering on data sets over 20 independent runs. Seen from Figure 3, PGWO is very stable for the majority of the data sets. For Art, Seeds, and CMC data sets, PGWO, PSO, and GGSA can obtain the relatively stable optimal values. For Heart and Wine data sets, the stability of PGWO, PSO, and CS is outstanding. For Cancer data set, most of the algorithms can obtain the stable optimal value except for ABC and PSO algorithms. For Iris data set, we can clearly see that PGWO is better in terms of the stability. For balance scale data set, GWO obtains the relatively stable optimal value. For Haberman’s Survival data set, the stability of CS and PSO is the best, but PGWO and GGSA follow them closely.Figure 3
ANOVA tests of clustering on data sets over 20 independent runs.Clustering results of Art, Iris, and Survival data sets by PGWO algorithm are presented in Figure4 which can make it visualized clearly. It can be seen from Figure 4 that the PGWO algorithm possesses superior effect on Art, Iris, and Survival data sets.Figure 4
The original data distribution of Art, Iris, and Survival data sets and the clustering results by PGWO algorithm.In summary, the results show that the proposed method successfully outperforms the other algorithms across the majority of benchmark functions. Furthermore, the test results of clustering problems show that PGWO is able to provide very competitive results, including the property of this algorithm. Therefore, it appears from this comparative study that the proposed method has merit in the field of evolutionary algorithm and optimization.


## 5.1. The Mathematical Description of Data Clustering

Clustering sample is set toX
=
X
i
,
i
=
1,2
,
…
,
n, where X
i is a p dimensional vector. Clustering problem is to find a partition C
=
C
1
,
C
2
,
…
,
C
m, satisfying the following conditions [26]:(1)
X
=
⋃
i
=
1
m
C
i.
(2)
C
i
≠
ϕ
,
i
=
1,2
,
…
,
m.
(3)
C
i
∩
C
j
=
ϕ
,
i
,
j
=
1,2
,
…
,
m
,
i
≠
j.


## 5.2. The Clustering Criterion

Clustering is the process of grouping a set of data objects into a number of clusters or groups. The aim of clustering is to make the data within a cluster have a high similarity while being very dissimilar to objects in other clusters. Dissimilarities and similarities are evaluated based on the attribute of data sets, containing distance metric. The most popular distance metric is Euclidean distance [55]. Let i
=
(
x
i
1
,
x
i
2
,
…
,
x
i
p
) and j
=
(
x
j
1
,
x
j
2
,
…
,
x
j
p
) be two objects described by p numeric attributes; the Euclidean distance between object i and j is presented as follows:(5)
d
i
,
j
=
x
i
1
-
x
j
1
2
+
x
i
2
-
x
j
2
2
+
⋯
+
x
i
p
-
x
j
p
2
.For given N objects, the clustering problem is to minimize the sum squared Euclidean distance between objects and allocate each object to one of k cluster centers [55]. The clustering aims at finding clusters center through minimizing the objective function. The objective function is defined as follows [26]:(6)
J
c
=
∑
k
=
1
m
∑
X
i
∈
C
k
d
X
i
,
Z
k
,where m indicates the number of clusters, C
k indicates the kth cluster, Z
k indicates thekth center of the clustering, and d
(
X
i
,
Z
k
) indicates the distance of the sample to the corresponding center of clustering; namely, d
(
X
i
,
Z
k
)
=
X
i
-
Z
k.


## 5.3. PGWO Algorithm on Clustering

In clustering analysis, each element in the data set is ap dimensional vector. Moreover, the actual position of a grey wolf represents the k cluster centers, so each grey wolf indicates a k
∗
p dimensional vector. For each grey wolf i, its position is denoted as a vector X
i
=
(
x
i
,
1
,
x
i
,
2
,
…
,
x
i
,
k
∗
p
). In the initialization phase, we utilize maximum and minimum values of each component of the data set (which is to be grouped) as PGWO algorithm the initialization search scope of the grey wolves, and the initialization solution is randomly generated in this range. We use (6) to calculate the fitness function of grey wolves’ individuals and the main steps of the fitness function are shown in Algorithm 4.Algorithm 4:Main steps of the fitness function.
(1) For data vector x
i
(2) Calculate the Euclidean distance by (5)
(3) Assign x
i to the closest cluster center
(4) Calculate the measure function by (6)
(5) End For
(6) Return value of the fitness function


## 5.4. Data Clustering Experimental Results and Discussion

In order to verify performance of the proposed PGWO approach for clustering, we compare the results of theK-means, GGSA, CS, ABC, PSO, GWO, and PGWO clustering algorithms using nine different data sets that are selected from the UCI machine learning repository [56].Artificial data set (N
=
600, d
=
2, and k
=
4) is a two-featured problem with four unique classes. A total of 600 patterns were drawn from four independent bivariate normal distributions, where classes were distributed according to(7)
N
2
μ
=
m
i
0
,
Σ
=
0.5
0.05
0.05
0.5
,where i
=
1,2
,
3,4    m
1
=
-
3, m
2
=
0, m
3
=
3, and m
=
6. μ and Σ are mean vector and covariance matrix, respectively [45, 57].Iris data (N
=
150, d
=
4, and K
=
3) is a data set with 150 random samples of flowers from theIris speciessetosa,versicolor, andvirginica collected by Anderson [58]. From each species there are 50 observations for sepal length, sepal width, petal length, and petal width in cm. This data set was used by Fisher [59] in his initiation of the linear-discriminant-function technique [28, 56, 57].Wisconsin breast cancer (N
=
683, d
=
9, and K
=
2) consists of 683 objects characterized by nine features: clump thickness, cell size uniformity, cell shape uniformity, marginal adhesion, single epithelial cell size, bare nuclei, bland chromatin, normal nucleoli, and mitoses. There are two categories in the data: malignant (444 objects) and benign (239 objects) [28, 56, 57].Contraceptive method choice (N
=
1473, d
=
10, K
=
3), CMC for short, is a data set that is a subset of the 1987 National Indonesia Contraceptive Prevalence Survey. The samples are married women who either were not pregnant or did not know if they were at the time of interview. The problem is to predict the current contraceptive method choice (no use, long-term methods, or short-term methods) of a woman based on her demographic and socioeconomic characteristics [28, 56, 57].Seeds data (N
=
210, d
=
7, and K
=
3) is a data set that consists of 210 patterns belonging to three different varieties of wheat: Kama, Rosa, and Canadian. From each species, there are 70 observations for area A, perimeter P, compactness C (C
=
4
∗
p
i
∗
A
/
p
∧
2), length of kernel, width of kernel, asymmetry coefficient, and length of kernel groove [56].Statlog (Heart) data (N
=
270, d
=
13, K
=
2) is a data set that is a heart disease database similar to a database already present in the repository (heart disease databases) but in a slightly different form [56].Wine data (N
=
178, d
=
13, K
=
3) is also taken from MCI laboratory. These data are the results of a chemical analysis of wines grown in the same region in Italy but derived from three different cultivars. The analysis determined the quantities of 13 constituents found in each of the three types of wines. There are 178 instances with 13 numeric attributes in wine data set. All attributes are continuous. There is no missing attribute value [28, 56, 57].Balance scale data (N
=
625, d
=
4, and K
=
3) is a data set that was generated to model psychological experimental results. Each example is classified as having the balance scale tip to the right, or left being balanced. The attributes are the left weight, the left distance, the right weight, and the right distance. The correct way to find the class is the greater of (left-distance ∗ left-weight) and (right-distance ∗ right-weight). If they are equal, it is balanced [56].Haberman’s Survival (N
=
306, d
=
3, and K
=
2) is a data set that contains cases from a study that was conducted between 1958 and 1970 at the University of Chicago’s Billings Hospital on the survival of patients who had undergone surgery for breast cancer. It records two survival status patients with the age of patient at time of operation, patient’s year of operation, and number of positive axillary nodes detected [56].The results of comparison of intracluster distances for the seven clustering algorithms over 20 independent runs are shown in Table5. Table 6 reports the p values produced by Wilcoxon’s rank-sum test comparing PGWO and the other algorithms over all the data sets.Table 5
Comparison of intracluster distances for the seven clustering algorithms over 20 independent runs.
Data set
Criteria
K-means
GGSA
CS
ABC
PSO
GWO
PGWO
Art
Best
5.1454E + 02
5.1390E + 02
5.1391E + 02
5.1752E + 02
5.1390E + 02
5.1455E + 02
5.1390
E
+
02
Worst
9.0474E + 02
5.1390E + 02
5.3213E + 02
5.5298E + 02
8.9248E + 02
5.2058E + 02
5.1390
E
+
02
Mean
6.0974E + 02
5.1390E + 02
5.1597E + 02
5.2620E + 02
5.3283E + 02
5.1687E + 02
5.1390
E
+
02
Std.
1.6903E + 02
1.5430
E
-
13
_
4.8536E + 00
9.7988E + 00
8.4652E + 01
1.8768E + 00
6.1243
E
-
08
Iris
Best
9.7190E + 01
9.6655E + 01
9.6659E + 01
9.7616E + 01
9.6655E + 01
9.6835E + 01
9.6655
E
+
01
Worst
1.2318E + 02
1.1839E + 02
9.7684E + 01
1.0401E + 02
1.2767E + 02
1.2108E + 02
9.6655
E
+
01
Mean
1.0226E + 02
9.8865E + 01
9.6862E + 01
9.9710E + 01
1.0596E + 02
9.9903E + 01
9.6655
E
+
01
Std.
1.0290E + 01
5.5454E + 00
3.1103E − 01
1.8460E + 00
1.4578E + 01
6.7482E + 00
8.5869
E
-
10
Cancer
Best
2.9763E + 03
2.9644E + 03
2.9645E + 03
2.9887E + 03
2.9644E + 03
2.9645E + 03
2.9644
E
+
03
Worst
2.9884E + 03
3.0611E + 03
2.9677E + 03
3.2650E + 03
4.7288E + 03
2.9650E + 03
2.9644
E
+
03
Mean
2.9830E + 03
2.9715E + 03
2.9651E + 03
3.0760E + 03
3.4055E + 03
2.9647E + 03
2.9644
E
+
03
Std.
4.8278E + 00
2.1284E + 01
7.3222E − 01
6.5283E + 01
7.8385E + 02
1.1452E − 01
1.6118
E
-
08
CMC
Best
5.7016E + 03
5.6938E + 03
5.7054E + 03
5.8219E + 03
5.6942E + 03
5.8236E + 03
5.6937
E
+
03
Worst
5.7053E + 03
5.7111E + 03
5.7762E + 03
6.3743E + 03
7.1580E + 03
6.0911E + 03
5.6937
E
+
03
Mean
5.7040E + 03
5.6971E + 03
5.7263E + 03
6.0868E + 03
5.7690E + 03
5.9239E + 03
5.6937
E
+
03
Std.
1.1022E + 00
3.7872E + 00
2.1119E + 01
1.7292E + 02
3.2694E + 02
9.0635E + 01
9.8777
E
-
04
Seeds
Best
3.1322E + 02
3.1180E + 02
3.1187E + 02
3.1573E + 02
3.1180E + 02
3.1323E + 02
3.1180
E
+
02
Worst
3.1373E + 02
3.1354E + 02
3.1746E + 02
3.4997E + 02
4.2035E + 02
3.1782E + 02
3.1180
E
+
02
Mean
3.1337E + 02
3.1189E + 02
3.1347E + 02
3.3161E + 02
3.2813E + 02
3.1475E + 02
3.1180
E
+
02
Std.
2.3686E − 01
3.8862E − 01
1.6383E + 00
1.0943E + 01
3.9697E + 01
1.5716E + 00
1.7815
E
-
09
Heart
Best
1.0682E + 04
1.0749E + 04
1.0623E + 04
1.0683E + 04
1.0623E + 04
1.0637E + 04
1.0623
E
+
04
Worst
1.0701E + 04
1.2684E + 04
1.0625E + 04
1.1622E + 04
1.0626E + 04
1.0683E + 04
1.0624
E
+
04
Mean
1.0693E + 04
1.1542E + 04
1.0624E + 04
1.0920E + 04
1.0624E + 04
1.0657E + 04
1.0623
E
+
04
Std.
8.1672E + 00
5.7044E + 02
4.6239E − 01
2.5921E + 02
7.6866E − 01
1.3817E + 01
2.3235
E
-
01
Wine
Best
1.6385E + 04
1.6493E + 04
1.6296E + 04
1.6566E + 04
1.6294E + 04
1.6316E + 04
1.6292
E
+
04
Worst
1.8437E + 04
2.0245E + 04
1.6311E + 04
1.7668E + 04
1.6312E + 04
1.6371E + 04
1.6294
E
+
04
Mean
1.6974E + 04
1.7999E + 04
1.6301E + 04
1.7010E + 04
1.6297E + 04
1.6345E + 04
1.6293
E
+
04
Std.
8.6757E + 02
1.1562E + 03
4.5677E + 00
3.6824E + 02
4.0149E + 00
1.4836E + 01
5.2338
E
-
01
Balance scale
Best
1.4239E + 03
1.4238E + 03
1.4256E + 03
1.4265E + 03
1.4238E + 03
1.4239E + 03
1.4238
E
+
03
Worst
1.4337E + 03
1.4291E + 03
1.4285E + 03
1.4310E + 03
1.4262E + 03
1.4260E + 03
1.4257
E
+
03
Mean
1.4275E + 03
1.4259E + 03
1.4268E + 03
1.4282E + 03
1.4248E + 03
1.4243
E
+
03
_
1.4245
E
+
03
Std.
3.6565E + 00
1.6015E + 00
7.3239E − 01
1.1831E + 00
9.7396E − 01
6.8756
E
-
01
_
8.6619
E
-
01
Haberman-Survival
Best
2.6251E + 03
2.5670E + 03
2.5670E + 03
2.5671E + 03
2.5670E + 03
2.5673E + 03
2.5670
E
+
03
Worst
3.1966E + 03
2.6226E + 03
2.5670
E
+
03
_
2.5709E + 03
2.5678E + 03
2.6686E + 03
2.5678
E
+
03
Mean
2.6554E + 03
2.5702E + 03
2.5670
E
+
03
_
2.5679E + 03
2.5671E + 03
2.5898E + 03
2.5673
E
+
03
Std.
1.2740E + 02
1.2354E + 01
1.7590
E
-
06
_
8.3107E − 01
3.0624E − 01
2.3346E + 01
4.0711
E
-
01Table 6
p values produced by Wilcoxon’s rank-sum test comparing PGWO versus GWO, PSO, ABC, CS, GGSA, and K-means over data sets (p
≥
0.05 have been underlined).
PGWO vs
GWO
PSO
ABC
CS
GGSA
K-means
Art
6.80E − 08
1.60E − 05
6.80E − 08
6.80E − 08
6.11E − 08
4.49E − 08
Iris
6.75E − 08
1.19E − 06
6.75E − 08
6.75E − 08
0.107346
5.17E − 08
Cancer
6.76E − 08
6.76E − 08
6.76E − 08
6.76E − 08
1.2E − 06
5.98E − 08
CMC
6.69E − 08
6.69E − 08
6.69E − 08
6.69E − 08
6.69E − 08
3.39E − 08
Seeds
6.80E − 08
6.80E − 08
6.80E − 08
6.80E − 08
1.23E − 03
4.81E − 08
Heart
6.61E − 08
2.02E − 06
6.61E − 08
5.87E − 07
6.61E − 08
6.07E − 08
Wine
6.80E − 08
1.23E − 07
6.80E − 08
6.80E − 08
6.80E − 08
6.63E − 08
Balance scale
0.067868
0.007712
6.8E − 08
1.06E − 07
0.003966
6.01E − 07
Haberman-Survival
6.90E − 07
0.524948
1.78E − 03
0.860418
0.090604
6.26E − 08For Art data set, the optimum value, the worst value, and the average value of PGWO and GGSA are all5.1390
E
+
02, while the standard deviation of GGSA is better than PGWO. For PSO, it only gets the optimum solution 5.1390
E
+
02.For Iris data set, Cancer data set, and Seeds data set, PGWO, PSO, and GGSA provide the optimum value in comparison to those obtained by other methods. However, the worst value, the average value, and the standard deviation value of PGWO are superior to those of the other methods.For Heart data set, PGWO, PSO, and CS all find the optimum solution1.0623
E
+
04. That means they all can find the global solution. However, PGWO is slightly better for the worst value, the average value, and the standard deviation.For CMC data set and Wine data set, Table5 shows that the average, best, worst, and standard deviation values of the fitness function for PGWO algorithm are much smaller than those of the other six methods. The PGWO clustering algorithm is capable of providing the same partition of the data points in all runs.For balance scale data set, the optimum values of the fitness function for PGWO, PSO, and GGSA are1.4238
E
+
03. That means they all can find the global solution. And the optimum values of the fitness function for GWO and K-means are 1.4239
E
+
03; the result is close to PGWO, PSO, and GGSA. However, the standard deviation values of GWO, K-means, PGWO, PSO, GGSA, and CS are 6.8756
E
-
01, 3.6565
E
+
00, 8.6619
E
-
01, 9.7396
E
-
01, 1.6015
E
+
00, and 7.3239
E
-
01, respectively. From the standard deviation, we can see that the GWO is more stable than the other methods.For Haberman’s Survival data set, the optimum value, the worst value, and the average value of the fitness function for CS and PSO are almost the same. The results of CS and PSO algorithms are better than those of the other methods.Thep values in Table 6 show that the results of PGWO are significantly better in Art data set, Cancer data set, CMC data set, Seeds data set, Heart data set, and Wine data set. From Table 6, we conclude that for Iris data set PGWO is performing superior to the other algorithms except for GGSA. Since the p value for balance scale data set of PGWO versus GWO is more than 0.05, there is no statistical difference between them both. For Haberman Survival data set, while comparing PGWO and the other algorithms, we can conclude that PGWO is significantly performing better in three out of six groups compared to algorithms. So, it can be claimed that PGWO provides better results than the other algorithms across the majority of the data sets.Figure2 shows convergence curves of clustering on data sets over 20 independent runs. As can be seen from the figure, the convergence speed of PGWO is the fastest.Figure 2
Convergence curves of clustering on data sets over 20 independent runs.Figure3 indicates ANOVA tests of clustering on data sets over 20 independent runs. Seen from Figure 3, PGWO is very stable for the majority of the data sets. For Art, Seeds, and CMC data sets, PGWO, PSO, and GGSA can obtain the relatively stable optimal values. For Heart and Wine data sets, the stability of PGWO, PSO, and CS is outstanding. For Cancer data set, most of the algorithms can obtain the stable optimal value except for ABC and PSO algorithms. For Iris data set, we can clearly see that PGWO is better in terms of the stability. For balance scale data set, GWO obtains the relatively stable optimal value. For Haberman’s Survival data set, the stability of CS and PSO is the best, but PGWO and GGSA follow them closely.Figure 3
ANOVA tests of clustering on data sets over 20 independent runs.Clustering results of Art, Iris, and Survival data sets by PGWO algorithm are presented in Figure4 which can make it visualized clearly. It can be seen from Figure 4 that the PGWO algorithm possesses superior effect on Art, Iris, and Survival data sets.Figure 4
The original data distribution of Art, Iris, and Survival data sets and the clustering results by PGWO algorithm.In summary, the results show that the proposed method successfully outperforms the other algorithms across the majority of benchmark functions. Furthermore, the test results of clustering problems show that PGWO is able to provide very competitive results, including the property of this algorithm. Therefore, it appears from this comparative study that the proposed method has merit in the field of evolutionary algorithm and optimization.


## 6. Conclusion and Future Works

In order to apply the grey wolf optimizer to solve complex optimization problems efficiently, this paper proposed a novel grey wolf optimizer based on Powell local optimization method, namely, PGWO. In PGWO, at first, original GWO algorithm is applied to shrink the search region to a more promising area. Thereafter, Powell’s method is implemented as a critical complement to perform the local search to exploit the limited area intensively to get better solutions. The PGWO makes an attempt at taking merits of the GWO and Powell’s method in order to avoid all grey wolves getting trapped in inferior local optimal regions. The PGWO enables the grey wolves to have more diverse exemplars to learn from as the grey wolves are updated each generation and also form new grey wolves to search in a larger search space. With both techniques combined, PGWO can balance exploration and exploitation and effectively solve complex problems. The experimental results show the effectiveness of Powell’s method in terms of solution quality and convergence speed. The proposed algorithm is benchmarked on seven well-known test functions, and the results are comparative study with GGSA, CS, ABC, PSO, and GWO. The results show that the PGWO algorithm is capable of providing very competitive results compared to these famous metaheuristics. Because of the superior performance of the PGWO algorithm, we use it to solve clustering problems. The algorithm has been tested on an artificial data set and eight real data sets. To justify the performance of the PGWO algorithm on clustering problems, we compare it with the original GWO, GGSA, CS, ABC, PSO, andK-means. The results prove that the PGWO algorithm is able to significantly outperform others on the majority of the data sets in terms of average value and standard deviations of fitness function. Moreover, the experimental results demonstrate that the proposed PGWO algorithm can be considered as a feasible and efficient method to solve optimization problems.Our future work will focus on the two issues. On one hand, we would apply our proposed PGWO algorithm to test higher dimensional problems and large number of patterns. On the other hand, the PGWO clustering algorithm will also be extended to dynamically determine the optimal number of clusters.



---

*Source: 481360-2015-11-02.xml*