# Gradient-Based Cuckoo Search for Global Optimization

Authors: Seif-Eddeen K. Fateen; Adrián Bonilla-Petriciolet
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

One of the major advantages of stochastic global optimization methods is the lack of the need of the gradient of the objective function. However, in some cases, this gradient is readily available and can be used to improve the numerical performance of stochastic optimization methods specially the quality and precision of global optimal solution. In this study, we proposed a gradient-based modification to the cuckoo search algorithm, which is a nature-inspired swarm-based stochastic global optimization method. We introduced the gradient-based cuckoo search (GBCS) and evaluated its performance vis-à-vis the original algorithm in solving twenty-four benchmark functions. The use of GBCS improved reliability and effectiveness of the algorithm in all but four of the tested benchmark problems. GBCS proved to be a strong candidate for solving difficult optimization problems, for which the gradient of the objective function is readily available.


---

## Body

## 1. Introduction

The use of stochastic global optimization methods has gained popularity in a wide variety of scientific and engineering applications as those methods have some advantages over deterministic optimization methods [1]. Those advantages include the lack of the need for a good initial guess and the ability to handle multimodal and nonconvex objective functions without the assumptions of continuity and differentiability. In addition, one of the important advantages is the lack of the need for information about the gradient of the objective function. Gradient-free optimization methods can be either deterministic or stochastic, but their applications can be found in many disciplines [2]. In many applications, however, the gradient of the objective function is already available or easily obtainable. Yet, this valuable piece of information is entirely ignored by traditional stochastic optimization methods. However, for the functions whose gradient is available, the use of the gradient may improve the reliability and efficiency of the stochastic search algorithm. In particular, the quality and precision of solutions obtained with gradient-based optimization methods outperform those obtained with traditional stochastic optimization methods. For a wide variety of engineering applications, it is needed to obtain solutions with a high precision and, therefore, the conventional stochastic optimization methods may fail to satisfy this requirement.Until now, several stochastic methods have been proposed and investigated in challenging optimization problems using continuous variables and they include simulated annealing, genetic algorithms, differential evolution, particle swarm optimization, harmony search, and ant colony optimization. In general, these methods may show different numerical performances and, consequently, the search for more effective and reliable stochastic global optimization methods is currently an active area of research. In particular, the cuckoo search (CS) [3] is a novel nature-inspired stochastic optimization method. This relatively new method is gaining popularity in finding the global minimum of diverse science and engineering application problems [4–8]. For example, it was recently used for the design of integrated power systems [5] and solving reliability-redundancy allocation [6], phase equilibrium [7], and mobile-robot navigation [8] problems. CS was selected to test the concept of using the gradient as a source of new information that guides cuckoos in their search.Therefore, in this study, a simple modification was done to the CS algorithm to make use of the gradient information and enhance the reliability and efficiency of the algorithm. The aim of this work is to present a modification to the existing CS algorithm based on the gradient of the objective function and to evaluate its performance in comparison with the original algorithm. The remainder of this paper is divided as follows. Section2 introduces the cuckoo search algorithm. Section 3 introduces the proposed modification and our new gradient-based cuckoo search (GBCS) algorithm. The numerical experiments performed to evaluate the modification are presented in Section 4. The results of the numerical experiments are presented and discussed in Section 5. Section 6 summarizes the conclusions of the work.


## 2. Cuckoo Search (CS) Algorithm

CS is a nature-inspired stochastic global optimization method that was developed by Yang and Deb [3, 9]. Its concept comes from the brood parasitism behavior of the cuckoo bird. Specifically, brood parasitism is a reproductive strategy followed by cuckoos in which they lay their eggs in the nests of other birds, which are usually other species. If these eggs are discovered by the host bird, it may abandon the nest completely or throw away the alien eggs. This natural phenomenon has led to the evolution of cuckoo eggs to mimic the egg appearance of local host birds. The following rules have been employed in the search algorithm to implement those concepts: (1) one egg is laid by each cuckoo in a random nest and it represents a set of solution coordinates, (2) the best eggs (i.e., solutions) are contained in a fraction of the nests and will carry over to the next generation, and (3) the number of nests is fixed and a host bird can find an alien egg with a specified probability pa∈[0,1]. If this condition occurs, the host bird can discard the egg or abandon the nest, and a new nest will be built elsewhere. For algorithm simplicity, this condition has been implemented in CS assuming that a fraction pa of n nests is replaced by new nests. The pseudocode of CS is reported in Algorithm 1 and details of this metaheuristic are reported in [3]. Note that Lévy flights are used in CS for performing effectively both local and global searches in the solution domain. A Lévy flight is a random walk (i.e., a trajectory that consists of taking successive random steps) and is characterized by a sequence of sudden jumps, which are chosen from a probability density function that has a power law tail. In fact, Lévy flight is considered as the optimum random search pattern and has been useful in stochastic simulations of random natural phenomena including applications of astronomy, physics, and biology. To generate a new egg in CS, a Lévy flight is performed using the coordinates of an egg selected randomly. This step can be represented by
(1)xit+1=xit+α⊕Lévy(λ),
where ⊕ denotes entry-wise multiplication, α is the step size, and Lévy (λ) is the Lévy distribution. Egg is displaced to this new position if the objective function value is found better than another randomly selected egg. The step size α controls the scale of random search and depends on scales of the optimization problems under analysis.Algorithm 1:Simplified algorithm of the cuckoo search algorithm.
begin
Objective functionf(x)
Generate initial population ofn host nests xi
while (t < MaxGeneration)
Get a cuckoo randomly by Lévy flights
Evaluate its quality/fitness
Choose a nest among n (say, j) randomly
If (Fi>Fj)
Replace j by the new solution
end
A fraction (1-pa) of nests are abandoned at random and new ones are built via random walk
Keep the best solutions
Rank the solutions and find the current best
end while
Postprocess results
endA fraction (1−pa) of the nests selected at random is abandoned and replaced by new ones at new locations via local random walks. The local random walk can be written as
(2)xit+1=xit+α(xjt-xkt),
where xjt and xkt are two different solutions selected randomly by random permutation and α is a random number drawn from a uniform distribution. An advantage of CS over genetic algorithm, particle swarm optimization, and other stochastic optimization methods is that there is only one parameter to be tuned, namely, the fraction of nests to be abandoned (1-pa). However, Yang and Deb [3, 9] found that the results obtained for a variety of optimization problems were not so dependent on the value of pa and suggested using pa=0.25.


## 3. Gradient-Based Cuckoo Search (GBCS) Algorithm

The purpose of this section is to introduce the simple modification of the original CS algorithm to incorporate information about the gradient of the objective function. Any modification to algorithm should not change its stochastic nature so as not to negatively affect its performance. A modification was made to the local random walk in which a fraction (1-pa) of the nests are replaced (2). In the original algorithm, when new nests are generated from the replaced nests via a random step, the magnitude and the direction of the step are both random. In the modified algorithm, the randomness of the magnitude of the step is reserved. However, the direction is determined based on the sign of the gradient of the function. If the gradient is negative, the step direction is made positive. If the gradient is positive, the step direction is made negative. Thus, new nests are generated randomly from the worse nests but in the direction of the minimum as seen from the point of view of the old nests. Thus, (2) is replaced by
(3)stepi=α(xjt-xkt),xit+1=xit+stepi⊗sign⁡(-stepidfi),
where sign function obtains the sign of its argument and dfi is the gradient of the objective function at each variable, that is, ∂f/∂xi.This simple modification does not change the structure of the CS algorithm but makes important usage of the available information about the gradient of the objective function. No additional parameter is needed to implement this change.


## 4. Numerical Experiments

Twenty-four classical benchmark functions were used to evaluate the performance of GBCS as compared to the original CS. These problems were chosen from amongst a list of forty-one functions. All forty-one functions were screened first by performing five optimization runs with both CS and GBCS. Functions for which both CS and GBCS performed extremely well with no significant difference in the results were deemed not suitable for comparison and excluded from the evaluation. Since CS is already a high-performance global optimizer, the excluded functions were not suitable for showing the differences between the two algorithms. Table1 lists the twenty-four benchmark functions used for the evaluation of the two algorithms along with their derivatives and the search domain. The number of variables, the number of iterations used, and the value at the global minimum for each problem are shown with the results in Table 2.Table 1
Test functions used for testing the performance of CS and GBCS.
Number
Name
Function and its derivative
Search Domain
1
Ackley
f
1
=
20
(
1
-
e
-
0.2
0.5
(
x
1
2
+
x
2
2
)
)
-
e
0.5
(
cos
⁡
2
π
x
1
+
cos
⁡
2
π
x
2
)
+
e
1 ∂f1∂xk=2xke-0.20.5(x12+x22)0.5(x12+x22)+πe0.5(cos⁡(2πx1)+cos⁡(2πx2))sin⁡⁡(2πxk)
[
-
35,35
]
2
Beale
f
2
=
(
1.5
-
x
1
+
x
1
x
2
)
2
+
(
2.25
-
x
1
+
x
1
x
2
2
)
2
+
(
2.625
-
x
1
+
x
1
x
2
3
)
2 ∂f2∂x1=2(x22-1)(x1x22-x1+2.25)+2(x23-1)(x1x23-x1+2.625)+2(x2-1)(x1x2-x1+1.5) ∂f2∂x2=2x1(x1x2-x1+1.5)+4x1x2(x1x22-x1+2.25)+6x1x22(x1x23-x1+2.625)
[
-
4.5,4.5
]
3
Booth
f
3
=
(
x
1
+
2
x
2
-
7
)
2
+
(
2
x
1
+
x
2
-
5
)
2 ∂f3∂x1=10x1+8x2-34 ∂f3∂x2=8x1+10x2-38
[
-
10,10
]
4
Cross-leg table
f
4
=
-
[
|
sin
⁡
⁡
(
x
1
)
sin
⁡
⁡
(
x
2
)
e
|
100
-
x
1
2
+
x
2
2
/
π
|
|
+
1
]
-
0.1 ∂f4∂x1=0.1σ1(sign⁡[sin⁡⁡(x1)sin⁡⁡(x2)]cos⁡(x1)sin⁡⁡(x2)+x1sign⁡(σ2/π-100)|sin⁡⁡(x1)sin⁡⁡(x2)|/πσ2)(σ1|sin⁡⁡(x1)sin⁡⁡(x2)|+1)1.1 ∂f4∂x2=0.1σ1(sign⁡[sin⁡⁡(x1)sin⁡⁡(x2)]cos⁡(x2)sin⁡⁡(x1)+x2sign⁡(σ2/π-100)|sin⁡⁡(x1)sin⁡⁡(x2)|/πσ2)(σ1|sin⁡⁡(x1)sin⁡⁡(x2)|+1)1.1, whereσ1=e|(σ2/π)-100|,σ2=x12+x22
[
-
10,10
]
5
Himmelblau
f
5
=
(
x
1
2
+
x
2
-
11
)
2
+
(
x
2
2
+
x
1
-
7
)
2 ∂f5∂x1=2x1+4x1(x12+x2-11)+2x22-14,∂f5∂x2=2x2+4x2(x22+x1-7)+2x12-22
[
-
5,5
]
6
Levy 13
f
6
=
sin
⁡
2
⁡
(
3
π
x
1
)
+
(
x
1
-
1
)
2
[
1
+
sin
⁡
2
(
3
π
x
2
)
]
+
(
x
2
-
1
)
2
[
1
+
sin
⁡
2
(
2
π
x
2
)
] ∂f6∂x1=(2x1-2)(sin⁡⁡(3πx2)2+1)+6πcos⁡(3πx1)sin⁡⁡(3πx1) ∂f6∂x2=(2x2-2)(sin⁡⁡(2πx2)2+1)+4πcos⁡(2πx2)sin⁡⁡(2πx2)(x2-1)2+6πcos⁡(3πx2)sin⁡⁡(3πx2)(x1-1)2
[
-
10,10
]
7
Matyas
f
7
=
0.26
x
1
2
-
0.48
x
1
x
2
+
0.26
x
2
2 ∂f7∂x1=0.52x1-0.48x2,∂f7∂x2=0.52x2-0.48x1
[
-
10,10
]
8
Schaffer
f
8
=
0.5
+
sin
⁡
2
⁡
[
x
1
2
+
x
2
2
]
-
0.5
[
0.001
(
x
1
2
+
x
2
2
)
+
1
]
2 ∂f8∂xk=-xk((0.004sin⁡⁡(σ)2-0.002)(0.001(x12+x22)+1)3-2cos⁡(σ)sin⁡⁡(σ)σ(0.001(x12+x22)+1)2), Whereσ=x12+x22
[
-
100,100
]
9
Powell
f
9
=
(
x
1
+
10
x
2
)
2
+
5
(
x
3
-
x
4
)
2
+
(
x
2
-
2
x
3
)
4
+
10
(
x
1
-
x
4
)
4 ∂f9∂x1=2x1+20x2+40(x1-x4)3,∂f9∂x2=20x1+200x2+4(x2-2x3)3 ∂f9∂x3=10x3-10x4-8(x2-2x3)3,∂f9∂x4=10x4-10x3-40(x1-x4)3
[
-
1000,1000
]
10
Power sum
f
10
=
∑
i
=
1
m
[
(
∑
j
=
1
m
x
j
i
)
-
b
i
]
2
,
b
=
[
8,18,42,114
]   ∂f10∂xk=2∑i=1mixki-1[(∑j=1mxji)-bi]
[
0,4
]
11
Shekel 5
f
11
=
-
∑
i
=
1
m
1
c
i
+
∑
j
=
1
n
(
x
j
-
a
i
j
)
2
=
1
,
m
=
10
,
∂
f
11
∂
x
k
=
∑
i
=
1
m
2
(
x
k
-
a
i
k
)
[
c
i
+
∑
j
=
1
n
(
x
j
-
a
i
j
)
2
]
2
[
0,10
]
12
Wood
f
12
=
100
(
x
1
2
-
x
2
)
2
+
(
x
1
-
1
)
2
+
(
x
3
-
1
)
2
+
90
(
x
3
2
-
x
4
)
2
+
10.1
[
(
x
2
-
1
)
2
+
(
x
4
-
1
)
2
]
+
19.8
(
x
2
-
1
)
(
x
4
-
1
) ∂f12∂x1=2x1-400x1(x2-x12)-2,∂f12∂x2=-200x12+220.2x2+19.8x4-40 ∂f12∂x3=2x3-360x3(x4-x32)-2,∂f12∂x4=-180x32+19.8x2+200.2x4-40
[
-
1000,1000
]
13
Cube
f
13
=
∑
i
=
1
m
-
1
100
(
x
i
+
1
-
x
i
3
)
2
+
(
1
-
x
i
)
2
,
∂
f
13
∂
x
1
=
2
x
1
-
600
x
1
2
(
x
2
-
x
1
3
)
-
2 ∂f13∂xk=202xk-600xk2(xk+1-xk3)-200xk-13-2,∂f13∂xm=200(xm-xm-13)
[
-
100,100
]
14
Stochastic Cube
f
14
=
∑
i
=
1
m
-
1
100
ε
i
(
x
i
+
1
-
x
i
3
)
2
+
(
1
-
x
i
)
2
,
∂
f
13
∂
x
1
=
2
x
1
-
600
x
1
2
ε
i
(
x
2
-
x
1
3
)
-
2 ∂f13∂xk=2xk-600εkxk2(xk+1-xk3)+200εi(xk-xk-13)-2,∂f13∂xm=200εm-1(xm-xm-13)
[
-
100,100
]
15
Sphere
f
15
=
∑
i
=
1
m
x
i
2
,
∂
f
15
∂
x
k
=
2
x
k
[
-
100,100
]
16
Hartmann
f
16
=
-
∑
i
=
1
4
α
i
exp
⁡
(
-
∑
j
=
1
6
A
i
j
(
x
j
-
P
i
j
)
2
)
,
∂
f
16
∂
x
k
=
∑
i
=
1
4
2
α
i
A
i
j
(
x
k
-
P
i
k
)
exp
⁡
(
-
∑
j
=
1
6
A
i
j
(
x
j
-
P
i
j
)
2
)
[
0,1
]
17
Dixon-price
f
17
=
(
x
1
-
1
)
2
+
∑
i
=
2
m
i
(
2
x
i
2
-
x
i
-
1
)
2
,
∂
f
17
∂
x
1
=
-
8
x
2
2
+
6
x
1
-
2, ∂f17∂xk=(2xk-4xk+12)(k+1)-8kxk(xk-1-2xk2),∂f17∂xm=-8mxm(xm-1-2xm2)
[
-
10,10
]
18
Griewank
f
18
=
1
4000
[
∑
i
=
1
m
(
x
i
-
100
)
2
]
-
[
∏
i
=
1
m
cos
⁡
(
x
i
-
100
i
)
]
+
1 ∂f18∂xk=(xk-100)2000+1itan⁡(xk-100i)[∏i=1mcos⁡(xi-100i)]
[
-
600,600
]
19
Griewank stochastic
f
19
=
1
4000
[
∑
i
=
1
m
ε
i
(
x
i
-
100
)
2
]
-
[
∏
i
=
1
m
cos
⁡
(
x
i
-
100
i
)
]
+
1 ∂f18∂xk=εk(xk-100)2000+1itan⁡(xk-100i)[∏i=1mcos⁡(xi-100i)]
[
-
600,600
]
20
Michaelwicz
f
20
=
-
∑
i
=
1
m
sin
⁡
⁡
(
x
i
)
sin
⁡
20
⁡
(
i
x
i
2
π
) ∂f20∂xk=-cos⁡(k-x)sin⁡⁡(kxk2π)20-40kxkπcos⁡(kxk2π)sin⁡⁡(kxk2π)19sin⁡⁡(xk)
[
0
,
π
]
21
Rosenbrock
f
21
=
∑
i
=
1
m
-
1
100
(
x
i
+
1
-
x
i
2
)
2
+
(
x
i
-
1
)
2
,
∂
f
21
∂
x
1
=
2
(
x
1
-
1
)
-
400
x
1
(
x
2
-
x
1
2
) ∂f21∂xk=-200xk-12+202xk-400xk(xk+1-xk2)-2,∂f21∂xm=200(xm-xm-12)
[
-
50,50
]
22
StochasticRosenbrock
f
22
=
∑
i
=
1
m
-
1
100
ε
i
(
x
i
+
1
-
x
i
2
)
2
+
(
x
i
-
1
)
2
,
∂
f
22
∂
x
1
=
2
(
x
1
-
1
)
-
400
ε
1
x
1
(
x
2
-
x
1
2
) ∂f22∂xk=2xk+200εk(xk-xk-12)-400εkxk(xk+1-xk2)-2,∂f22∂xm=200εi(xm-xm-12)
[
-
50,50
]
23
Trigonometric
f
23
=
∑
i
=
1
m
[
m
+
i
(
1
-
cos
⁡
x
i
)
-
sin
⁡
⁡
x
i
-
∑
j
=
1
m
cos
⁡
x
j
]
2 ∂f23∂xk=2[m+k(1-cos⁡xi)-sin⁡⁡xi-∑j=1mcos⁡xj][(i+1)sin⁡⁡xk-cos⁡xk]
[
-
1000,1000
]
24
Zacharov
f
24
=
∑
i
=
1
m
x
i
2
+
(
∑
i
=
1
m
0.5
i
x
i
)
2
+
(
∑
i
=
1
m
0.5
i
x
i
)
4
,
∂
f
24
∂
x
k
=
2
x
k
+
k
∑
i
=
1
m
0.5
i
x
i
+
2
k
(
∑
i
=
1
m
0.5
i
x
i
)
3
[
-
5,10
]Table 2
Values of the mean minima and standard deviations obtained by the CS and GBCS algorithms compared with the value of the global minima of the twenty-four benchmark problems.
Number
Benchmark  function
Number of  variables
Globalmin.
Number of iterations
GBCS
CS
Mean
Std. Dev.
Mean
Std. Dev.
1
Ackley
2
0
1000
0
0
2.2204E-16
6.7752E-16
2
Beale
2
0
1000
0
0
5.7891E-30
1.2165E-29
3
Booth
2
0
1000
0
0
0
0
4
Cross-leg table
2
−1
1000
−1.1463E-2
7.672E-3
−6.2704E-3
3.6529E-3
5
Himmelblau
2
0
1000
1.7058E-28
2.836E-28
2.5958E-19
5.3451E-19
6
Levy 13
2
0
1000
1.3498E-31
6.6809E-47
1.3498E-31
6.6809E-47
7
Matyas
2
0
1000
2.7691E-54
4.728E-54
2.0407E-38
5.0616E-38
8
Schaffer
2
0
3000
0
0
7.4015E-18
1.9193E-17
9
Powell
4
0
1000
1.8694E-8
3.5848E-8
1.6296E-13
3.4802E-13
10
Power sum
4
0
1000
1.8328E-4
1.6761E-4
2.5432E-4
1.8167E-4
11
Shekel 5
4
−10.536
200
−10.536
1.6289E-5
−10.536
1.8421E-2
12
Wood
4
0
1000
2.3726
2.2208
0.40838
0.337
13
Cube
5
0
5000
1.2567
0.86542
5.782E-8
2.5596E-7
14
Stochastic cube
5
0
5000
7.7438
6.9815
6.4369
5.0292
15
Sphere
5
0
1000
2.5147E-38
5.1577E-38
1.1371E-21
1.2967E-21
16
Hartmann
6
−3.3224
200
−3.3224
4.3959E-10
−3.3215
6.0711E-4
17
Dixon-price
50
0
5000
4.7094E-2
1.6904E-1
6.6667E-1
2.6103E-6
18
Griewank
50
0
5000
0
0
3.3651E-10
9.4382E-10
19
Stochastic Griewank
50
0
5000
7.2758E-13
2.8579E-12
6.9263
2.0451
20
Michaelwicz
50
5000
−32.263
1.3729
−27.383
1.3551
21
Rosenbrock
50
0
5000
0.97368
0.5885
35.286
3.7012
22
Stochastic Rosenbrock
50
0
5000
58.599
19.122
4894.4
4678.3
23
Trigonometric
50
0
5000
5356.0
4536.0
19435
4033.7
24
Zacharov
50
0
5000
6.5769
1.3288
27.031
5.2748Note that the benchmark functions used include three stochastic test functions. Most deterministic algorithms such as Nelder-Mead downhill simplex method would fail with those stochastic functions [9]. However, the inclusion of those stochastic functions in the present evaluation is to ensure that the modification of the stochastic method not only preserves its stochastic nature and ability to solve stochastic functions but also improves the performance of minimization algorithm. The stochastic functions were developed by turning three deterministic functions (Cube, Rosenbrock, and Griewank) into stochastic test functions by adding a stochastic vector ε drawn from a uniform distribution in ∈[0,1] [9].The twenty-four problems constitute comprehensive testing for the reliability and effectiveness of the suggested modification to the original CS. Eight functions have two variables only and their surface plots are shown in Figure1.Surface plots of the two-variable benchmark functions used in this study: (a) Ackley, (b) Beale, (c) Booth, (d) cross-leg table, (e) Himmelblau, (f) Levy 13, (g) Matyas, and (h) Schaffer functions.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)Each test function was run for thirty times on MATLAB on an HP Pavilion dv6 laptop with Core i7 processor. The function value was recorded at every iteration and the average and standard deviation for the thirty runs were calculated at every iteration. The maximum number of iterations was 1000 for functions with 2 variables and 5000 for functions with 50 variables. The population size for the cuckoo swarm was 100. The only parameter used in the original CS,pa, was kept constant at a value of 0.25. The MATLAB code for all the functions and their derivatives is available online at http://dx.doi.org/10.1155/2014/493740 as supplementary material to this paper.


## 5. Results and Discussion

As stated, each of the numerical experiments was repeated 30 times with different random seeds for GBCS and for the original CS algorithm. The value at each iteration for each trial was recorded. The mean and the standard deviation of the function values were calculated at each iteration. The progress of the mean values is presented in Figures2–4 for each function and a brief discussion of those results follows. The total CPU time for running the two algorithms for 30 times on each problem varied depending on the problem’s number of variables and the number of iterations used. CPU time ranged from 38.27 min for two-variable problems to 91.03 min for fifty-variable problems.Evolution of mean best values for GBCS and the original CS algorithm for (a) Ackley, (b) Beale, (c) Booth, (d) Cross-leg table, (e) Himmelblau, (f) Levy 13, (g) Matyas, and (h) Schaffer functions.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)Evolution of mean best values for GBCS and the original CS algorithm for (a) Powell, (b) Power Sum, (c) Shekel 5, (d) Wood, (e) Cube, (f) Stochastic Cube, (g) Sphere, and (h) Hartmann functions.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)Evolution of mean best values for GBCS and the original CS algorithm for (a) Dixon-Price, (b) Griewank, (c) Stochastic Griewank, (d) Michaelwics, (e) Rosenbrock, (f) Stochastic Rosenbrock, (g) Trigonometric, and (h) Zacherov functions.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)The Ackley function has one minimum only, which was obtained using the two methods, as shown in Figure2(a). However, GBCS is clearly more effective than CS in reaching the minimum at less number of iterations. The improvement in performance was also clear with the Beale function (Figure 2(b)). Beale has one minimum only, which was obtained satisfactory by the two methods. This pattern of behavior was also observed for the booth function; GBCS was significantly more effective than the CS as it reached the global optimum within a tolerance of 10-32 in less than half of the number of iterations of CS, as shown in Figure 2(c). The first three functions are relatively easy to optimize; the global optima were easily obtained.The cross-leg table function is a difficult one to minimize. Its value at the global minimum is −1. Both algorithms have not been able to reach the global minimum. Yet, GBCS performed significantly better than CS, as shown in Figure2(d). On the other hand, both algorithms were able to identify the global minimum of the Himmelblau function. Figure 2(e) shows the evolution of the mean best values. GBCS performed more effectively than CS. Both algorithms were also able to identify the minimum of the Levy 13 function (Figure 2(f)). However, GBCS was significantly more effective than CS. This pattern was repeated with the Matyas function, as shown in Figure 2(g).The Schaffer function is multimodal. Both GBCS and CS failed to converge to the global minimum within the 10−10 tolerance at 1000 iterations. Running both algorithms to 3000 iterations resulted in GBCS reaching the global optimum, while CS not, as shown in Figure 2(f). Schaffer function concludes the 2-variable functions. GBCS performed better than CS in all of them.The Powell function has four variables. The evolution of the mean best values of CS and GBCS, plotted in Figure3(a), showed that performance of CS was better than GBCS, although they were close to the global minimum. Powell is one of the few test functions, for which the use of the gradient did not improve the performance of the CS algorithm.Minor improvements were observed with the power sum and the Shekel 5 functions. Power sum’s global optimum was not obtained by the two algorithms, as shown in Figure3(b), as both algorithms seem to be trapped in a local minimum. For the Shekel 5 function, Figure 3(c), the global optimum was easily optimized, with minor improvement in performance for the GBCS algorithm. The global optimum for the Wood function, which has four variables as well, was not achieved by both algorithms within 1000 iterations, as shown in Figure 3(d). When they were run for 5000, GBCS seemed to be trapped in a local minimum and was not able to reach the global optimum, which was obtained by CS. The Wood function is the only function for which GBCS performance was much worse than that of the original CS algorithm.The performance of both algorithms for the Cube and the Stochastic Cube functions was peculiar. In both cases, GBCS outperformed CS at the early iterations. At larger iterations CS did better. For the two functions, as depicted in Figures3(e) and 3(f), the global minimum was not obtained with 1000 iterations so both problems were run for 5000. CS was able to come close to the global minimum for the Cube function, while GBCS seemed to be trapped in a local minimum. For the Stochastic Cube function, CS did slightly better at larger iterations, but both algorithms failed to reach the global minimum after 5000 iterations.The sphere function has five variables as the Cube function, but it is easier to solve. Both algorithms were able to identify the global optimum, as depicted in Figure3(g), but GBCS considerably outperformed CS in terms of performance efficiency. The Hartmann function, which has 6 variables, was relatively easy to solve for GBCS algorithms as shown in Figure 3(h). GBCS arrived at the global minimum but CS could not do so within the tolerance. GBCS also outperformed CS in terms of performance efficiency in reaching the global optimum of the Hartmann function.Figure4 shows the performance of test functions with 50 variables. These are the most challenging problems due to the large domain space. Figure 4(a) depicts the performance of both algorithms for the Dixon-Price function. Clearly, GBCS performed better than CS in the early iterations. Even though both algorithms were not able to attain the global minimum, the minimum arrived at by GBCS at early iterations is orders-of-magnitude lower than that arrived at by CS.GBCS outperformed CS in both the Griewank (Figure4(b)) and the Stochastic Griewank (Figure 4(c)). For the Griewank function, both algorithms were able to identify the global minimum. However, GBCS arrived at the global minimum at less than the half number of iterations compared to CS. For the Stochastic Griewank, CS was not able to identify the global minimum even after 5000 iterations. The result predicted by CS was more than 10 orders-of-magnitude higher than the result predicted by GBCS.Figure4(d) shows the results for the Michaelwics function. To the best of the authors’ knowledge, the global minimum for the 50-variable Michaelwics is not known since there is no published literature identifying it. GBCS identified a minimum that is lower than that identified by CS as depicted in Figure 4(d).Although both algorithms were not able to identify the global optima of the Rosenbrock and the Stochastic Rosenbrock functions, as depicted in Figures4(e) and 4(f), respectively, the results obtained by GBCS were orders-of-magnitude lower than those obtained by CS. The superiority of the GBCS is clearly demonstrated with those two functions.Figure4(g) shows the results for the Trigonometric function. For this difficult problem, both algorithms were not able to reach the global minimum within 5000 iterations. In this function, nonetheless, as in most cases studied, GBCS achieved a lower result than CS.The last of the 50-variable functions tested was the Zacherov function. The results of Figure4(h) also show the superiority of GBCS in reaching results that are orders-of-magnitude lower than those obtained by the original CS algorithm.Table2 shows a summary of the evaluation results for the twenty-four benchmark problems. GBCS was able to provide better solutions to all challenging problems. GBCS was able to achieve better performance in eighteen problems, equivalent performance in two problems, and worse performance in four out of the twenty-four problems. In one case, the Stochastic Griewank function, the global optimum was successfully obtained by GBCS for this 50-variable problem, while CS failed to find the global optimum even after 5000 iterations. On the other hand, the four functions for which CS performed better were the Powell, Wood, Cube, and Stochastic Cube functions.It is interesting to note that GBCS outperformed CS in handling two of the three stochastic functions tested in this study. This result confirms that despite the use of the gradient as a guide upon which some moves are based, the stochastic nature of the algorithm remained unaffected. In fact, the additional information was used quite subtly. Cuckoos still search for nests randomly but with the help of some intelligence.


## 6. Conclusions

In this study, we use the gradient of the objective function, which could be available or easily obtainable for several objective functions in engineering calculations, to improve the performance of one of the most promising stochastic algorithms, the cuckoo search. The proposed modification was subtly implemented in the algorithm by changing the direction of the local random walk of cuckoos towards the direction of the minimum value from the point of view of the cuckoo’s location. We evaluated this modification by attempting to find the global optimum of twenty-four benchmark functions. The newly developed GBCS algorithm led to improved reliability and effectiveness of the algorithm in all but four of tested benchmark problems, which included three stochastic functions. In some cases, the global minimum could not have been obtained via the original CS algorithm but was obtained via GBCS. Although the improvement in performance was not achieved with the entire set of benchmark problems, GBCS proved to be more reliable and efficient in the majority of the tested problems.



---

*Source: 493740-2014-05-12.xml*