# Stochastic Multiple Chaotic Local Search-Incorporated Gradient-Based Optimizer

Authors: Hang Yu; Yu Zhang; Pengxing Cai; Junyan Yi; Sheng Li; Shi Wang
Journal: Discrete Dynamics in Nature and Society
 (2021)

---

## Abstract

In this study, a hybrid metaheuristic algorithm chaotic gradient-based optimizer (CGBO) is proposed. The gradient-based optimizer (GBO) is a novel metaheuristic inspired by Newton’s method which has two search strategies to ensure excellent performance. One is the gradient search rule (GSR), and the other is local escaping operation (LEO). GSR utilizes the gradient method to enhance ability of exploitation and convergence rate, and LEO employs random operators to escape the local optima. It is verified that gradient-based metaheuristic algorithms have obvious shortcomings in exploration. Meanwhile, chaotic local search (CLS) is an efficient search strategy with randomicity and ergodicity, which is usually used to improve global optimization algorithms. Accordingly, we incorporate GBO with CLS to strengthen the ability of exploration and keep high-level population diversity for original GBO. In this study, CGBO is tested with over 30 CEC2017 benchmark functions and a parameter optimization problem of the dendritic neuron model (DNM). Experimental results indicate that CGBO performs better than other state-of-the-art algorithms in terms of effectiveness and robustness.


---

## Body

## 1. Introduction

Metaheuristic algorithms (MHAs) have developed rapidly in the field of computational intelligence [1, 2]. MHAs perform effectively and efficiently for many functional optimization problems or real-world problems [3, 4]. Most of the MHAs drew their inspiration from natural phenomena or mathematical formulas. Population-based MHAs can be applied to a variety of optimization problems in comparison with the limitations of deterministic algorithms. Traditional deterministic algorithms are usually proposed for specific problems and sometimes involve the local characteristics of the objective function. By contrast, MHAs are not limited and can be applied to any optimization problem, and compared with deterministic algorithms, these are more understandable and extensible. The vast majority of experimental data show that MHAs have outstanding performance for functional optimization or practical problems [5, 6].Population-based MHAs can be roughly divided into three categories. Evolutionary algorithms are inspired by the biological evolution of nature. For example, differential evolution (DE) employs crossover and mutation operator to generate better offspring through difference between parents [7]. Genetic algorithm (GA) and evolution strategy (ES) are all common evolutionary algorithms [8–10]. Swarm-based algorithms are also the significant part of MHAs that contain ant colony optimization (ACO) [11], grey wolf optimizer (GWO) [12], particle swarm optimization (PSO) [13], artificial bee colony (ABC) [14], bat algorithm (BA) [15], gravitational search algorithms [16–18], and moth flame optimization (MFO) [19]. In addition to these mainstream algorithms; e.g., a large number of algorithms based on natural phenomena have emerged in recent years, vortex search (VS) was inspired from vortex pattern created by vortical flow of the stirred fluids [20], mine bomb explosion concept is the inspiration of mine blast algorithm (MBA) [21], and gravitational search algorithm (GSA) is based on the law of gravity [22]. Despite novel algorithms being sprung up rapidly and their inspirations being diverse, arbitrary MHAs cannot perform well for all optimization problems on the basis of no free lunch theory. Researchers found that searching characteristics of MHAs can be divided into exploration and exploitation in common [23, 24]. Exploration indicates the ability of finding some potential points in the global search space, and exploitation represents how algorithms find a better neighbouring point of optimum. On this basis, balancing between exploration and exploitation becomes an important way to optimize performance of MHAs [25].Most of MHAs set up a moderate trade-off between exploitation and exploration [2], but for better performance of the algorithm, many studies make decisions to create the hybrid algorithm which incorporated local search operator into original global search [26]. For the past few years, many attempts on the incorporation of local search operators or adaptive strategy into global search have been done and with excellent achievements, in addition to regular single-objective function optimization [27, 28], some of them perform remarkably even for multiple objective function optimization problems [29, 30].Gradient-based optimizer (GBO) is a novel population-based algorithm based on Newton’s method [31] to solve real-world optimization problems in the field of engineering. GBO employs two different strategies to find the optimal value in the whole search space and performs well for function optimization, but the exploration ability is limited to multiple factors which include the fixed parameter and similar operators. Therefore, we attempt to incorporate a random strategy into GBO by some mechanism to improve the performance. It is not hard to see that chaotic local search (CLS) has remarkable capacity to promote performance of many metaheuristics algorithm because of the ergodicity and nonrepetition, e.g., chaotic differential evolution (CJADE) [32], chaotic cuckoo search (CCS) [33], and chaotic whale optimization algorithm (CWOA) [34]. Hence, we attempt to hybridize GBO with CLS through some deterministic mechanism propose a new algorithm termed as CGBO, and testify the feasibility and efficiency of CGBO by 30 CEC2017 benchmark functions and a parameter optimization of dendritic neuron model.The rest of paper is constituted as follows: a detailed introduction of GBO is given in Section2. Chaotic search mechanism and various chaotic maps are depicted in Section 3. Hybrid algorithm CGBO is described in Section 4. Section 5 gives the related introduction of three experiments and experimental results. Finally, the conclusion is placed in Section 6.


## 2. Introduction of Gradient-Based Optimizer

In recent years, MHAs have been developed rapidly as optimization tools that successfully applied to a variety of complicated and real-world optimization problems [35, 36]. GBO was proposed based on Newton’s method and performed well in some engineering problems which included speed reducer problem, three-bar truss problem, I-beam design problem, cantilever beam problem, rolling element bearing design problem, and tension/compression spring design problem. According to theory of Newton’s method, GBO derives two primary operators, gradient search rule (GSR) and local escaping operator (LEO), which utilize these two strategies and searches for the global optimal point in the entire search space based on the populations.


### 2.1. Initialization

An optimization problem can be represented by three parameters: decision variables, objective function values, and constraint conditions. GBO is a population-based metaheuristic algorithm, and each member of population is a decision variable which called individual or vector. The whole population can be depicted as(1)X=X1,X2,X3,…,XNXi=Xi,1,Xi,2,Xi,3,…,Xi,D,i=1,2,3,…,N,where N is the scale of population and D represents the size of decision variables which is also called dimension. Initialization of each dimension of individual employs regular random operator which is formulated as(2)Xi,j=Lbj+rand×Ubj−Lbjj=1,2,3,…,D,where Ubj and Lbj express upper and lower boundary which stem from the constraint condition defined by a specific problem. It is worth mentioning that the boundary of each dimension of engineering problems or other real-world problems is disparate, but the constraint of CEC benchmark functions is uniform. Accordingly, relevant equations are shown in(3)Ub=Ub1,Ub2,Ub3,…,UbD,Lb=Lb1,Lb2,Lb3,…,LbD.Each individualXi corresponds to a target value of function fXi, and our goal is to optimize individuals of the whole population to global optima by operators.


### 2.2. Newton–Raphson Method

Most equations have no root-finding formula, so it is very difficult or even unsolvable to find the exact root. Therefore, it is particularly important to find the approximate root of the equation. The Newton–Raphson method (Newton’s method) is a method proposed by Newton in the 17th century for approximate solution of equations in real and complex fields, which employs the first few terms of Taylor series of functionfx to find roots of the equation. Taylor series of fx can be represented as follows:(4)fx=fx0+f′x0x−x0+f″x0x−x022!+f″′x0x−x033!+⋯+fnx0x−x0nn!,where f′x0, f″x0, f″′x0, and fnx0 represent 1st−, 2nd−, 3rd−, and nth− order derivatives, respectively. Since x−x0 is close to 0, x−x0 to the power of nn>1 approaches to 0, even negligible. After ignoring the remainder of Taylor formula, fx is set to zero, and the following formula can be derived:(5)x1=x0−fx0f′x0,where x1 is the first approximation solution to the root of equation. By repeating n iterations, approximation xn+1 can be depicted as(6)xn+1=xn−fxnf′xn.In other words, Newton’s method continually approximates the roots of equations by means of iteration process and Taylor series. In addition to this, new variant of Newton’s method was proposed, which can be formulated as(7)xn+1=xn−fxnf′zn+1+xn/2,where(8)zn+1=xn−fxnf′xn.Since the iterative process and optimization mechanism of Newton’s iterative method are very consistent with the solution of optimization problem, it is theoretically feasible to apply Newton’s method to optimization algorithm.


### 2.3. Gradient Search Rule

The gradient search rule (GSR) originated from Newton’s method, which can control all individuals in the population and make the whole population close to the global optimal region. The derivation of the GSR formula and the overall mechanism are described in detail. Firstly, functionfx±Δx can be represented as the following equation based on Taylor series:(9)fx±Δx=fx±f′x0Δx+f″x0Δx22!±f″′x0Δx33!+…,where Δx is increment; according to simultaneous equation (9) and ignoring Taylor series above the third order, we can derive the following formula:(10)f′x=fx+Δx−fx−Δx2Δx.From equations (6) and (10), the next point xn+1 can be denoted as(11)xn+1=xn−2Δx×fxnfxn+Δx−fxn−Δx.GBO selects pointsxworst and xbest instead of two neighbouring points xn±Δx in equation (11). For general optimization problem, fx which is a minimization problem and the fitness of xworst is greater than xbest. In addition to this, xn substitutes for fxn because it takes time to compute functions. Therefore, the difference between xn and xn+1 can be formulated as(12)GSR=rand×2Δx×xnxworst−xbest+ε,where rand is a random number which obeys uniform distribution between 0 and 2π, ε is a real number within 0,0.1, and xworst and xbest, respectively, represent the current worst and best point in the population. GSR can lead the candidate point to the better position, but for more superior balance between the exploration and exploitation, GSR will be modified by an adaptive parameter to enhance the exploration ability of GBO. The improved formulation is shown as(13)GSR=rand×ρ1×2Δx×xnxworst−xbest+ε,where(14)ρ1=α×2×rand−1α=β×sin3π2+sinβ×3π2β=βmin+βmax−βmin×1−mM32.βmax and βmin are set to 1.2 and 0.2; m and M, respectively, mean the current iteration number and maximum iteration number. α generally decreases with the increase in the number of iterations, but it is not a regular monotonous increasing function. Δx represents the difference between best individual xbest and a random individual xr1 which be defined by(15)Δx=rand×stepstep=xbest−xr1+σ2σ=2×rand×xr1+xr1+xr3+xr44−xn,where rand is a N-dimensional random vector with a range of 0,1 and xr1, xr2, xr3, and xr4 are different randomly selected individuals. According to the variant of Newton’s method which is proposed by Ozban, the equation of GSR can be modified by(16)GSR=rand×ρ1×2Δx×xnyp−yq+εyp=rand×zn+1+xn2+rand×Δxyq=rand×zn+1+xn2−rand×Δx,where zn+1 is calculated by original GSR formulation equation (13). In addition to GSR, the direction of movement (DM) is created as an added increment to improve the performance of GSR; the process of DM promotes the speed of convergence and the equation is(17)DM=rand×ρ2×xbest−xn,where ρ2 represents a random number which is the same as the formulation of ρ1. According to GSR and DM, the updating operation can be represented as(18)xGSR=r1×r2×x1+1−r2×x2+1−r1×x3,where r1 and r2 are two random real numbers within the range of 0,1 and solutions x1, x2, and x3 are given as(19)x1=xn−GSR+DMx2=xbest−GSR+DMx3=xn−ρ1×x2−x1.The whole searching process of GSR can be shown in Figure1. Due to the limitation of single search operation, a local escaping operator is proposed to improve the performance of the algorithm.Figure 1
The 2-dimensional diagram of gradient search rule.


### 2.4. Local Escaping Operator (LEO)

LEO is an optional operator to change the candidate solutions based on several acquired solutions. LEO generates two different solutions, and the final solutionxnew is randomly selected from these two solutions. The whole process of LEO is defined as follows:(20)if rand<0.5xnew=xGSR+f1×step1+f2×step1elsexnew=xbest+f1×step1+f2×step2,where f1 is a random number between 0 and 1 and f2 is a random number that obeys standard Gaussian distribution. step1 and step2 are step lengths which can be described as(21)step1=u1×xbest−u2×xkstep2=ρ1×u3×x2−x1+u2×xr1−xr22,where u1, u2, and u3 are random number and(22)xk=xrand,ifrand<0.5,xp,otherwise,where xrand is a randomly generated individual with a range of constraint and xp is the randomly selected individual in the population.


## 2.1. Initialization

An optimization problem can be represented by three parameters: decision variables, objective function values, and constraint conditions. GBO is a population-based metaheuristic algorithm, and each member of population is a decision variable which called individual or vector. The whole population can be depicted as(1)X=X1,X2,X3,…,XNXi=Xi,1,Xi,2,Xi,3,…,Xi,D,i=1,2,3,…,N,where N is the scale of population and D represents the size of decision variables which is also called dimension. Initialization of each dimension of individual employs regular random operator which is formulated as(2)Xi,j=Lbj+rand×Ubj−Lbjj=1,2,3,…,D,where Ubj and Lbj express upper and lower boundary which stem from the constraint condition defined by a specific problem. It is worth mentioning that the boundary of each dimension of engineering problems or other real-world problems is disparate, but the constraint of CEC benchmark functions is uniform. Accordingly, relevant equations are shown in(3)Ub=Ub1,Ub2,Ub3,…,UbD,Lb=Lb1,Lb2,Lb3,…,LbD.Each individualXi corresponds to a target value of function fXi, and our goal is to optimize individuals of the whole population to global optima by operators.


## 2.2. Newton–Raphson Method

Most equations have no root-finding formula, so it is very difficult or even unsolvable to find the exact root. Therefore, it is particularly important to find the approximate root of the equation. The Newton–Raphson method (Newton’s method) is a method proposed by Newton in the 17th century for approximate solution of equations in real and complex fields, which employs the first few terms of Taylor series of functionfx to find roots of the equation. Taylor series of fx can be represented as follows:(4)fx=fx0+f′x0x−x0+f″x0x−x022!+f″′x0x−x033!+⋯+fnx0x−x0nn!,where f′x0, f″x0, f″′x0, and fnx0 represent 1st−, 2nd−, 3rd−, and nth− order derivatives, respectively. Since x−x0 is close to 0, x−x0 to the power of nn>1 approaches to 0, even negligible. After ignoring the remainder of Taylor formula, fx is set to zero, and the following formula can be derived:(5)x1=x0−fx0f′x0,where x1 is the first approximation solution to the root of equation. By repeating n iterations, approximation xn+1 can be depicted as(6)xn+1=xn−fxnf′xn.In other words, Newton’s method continually approximates the roots of equations by means of iteration process and Taylor series. In addition to this, new variant of Newton’s method was proposed, which can be formulated as(7)xn+1=xn−fxnf′zn+1+xn/2,where(8)zn+1=xn−fxnf′xn.Since the iterative process and optimization mechanism of Newton’s iterative method are very consistent with the solution of optimization problem, it is theoretically feasible to apply Newton’s method to optimization algorithm.


## 2.3. Gradient Search Rule

The gradient search rule (GSR) originated from Newton’s method, which can control all individuals in the population and make the whole population close to the global optimal region. The derivation of the GSR formula and the overall mechanism are described in detail. Firstly, functionfx±Δx can be represented as the following equation based on Taylor series:(9)fx±Δx=fx±f′x0Δx+f″x0Δx22!±f″′x0Δx33!+…,where Δx is increment; according to simultaneous equation (9) and ignoring Taylor series above the third order, we can derive the following formula:(10)f′x=fx+Δx−fx−Δx2Δx.From equations (6) and (10), the next point xn+1 can be denoted as(11)xn+1=xn−2Δx×fxnfxn+Δx−fxn−Δx.GBO selects pointsxworst and xbest instead of two neighbouring points xn±Δx in equation (11). For general optimization problem, fx which is a minimization problem and the fitness of xworst is greater than xbest. In addition to this, xn substitutes for fxn because it takes time to compute functions. Therefore, the difference between xn and xn+1 can be formulated as(12)GSR=rand×2Δx×xnxworst−xbest+ε,where rand is a random number which obeys uniform distribution between 0 and 2π, ε is a real number within 0,0.1, and xworst and xbest, respectively, represent the current worst and best point in the population. GSR can lead the candidate point to the better position, but for more superior balance between the exploration and exploitation, GSR will be modified by an adaptive parameter to enhance the exploration ability of GBO. The improved formulation is shown as(13)GSR=rand×ρ1×2Δx×xnxworst−xbest+ε,where(14)ρ1=α×2×rand−1α=β×sin3π2+sinβ×3π2β=βmin+βmax−βmin×1−mM32.βmax and βmin are set to 1.2 and 0.2; m and M, respectively, mean the current iteration number and maximum iteration number. α generally decreases with the increase in the number of iterations, but it is not a regular monotonous increasing function. Δx represents the difference between best individual xbest and a random individual xr1 which be defined by(15)Δx=rand×stepstep=xbest−xr1+σ2σ=2×rand×xr1+xr1+xr3+xr44−xn,where rand is a N-dimensional random vector with a range of 0,1 and xr1, xr2, xr3, and xr4 are different randomly selected individuals. According to the variant of Newton’s method which is proposed by Ozban, the equation of GSR can be modified by(16)GSR=rand×ρ1×2Δx×xnyp−yq+εyp=rand×zn+1+xn2+rand×Δxyq=rand×zn+1+xn2−rand×Δx,where zn+1 is calculated by original GSR formulation equation (13). In addition to GSR, the direction of movement (DM) is created as an added increment to improve the performance of GSR; the process of DM promotes the speed of convergence and the equation is(17)DM=rand×ρ2×xbest−xn,where ρ2 represents a random number which is the same as the formulation of ρ1. According to GSR and DM, the updating operation can be represented as(18)xGSR=r1×r2×x1+1−r2×x2+1−r1×x3,where r1 and r2 are two random real numbers within the range of 0,1 and solutions x1, x2, and x3 are given as(19)x1=xn−GSR+DMx2=xbest−GSR+DMx3=xn−ρ1×x2−x1.The whole searching process of GSR can be shown in Figure1. Due to the limitation of single search operation, a local escaping operator is proposed to improve the performance of the algorithm.Figure 1
The 2-dimensional diagram of gradient search rule.


## 2.4. Local Escaping Operator (LEO)

LEO is an optional operator to change the candidate solutions based on several acquired solutions. LEO generates two different solutions, and the final solutionxnew is randomly selected from these two solutions. The whole process of LEO is defined as follows:(20)if rand<0.5xnew=xGSR+f1×step1+f2×step1elsexnew=xbest+f1×step1+f2×step2,where f1 is a random number between 0 and 1 and f2 is a random number that obeys standard Gaussian distribution. step1 and step2 are step lengths which can be described as(21)step1=u1×xbest−u2×xkstep2=ρ1×u3×x2−x1+u2×xr1−xr22,where u1, u2, and u3 are random number and(22)xk=xrand,ifrand<0.5,xp,otherwise,where xrand is a randomly generated individual with a range of constraint and xp is the randomly selected individual in the population.


## 3. Chaotic Search Mechanism

Chaos refers to the unpredictable motion due to its sensitivity to initial values in the deterministic dynamical system. Chaos is a seemingly irregular and complex motion pattern in the real world [37]. Its characteristic is that the original orderly motion pattern following simple physical laws deviates from the expected regularity and turns into a disordered form under certain conditions. Chaos theory has been successfully applied to many fields which include evolutionary algorithms [38, 39]. Since most of the population-based optimization algorithms contain the random operators, chaos can be utilized to replace the general random operator to optimize the global search algorithm due to its ergodicity and randomicity.


### 3.1. Chaotic Map

Chaotic maps are mainly adopted to generate chaotic sequences to adjust parameters during the population initialization and iterative process. It has a good effect in many bionic optimization algorithms, such as genetic algorithm [40], artificial immune system algorithm [41], and differential evolution algorithm. Chaotic maps are mostly employed in the adjustment of crossover, mutation, and selection operators. Here, we choose twelve common chaotic maps to establish the chaotic search mechanism:(1)
Logistic map: It is also called unimodal mapping, which is a quadratic polynomial mapping. Logistic map is invariably used as a typical example to explain that many complex chaotic phenomena stem from a simple nonlinear dynamics equation. The formulation of logistic map is expressed as(23)xt+1=μxt1−xt,wherext is the chaos in t iteration and the map functions are all the same. For μ=3.98, x0∈0,1 and chaotic sequence xt∈0,1. It is worth noting that x0∉0.25,0.5,0.75 to keep the chaotic system from being disabled; here, x0 is set to be 0.7.(2)
Piecewise linear chaotic map (PWLCM): PWLCM has attracted much attention because of its simple representation and excellent dynamic characteristics. PWLCM is ergodic and has a uniformly invariant density function between the defined interval. The original mathematical formula is represented as(24)xt+1=xtp,xt∈0,p,1−xt1−p,xt∈p,1,wherep=0.7, x0=0.4, and xt∈0,1.(3)
Chebyshev map: the Chebyshev map originated from the Chebyshev polynomial which is inspired by expansion of cosine and sine of multiple angles [42]; the relative formula is given as(25)xt+1=cosϕcosxt−1,whereϕ=8, xt∈0,1, and the initial value x0 is set to 0.7; if negative output appears, we take its absolute value.(4)
Circle map: The circle map can be thought of as a one-dimensional map that simplifies (due to dissipation) the more general map from a plane to itself [43]. The equation of circle map can be described as(26)xt+1=xt+b−moda2πsin2πxt,1a=2.6,b=0.5.Chaotic sequence isxt∈0,1, and initial value is also set as x0=0.7.(5)
Cubic map: It is one of the simplest polynomial maps and the corresponding formula is defined as(27)xt+1=μxt1−xt2,whereμ=2.59 and initial value x0=0.7.(6)
Sine map: the input of a sine function is set to0,π, and its corresponding output is between 0 and 1. The sine map is obtained by converting the original range of input to 0,1. It is formulated as follows:(28)xt+1=4asinπxta∈0,4.Fora=4, xt∈0,1 and x0=0.7.(7)
Singer map: singer map is a one-dimensional map function which can be represented as(29)xt+1=μ7.86xt−23.31xt2+28.75xt3−13.303xt4μ∈0.9,1.08.In our experiments,μ=1.071 and x0=0.7. The value of xt is between 0 and 1.(8)
Gaussian map [44]: Gaussian map is a one-dimensional nonlinear map based on an exponential function. The formula of Gaussian map is formulated as(30)xt+1=0,xt=0,modμxt,1,xt≠0.Forμ=1, x0=0.52 and chaotic sequence xt∈0,1.(9)
Sinusoidal map [45]: this map generates chaotic sequences based on sine function, which is determinedly formulated as(31)xt+1=axt2sinπxt.It is worth to note that these equations can be simplified to the following formula whena=2.3 and x0=0.7:(32)xt+1=sinπxt.Here, we seta=2.3, x0=0.242, and xt∈0,1.(10)
Tent map: tent map has a special chaos mechanism which is akin to logistic map, and the whole mapping process can be expressed as(33)xt+1=xtβ,xt∈0,β,1−xt1−β,xt∈β,1,whereβ=0.4, xt∈0,1, and x0=0.7.(11)
Iterative chaotic map with infinite collapses (ICMICs) [46]: this map function generates chaotic signals with infinite collapses in a determinate region. The formula is shown as(34)xt+1=sinaxta∈0,∞,wherea is set to 15. There are negative numbers in the chaotic sequence generated by ICMIC, so we will call an absolute value operator when generating chaos.(12)
Bernoulli map: Bernoulli map is a weak-mixing piecewise function with ergodic properties. It can be depicted by(35)xt+1=xt1−λ,xt∈0,β,1−xt1−β,xt∈β,1.Here,λ=0.4 and x0=0.7.


### 3.2. Chaotic Local Search

According to these twelve chaotic mapping formulas, we can obtain a series of chaotic sequence to replace pseudo-random numbers with a range of0,1. Figure 2 can exhibit distribution status of numbers for twelve chaotic maps and each chaotic map has its own characteristic. From it, it is obvious that the singer map easily generates chaotic sequence with a range of 0.7,1 and the circle map is easier to generate output of a number between 0.3,0.7. Due to these distinct numerical peculiarities of chaotic maps, we utilize a chaotic local search strategy to improve original GBO algorithm.Figure 2
Histograms of distribution for twelve different chaotic maps.Recently, the local search strategy has been successfully employed to improve the global single-objective and multiobjective optimization algorithms [47, 48], and the local search strategy is usually used to jump out of the local optimal point when global algorithm is trapped into stagnation, such as well-known tabu search [49, 50] and simulated annealing algorithm [51, 52]. The CLS strategy was recently emerged as an efficient local search method, which is utilized to optimize many state-of-the-art global optimization algorithms. The CLS strategy takes advantage of chaotic sequence with a range of 0,1 for adjusting current individuals. The adjusting method can be formulated as(36)xnew=xold+radius×ub−lb×chaos−z,where xold is the selected individual to be optimized, xnew is the optimized individual by CLS, radius is the scale parameter to adjust the influence of chaotic sequence between 0,0.01, and ub and lb denote the upper limit and lower limit defined by the specific optimization problems, respectively. chaos is the number which is obtained by the chaotic map, and z is the parameter to change the range of chaos between −0.5,0.5. The whole procedure of CLS can be shown in Figure 3.Figure 3
Chaotic local search strategy for optimization problems.


## 3.1. Chaotic Map

Chaotic maps are mainly adopted to generate chaotic sequences to adjust parameters during the population initialization and iterative process. It has a good effect in many bionic optimization algorithms, such as genetic algorithm [40], artificial immune system algorithm [41], and differential evolution algorithm. Chaotic maps are mostly employed in the adjustment of crossover, mutation, and selection operators. Here, we choose twelve common chaotic maps to establish the chaotic search mechanism:(1)
Logistic map: It is also called unimodal mapping, which is a quadratic polynomial mapping. Logistic map is invariably used as a typical example to explain that many complex chaotic phenomena stem from a simple nonlinear dynamics equation. The formulation of logistic map is expressed as(23)xt+1=μxt1−xt,wherext is the chaos in t iteration and the map functions are all the same. For μ=3.98, x0∈0,1 and chaotic sequence xt∈0,1. It is worth noting that x0∉0.25,0.5,0.75 to keep the chaotic system from being disabled; here, x0 is set to be 0.7.(2)
Piecewise linear chaotic map (PWLCM): PWLCM has attracted much attention because of its simple representation and excellent dynamic characteristics. PWLCM is ergodic and has a uniformly invariant density function between the defined interval. The original mathematical formula is represented as(24)xt+1=xtp,xt∈0,p,1−xt1−p,xt∈p,1,wherep=0.7, x0=0.4, and xt∈0,1.(3)
Chebyshev map: the Chebyshev map originated from the Chebyshev polynomial which is inspired by expansion of cosine and sine of multiple angles [42]; the relative formula is given as(25)xt+1=cosϕcosxt−1,whereϕ=8, xt∈0,1, and the initial value x0 is set to 0.7; if negative output appears, we take its absolute value.(4)
Circle map: The circle map can be thought of as a one-dimensional map that simplifies (due to dissipation) the more general map from a plane to itself [43]. The equation of circle map can be described as(26)xt+1=xt+b−moda2πsin2πxt,1a=2.6,b=0.5.Chaotic sequence isxt∈0,1, and initial value is also set as x0=0.7.(5)
Cubic map: It is one of the simplest polynomial maps and the corresponding formula is defined as(27)xt+1=μxt1−xt2,whereμ=2.59 and initial value x0=0.7.(6)
Sine map: the input of a sine function is set to0,π, and its corresponding output is between 0 and 1. The sine map is obtained by converting the original range of input to 0,1. It is formulated as follows:(28)xt+1=4asinπxta∈0,4.Fora=4, xt∈0,1 and x0=0.7.(7)
Singer map: singer map is a one-dimensional map function which can be represented as(29)xt+1=μ7.86xt−23.31xt2+28.75xt3−13.303xt4μ∈0.9,1.08.In our experiments,μ=1.071 and x0=0.7. The value of xt is between 0 and 1.(8)
Gaussian map [44]: Gaussian map is a one-dimensional nonlinear map based on an exponential function. The formula of Gaussian map is formulated as(30)xt+1=0,xt=0,modμxt,1,xt≠0.Forμ=1, x0=0.52 and chaotic sequence xt∈0,1.(9)
Sinusoidal map [45]: this map generates chaotic sequences based on sine function, which is determinedly formulated as(31)xt+1=axt2sinπxt.It is worth to note that these equations can be simplified to the following formula whena=2.3 and x0=0.7:(32)xt+1=sinπxt.Here, we seta=2.3, x0=0.242, and xt∈0,1.(10)
Tent map: tent map has a special chaos mechanism which is akin to logistic map, and the whole mapping process can be expressed as(33)xt+1=xtβ,xt∈0,β,1−xt1−β,xt∈β,1,whereβ=0.4, xt∈0,1, and x0=0.7.(11)
Iterative chaotic map with infinite collapses (ICMICs) [46]: this map function generates chaotic signals with infinite collapses in a determinate region. The formula is shown as(34)xt+1=sinaxta∈0,∞,wherea is set to 15. There are negative numbers in the chaotic sequence generated by ICMIC, so we will call an absolute value operator when generating chaos.(12)
Bernoulli map: Bernoulli map is a weak-mixing piecewise function with ergodic properties. It can be depicted by(35)xt+1=xt1−λ,xt∈0,β,1−xt1−β,xt∈β,1.Here,λ=0.4 and x0=0.7.


## 3.2. Chaotic Local Search

According to these twelve chaotic mapping formulas, we can obtain a series of chaotic sequence to replace pseudo-random numbers with a range of0,1. Figure 2 can exhibit distribution status of numbers for twelve chaotic maps and each chaotic map has its own characteristic. From it, it is obvious that the singer map easily generates chaotic sequence with a range of 0.7,1 and the circle map is easier to generate output of a number between 0.3,0.7. Due to these distinct numerical peculiarities of chaotic maps, we utilize a chaotic local search strategy to improve original GBO algorithm.Figure 2
Histograms of distribution for twelve different chaotic maps.Recently, the local search strategy has been successfully employed to improve the global single-objective and multiobjective optimization algorithms [47, 48], and the local search strategy is usually used to jump out of the local optimal point when global algorithm is trapped into stagnation, such as well-known tabu search [49, 50] and simulated annealing algorithm [51, 52]. The CLS strategy was recently emerged as an efficient local search method, which is utilized to optimize many state-of-the-art global optimization algorithms. The CLS strategy takes advantage of chaotic sequence with a range of 0,1 for adjusting current individuals. The adjusting method can be formulated as(36)xnew=xold+radius×ub−lb×chaos−z,where xold is the selected individual to be optimized, xnew is the optimized individual by CLS, radius is the scale parameter to adjust the influence of chaotic sequence between 0,0.01, and ub and lb denote the upper limit and lower limit defined by the specific optimization problems, respectively. chaos is the number which is obtained by the chaotic map, and z is the parameter to change the range of chaos between −0.5,0.5. The whole procedure of CLS can be shown in Figure 3.Figure 3
Chaotic local search strategy for optimization problems.


## 4. Mechanism of the Proposed Algorithm CGBO

### 4.1. Motivation

According to the brief introduction of GBO and CLS, we can find that GBO generates new individual based on Newton’s method and current population; in other words, it takes advantage of information about existing population to create the new individuals, despite a mechanism LEO being integrated into the principal part of GBO to avoid the stagnation of the whole algorithm, GBO will still be trapped into local optima under some circumstances due to incomplete judgement standard and operators. At the same time, CLS can effectively avoid local optima because of ergodicity and randomicity, and it can adjust individuals only through the sequence produced by the chaotic map. If taking into account the exploration and exploitation of metaheuristic algorithm, GBO performs well for global exploration ability and CLS has excellent local exploitation capability, which promoted us to mix GBO with CLS to improve the performance of the original algorithm.


### 4.2. Mechanism of CGBO and Some Fine-Tuning

On account of the above motivation, we incorporated CLS into GBO based on a randomly selection mechanism, modified the details, and proposed the algorithm CGBO. The pseudo-code of CGBO is shown in Algorithm1, where N is the number of individuals in the population, f expresses objective function, and r1, r2, and r3 are positive integers between 1,np; different from the original algorithm, we use Gaussian random numbers to generate r1, r2, and r3. pc is the probability of using chaotic search. In order to ensure the randomness of chaotic search strategy, twelve different mapping functions were selected with the same probability in this experiment. For local chaotic search strategy to better integrate into the original algorithm, we fine-tune equation (15) of GBO. σ is set to be(37)σ=2×rand×xr1+xr2+xr3+xbest4−xn.Algorithm 1:Pseudo-code of CGBO.
(1)
begin(2)
/∗initialization of algorithm∗/(3)
fort=1toN do(4)
Randomly generate individualxt(5)
fxt = evaluate xt(6)
fori=1toN do(7)
Select three random individualsxr1, xr2, xr3 and current best xbest.(8)
/∗Parameter setting∗/(9)
Generate parameterβ, α and ρ1 by equation (14).(10)
Determine scale factorradius by means of iteration number.(11)
Generate adjustment vectors GSR and DM.(12)
/∗Gradient search rule∗/(13)
forj=1to Dim do(14)
Optionally modify the value on each dimensionj, and obtain xGSRi by equation (19).(15)
ifrand<prthen(16)
/∗Local escaping operator∗/(17)
xnewi is given by equation (20).(18)
else(19)
/∗Chaotic local search∗/(20)
ifrand<pcthen(21)
fork=1to Dimdo(22)
ifrand<0.6then(23)
Calculatexnewi,j based on xGSRi and equation (36).(24)
else(25)
xnewi = xGSRTo make CGBO be easily understood, the flowchart is also shown in Figure4. Due to chaotic local search greatly increasing the diversity of the population, our proposed algorithm will generate various individuals based on more complex population and have a higher probability of finding the global optimum in a wider search space, so that proposed algorithm CGBO theoretically achieves a better balance between exploitation and exploration.Figure 4
The whole flowchart of CGBO.


## 4.1. Motivation

According to the brief introduction of GBO and CLS, we can find that GBO generates new individual based on Newton’s method and current population; in other words, it takes advantage of information about existing population to create the new individuals, despite a mechanism LEO being integrated into the principal part of GBO to avoid the stagnation of the whole algorithm, GBO will still be trapped into local optima under some circumstances due to incomplete judgement standard and operators. At the same time, CLS can effectively avoid local optima because of ergodicity and randomicity, and it can adjust individuals only through the sequence produced by the chaotic map. If taking into account the exploration and exploitation of metaheuristic algorithm, GBO performs well for global exploration ability and CLS has excellent local exploitation capability, which promoted us to mix GBO with CLS to improve the performance of the original algorithm.


## 4.2. Mechanism of CGBO and Some Fine-Tuning

On account of the above motivation, we incorporated CLS into GBO based on a randomly selection mechanism, modified the details, and proposed the algorithm CGBO. The pseudo-code of CGBO is shown in Algorithm1, where N is the number of individuals in the population, f expresses objective function, and r1, r2, and r3 are positive integers between 1,np; different from the original algorithm, we use Gaussian random numbers to generate r1, r2, and r3. pc is the probability of using chaotic search. In order to ensure the randomness of chaotic search strategy, twelve different mapping functions were selected with the same probability in this experiment. For local chaotic search strategy to better integrate into the original algorithm, we fine-tune equation (15) of GBO. σ is set to be(37)σ=2×rand×xr1+xr2+xr3+xbest4−xn.Algorithm 1:Pseudo-code of CGBO.
(1)
begin(2)
/∗initialization of algorithm∗/(3)
fort=1toN do(4)
Randomly generate individualxt(5)
fxt = evaluate xt(6)
fori=1toN do(7)
Select three random individualsxr1, xr2, xr3 and current best xbest.(8)
/∗Parameter setting∗/(9)
Generate parameterβ, α and ρ1 by equation (14).(10)
Determine scale factorradius by means of iteration number.(11)
Generate adjustment vectors GSR and DM.(12)
/∗Gradient search rule∗/(13)
forj=1to Dim do(14)
Optionally modify the value on each dimensionj, and obtain xGSRi by equation (19).(15)
ifrand<prthen(16)
/∗Local escaping operator∗/(17)
xnewi is given by equation (20).(18)
else(19)
/∗Chaotic local search∗/(20)
ifrand<pcthen(21)
fork=1to Dimdo(22)
ifrand<0.6then(23)
Calculatexnewi,j based on xGSRi and equation (36).(24)
else(25)
xnewi = xGSRTo make CGBO be easily understood, the flowchart is also shown in Figure4. Due to chaotic local search greatly increasing the diversity of the population, our proposed algorithm will generate various individuals based on more complex population and have a higher probability of finding the global optimum in a wider search space, so that proposed algorithm CGBO theoretically achieves a better balance between exploitation and exploration.Figure 4
The whole flowchart of CGBO.


## 5. Experimental Results and Statistical Analysis

To testify the effectiveness of our proposed mechanism, we select two different experiments to prove its effectiveness, which consists of conventional benchmark function optimization and a parameter optimization of the dendritic neuron model. For all experiments, we utilized a computer with processor Intel (R) Core (TM) i7-9700 CPU@ 3.00GHZ RAM (8 GB) to test.


### 5.1. CEC2017 Benchmark Function

IEEE CEC2017 benchmark contains three unimodal functions, seven simple multimodal functions, ten hybrid functions, and ten composition functions. Except for the unimodal functions, the others have quite a few local optimal points, which require excellent optimization algorithms with robustness and effectiveness. Therefore, we compare the experimental results of CGBO for CEC2017 with original algorithm GBO and some state-of-the-art algorithms. In addition to the original algorithm GBO, genetic learning particle swarm optimization (GLPSO) [53], brain storm optimization based on orthogonal learning design (OLBSO) [54], dynamic neighborhood learning-based gravitational search algorithm (DNLGSA) [55], and multiple chaotic maps-incorporated grey wolf optimization algorithm (CGWO) [56] are selected. The relative parameter setting of algorithms to be compared is listed in Table 1.Table 1
Detailed parameter setting of algorithms.
AlgorithmParameter settingGLPSOω=0.7298, c=1.49618, pm=0.01, sg=7OLBSOcluster_num=5, pg=0.8DNLGSAcluster_num=10, k=10, gm=5CGWOα∈0,2, r=5In our experiment, thirty benchmark functions of CEC2017 are used to test the performance of algorithm. The population sizeN of each algorithm is set to be 100, number of dimensions dim=30, the maximum of iterative number is 3000, and the running times of each algorithm are set to be 51 due to the randomness of metaheuristic algorithms, which are sufficient to evaluate the overall performance of algorithms. Table 2 lists twenty-nine convergence results of CEC2017 functions. It is worth mentioning that F2 is usually discarded because of its instability, where mean and std, respectively, represent average values and standard deviations of final optimal values under fifty-one tests. The highlighted numerical values have significant advantage in comparison with all other competitors. In addition to this, the last line W/T/L severally signifies that our proposed CGBO performs better/tied/worse than the rest of algorithms under five percent significant level. The experimental results manifest that CGBO is more competitive than other advanced algorithms for the optimization of hybrid functions and composition functions.Table 2
Experimental results on CEC2017.
CGBOGBOGLPSOOLBSODNLGSACGWOMeanStdMeanStdMeanStdMeanStdMeanStdMeanStdF18.33E +029.87E +026.90E +027.94E +021.33E +047.37E +042.25E +032.57E +035.38E +073.68E +085.60E +076.47E +07F33.00E +023.43E −053.00E +023.77E −051.65E +041.54E +043.01E +024.20E −011.80E +042.14E +044.42E +032.58E +03F44.22E +022.84E +014.20E +022.80E +014.93E +024.44E +014.72E +021.51E +016.91E +022.14E +025.14E +021.86E +01F56.50E +023.82E +016.40E +023.47E +015.64E +021.47E +016.47E +023.49E +016.38E +022.51E +015.93E +022.82E +01F66.18E +029.70E +006.15E +028.37E +006.00E +021.74E −016.05E +022.60E +006.40E +026.28E +006.05E +022.75E +00F79.29E +025.98E +019.16E +026.07E +018.34E +022.02E +019.01E +023.50E +019.81E +026.81E +018.51E +024.25E +01F89.31E +022.52E +019.29E +022.81E +018.64E +021.41E +019.14E +022.40E +019.15E +022.79E +018.78E +022.37E +01F92.19E +038.87E +022.02E +038.08E +021.27E +032.17E +023.48E +037.13E +023.77E +031.19E +031.44E +033.63E +02F105.03E +037.08E +024.89E +037.06E +023.72E +034.74E +024.70E +034.86E +025.01E +036.62E +024.29E +031.10E +03F111.22E +035.38E +011.24E +036.27E +011.27E +031.52E +021.23E +033.95E +011.44E +032.23E +021.25E +033.65E +01F121.81E +041.22E +042.29E +041.39E +042.08E +062.25E +061.79E +061.27E +061.46E +081.65E +081.25E +071.14E +07F138.18E +038.56E +038.34E +038.12E +031.60E +041.49E +041.99E +041.39E +042.60E +051.69E +061.64E +057.23E +04F141.68E +037.18E +011.67E +038.24E +015.95E +041.48E +054.98E +032.48E +035.49E +041.02E +054.98E +036.10E +03F153.42E +033.02E +034.93E +034.34E +036.33E +036.00E +036.01E +034.88E +041.33E +041.04E +043.86E +043.05E +04F162.62E +033.08E +022.52E +032.76E +022.65E +033.22E +022.87E +032.43E +022.85E +033.03E +022.33E +032.64E +02F172.22E +032.31E +022.21E +032.17E +022.09E +031.81E +022.27E +032.36E +022.29E +032.49E +021.96E +031.18E +02F181.47E +041.36E +042.06E +041.69E +043.26E +058.87E +051.27E +051.08E +051.76E +051.32E +051.06E +058.51E +04F192.90E +032.03E +033.68E +033.41E +039.05E +031.22E +048.84E +038.77E +041.44E +043.89E +041.24E +051.44E +05F202.46E +031.61E +022.41E +031.57E +022.40E +031.57E +022.50E +031.69E +022.62E +032.20E +022.35E +031.51E +02F212.43E +033.41E +012.42E +032.63E +012.37E +031.78E +012.46E +033.84E +012.44E +033.12E +012.38E +032.66E +01F223.29E +031.82E +032.89E +031.50E +032.54E +038.27E +024.19E +031.98E +035.07E +032.19E +033.61E +031.62E +03F232.79E +034.00E +012.79E +034.76E +012.74E +032.26E +012.90E +037.28E +022.99E +038.65E +012.76E +034.43E +01F242.96E +034.34E +012.96E +034.10E +012.95E +033.92E +013.06E +035.60E +013.18E +039.42E +012.94E +034.69E +01F252.90E +031.51E +012.89E +031.18E +012.90E +031.45E +012.88E +031.12E +013.01E +037.12E +012.91E +032.04E +01F265.07E +039.97E +024.72E +031.28E +034.50E +039.29E +026.15E +031.54E +035.86E +031.31E +034.52E +035.16E +02F273.25E +033.14E +013.24E +032.24E +013.26E +032.26E +013.34E +038.58E +013.42E +031.54E +023.23E +031.57E +01F283.15E +036.44E +013.15E +036.08E +013.22E +035.23E +013.18E +035.19E +013.44E +031.35E +023.29E +033.51E +01F293.99E +032.77E +023.89E +032.58E +023.67E +032.01E +024.35E +033.05E +024.29E +033.01E +023.68E +031.37E +02F306.68E +031.33E +037.87E +032.44E +034.58E +045.15E +043.94E +052.34E +054.90E +061.15E +071.36E +068.88E +05W/T/L—8/20/115/2/1221/3/525/3/114/1/14


### 5.2. Convergence Graphs and Box-and-Whisker Graphs

Except for table data, we will employ a variety of illustrations to demonstrate the advantages of CGBO. Several convergence figures and boxplot graphs for functionsF11,F12, and F15 are summarized in Figures 5 and 6.Figure 5
Convergence figures of five algorithms for F11, F12, and F15.Figure 6
Boxplot graphs of five algorithms for F11, F12, and F15.The horizontal axis and vertical axis of convergence figures, respectively, denote the number of iterations and average value of optimization. From Figures5 and 6, we can visibly find out the overall slope of CGBO is significantly larger than that of other algorithms. Meanwhile, the final convergence results of CGBO are lower than of competitor algorithms. These characteristics reveal that CGBO can not only converge quickly but also effectively keep from trapping into local optima and seek out the global optimal solution for solving some hybrid functions.Figure6 exhibits three boxplot graphs, where the red plus sign represents the outlier which is defined as a measured value that deviates from the mean by more than two times the standard deviation. The top (bottom) line is the maximum (minimum) value except for outliers, the upper and lower border lines, respectively, denote the upper and lower quantiles, and the red line in the middle is the median. These boxplots demonstrate that most assessment criteria of CGBO are better than those of other algorithms. It proves that CGBO has stability and robustness for a part of function optimization problems.


### 5.3. Population Diversity

In order to prove that the proposed algorithm has better efficiency in global exploration capability, we calculate the population diversity of the two algorithms to compare the improved algorithm CGBO with GBO. The population diversity is an important evaluation for metaheuristic algorithms. An excellent improvement mechanism will keep a high-level population diversity throughout the whole iterative process. Population diversity can be obtained by calculating the sum of standard deviations between each individual and the mean value of the entire population. Therefore, population diversity oftth iteration Divt can be formulated as follows:(38)Divt=∑i=1NNormxi−xmeanN,where Norm returns the Euclidean norm, xmean represents the mean value of each dimension in the whole population, and N is the number of individuals. Several functions are selected to draw up the variation curves of population diversity during the whole convergence process, and the diagram is demonstrated in Figure 7.Figure 7
Diagram of population diversity about CGBO by comparison with GBO for F13, F15, and F30.The population-based stochastic algorithms need to maintain a sophisticated population diversity during the inchoate iterative process to avoid falling into local optima. However, if population diversity keeps a high level all the time, its convergence speed is bound to be affected. From three diagrams, we can discover that CGBO has high population diversity before 1500 iterations and then steadily decline during the later optimization process in order to speed up convergence. This further demonstrates the efficiency of keeping the balance between exploration and exploitation for our proposed mechanism.


### 5.4. Contour Map

In order to more intuitively reflect the changing situation of individuals in the population, we utilized a two-dimensional contour map to exhibit the entire optimization process, as shown in Figure8. We select three kinds of individual distribution under the same number of iterations (iteration = 1, 50, 3000), where red dot represents the value of the first two dimensions for the individual. At the end of the optimization, all the individuals are clustered into a local optima (global optima), which is represented by a red five-pointed star. From the contour map of F28, we can find that all points assemble between Dim1∈−50,−25 and Dim2∈60,100 in the beginning of iteration, but in the end, population converges to a global optimum near point −30,−38. This indicates that CGBO has the probability to jump out of the local optimum when it is trapped into a local optimum; in other words, CGBO has excellent exploration ability.Figure 8
The 2-dimensional contour map of CGBO for F5, F22, and F28.


### 5.5. Parameter Optimization of DNM

The earliest single-layer perceptron only considered the function and structure of a single neuron [57], and it could only solve linear problems, whereas synapses were nonlinear. According to these theories, a single neuron model DNM was proposed and successfully applied to many real-world problems, for instance, liver disorder analysis, breast cancer classification, and financial time series prediction [58, 59].DNM has four layers which include synaptic layer, dendrite layer, membrane layer, and soma layer. The detailed structure of DNM is shown in Figure9. The input data are calculated by a modified sigmoid function in the synaptic layer, which normalizes the original data. The dendrite layer and membrane layer are, respectively, the multiplication operator and accumulation operator. It is worth noting that the multiplication operator can be equivalent to logic AND and the accumulation operator approximates logic OR when input data correspond to 0 or 1. Finally, the soma layer includes a sigmoid function to receive the entire output of the neuron.Figure 9
Dendritic neuron model.DNM is a neural network with a simple structure, which fully embodies the characteristics of a real neuron [60, 61]. A usable neural network structure cannot be achieved without an excellent training algorithm, and we can find that a simple DNM has only two parameters that need to be trained including weight W and threshold θ. In allusion to different classification or prediction problems, the input data are distinct and the structure of DNM also changes accordingly. The number of inputs on dendrite layer corresponds to features number, and the number of dendrites equals the number of individuals in metaheuristic algorithms.According to the above mechanism, we can take the advantage of our proposed CGBO algorithm to train DNM. Data sets involve Iris, Liver, Mackey Glass, Cancer, and Wine are selected to test the performance of the posttraining DNM. Table3 lists the best accuracy of DNM optimized by CGBO, GBO, and other three well-known training algorithms [62, 63] under 30 running times.Table 3
Experimental results for five classification data.
IrisLiverGlassCancerWineAlgorithmTrain set (%)Test set (%)Train set (%)Test set (%)Train set (%)Test set (%)Train set (%)Test set (%)Train set (%)Test set (%)CGBO97.3396.0075.0063.0097.1995.3399.4295.1498.8792.13GBO98.6788.0072.0958.3898.1390.6599.4294.0093.2586.51BP94.6796.0062.7953.1710094.3996.8494.2864.0456.17PSO93.3389.3372.6758.9597.1988.7898.5694.5782.0267.41JADE97.3394.6774.4159.5398.1395.3299.1494.8597.7586.51Here, each data set is divided into 50 percent test set and 50 percent train set, and each bold numerical value represents the best accuracy of them. CGBO apparently performs superior to other competitive algorithms. From the tables and graphs above, we can conclude that CGBO has excellent performance for function optimization or parameter training in terms of effectiveness and fault tolerance.


### 5.6. Time Complexity

Time complexity qualitatively describes the running time of the algorithm. An exponential time complexity will explode as the number of input data increases, which requires enormous computer resources, so the time complexity is also a significant indicator to evaluate the performance of the algorithm.Here, we compare the original algorithm GBO with the improved algorithm CGBO. Firstly, we take into account what these two algorithms have in common. The time complexity for procedure of initialization isON, and the GSR strategy traverses each dimension, whose time complexity is OM∗NM is generally much less than N. In addition, LEO costs ON and the CLS contains two parts; the time to generate chaotic sequences needs O12∗N and chaotic search process takes OM∗N. Obviously, in neither GBO nor CGBO, the total time complexity can be summed up as ON, which is rare and commendable for an algorithm with excellent computational performance. In the real machine demo, CGBO has precious less time consumption except for function evaluation. It is worth noting that the time complexity of competitor algorithms GLPSO, OLBSO, DNLGSA, JADE, and CGWO are all ON2 in the worst situation.


## 5.1. CEC2017 Benchmark Function

IEEE CEC2017 benchmark contains three unimodal functions, seven simple multimodal functions, ten hybrid functions, and ten composition functions. Except for the unimodal functions, the others have quite a few local optimal points, which require excellent optimization algorithms with robustness and effectiveness. Therefore, we compare the experimental results of CGBO for CEC2017 with original algorithm GBO and some state-of-the-art algorithms. In addition to the original algorithm GBO, genetic learning particle swarm optimization (GLPSO) [53], brain storm optimization based on orthogonal learning design (OLBSO) [54], dynamic neighborhood learning-based gravitational search algorithm (DNLGSA) [55], and multiple chaotic maps-incorporated grey wolf optimization algorithm (CGWO) [56] are selected. The relative parameter setting of algorithms to be compared is listed in Table 1.Table 1
Detailed parameter setting of algorithms.
AlgorithmParameter settingGLPSOω=0.7298, c=1.49618, pm=0.01, sg=7OLBSOcluster_num=5, pg=0.8DNLGSAcluster_num=10, k=10, gm=5CGWOα∈0,2, r=5In our experiment, thirty benchmark functions of CEC2017 are used to test the performance of algorithm. The population sizeN of each algorithm is set to be 100, number of dimensions dim=30, the maximum of iterative number is 3000, and the running times of each algorithm are set to be 51 due to the randomness of metaheuristic algorithms, which are sufficient to evaluate the overall performance of algorithms. Table 2 lists twenty-nine convergence results of CEC2017 functions. It is worth mentioning that F2 is usually discarded because of its instability, where mean and std, respectively, represent average values and standard deviations of final optimal values under fifty-one tests. The highlighted numerical values have significant advantage in comparison with all other competitors. In addition to this, the last line W/T/L severally signifies that our proposed CGBO performs better/tied/worse than the rest of algorithms under five percent significant level. The experimental results manifest that CGBO is more competitive than other advanced algorithms for the optimization of hybrid functions and composition functions.Table 2
Experimental results on CEC2017.
CGBOGBOGLPSOOLBSODNLGSACGWOMeanStdMeanStdMeanStdMeanStdMeanStdMeanStdF18.33E +029.87E +026.90E +027.94E +021.33E +047.37E +042.25E +032.57E +035.38E +073.68E +085.60E +076.47E +07F33.00E +023.43E −053.00E +023.77E −051.65E +041.54E +043.01E +024.20E −011.80E +042.14E +044.42E +032.58E +03F44.22E +022.84E +014.20E +022.80E +014.93E +024.44E +014.72E +021.51E +016.91E +022.14E +025.14E +021.86E +01F56.50E +023.82E +016.40E +023.47E +015.64E +021.47E +016.47E +023.49E +016.38E +022.51E +015.93E +022.82E +01F66.18E +029.70E +006.15E +028.37E +006.00E +021.74E −016.05E +022.60E +006.40E +026.28E +006.05E +022.75E +00F79.29E +025.98E +019.16E +026.07E +018.34E +022.02E +019.01E +023.50E +019.81E +026.81E +018.51E +024.25E +01F89.31E +022.52E +019.29E +022.81E +018.64E +021.41E +019.14E +022.40E +019.15E +022.79E +018.78E +022.37E +01F92.19E +038.87E +022.02E +038.08E +021.27E +032.17E +023.48E +037.13E +023.77E +031.19E +031.44E +033.63E +02F105.03E +037.08E +024.89E +037.06E +023.72E +034.74E +024.70E +034.86E +025.01E +036.62E +024.29E +031.10E +03F111.22E +035.38E +011.24E +036.27E +011.27E +031.52E +021.23E +033.95E +011.44E +032.23E +021.25E +033.65E +01F121.81E +041.22E +042.29E +041.39E +042.08E +062.25E +061.79E +061.27E +061.46E +081.65E +081.25E +071.14E +07F138.18E +038.56E +038.34E +038.12E +031.60E +041.49E +041.99E +041.39E +042.60E +051.69E +061.64E +057.23E +04F141.68E +037.18E +011.67E +038.24E +015.95E +041.48E +054.98E +032.48E +035.49E +041.02E +054.98E +036.10E +03F153.42E +033.02E +034.93E +034.34E +036.33E +036.00E +036.01E +034.88E +041.33E +041.04E +043.86E +043.05E +04F162.62E +033.08E +022.52E +032.76E +022.65E +033.22E +022.87E +032.43E +022.85E +033.03E +022.33E +032.64E +02F172.22E +032.31E +022.21E +032.17E +022.09E +031.81E +022.27E +032.36E +022.29E +032.49E +021.96E +031.18E +02F181.47E +041.36E +042.06E +041.69E +043.26E +058.87E +051.27E +051.08E +051.76E +051.32E +051.06E +058.51E +04F192.90E +032.03E +033.68E +033.41E +039.05E +031.22E +048.84E +038.77E +041.44E +043.89E +041.24E +051.44E +05F202.46E +031.61E +022.41E +031.57E +022.40E +031.57E +022.50E +031.69E +022.62E +032.20E +022.35E +031.51E +02F212.43E +033.41E +012.42E +032.63E +012.37E +031.78E +012.46E +033.84E +012.44E +033.12E +012.38E +032.66E +01F223.29E +031.82E +032.89E +031.50E +032.54E +038.27E +024.19E +031.98E +035.07E +032.19E +033.61E +031.62E +03F232.79E +034.00E +012.79E +034.76E +012.74E +032.26E +012.90E +037.28E +022.99E +038.65E +012.76E +034.43E +01F242.96E +034.34E +012.96E +034.10E +012.95E +033.92E +013.06E +035.60E +013.18E +039.42E +012.94E +034.69E +01F252.90E +031.51E +012.89E +031.18E +012.90E +031.45E +012.88E +031.12E +013.01E +037.12E +012.91E +032.04E +01F265.07E +039.97E +024.72E +031.28E +034.50E +039.29E +026.15E +031.54E +035.86E +031.31E +034.52E +035.16E +02F273.25E +033.14E +013.24E +032.24E +013.26E +032.26E +013.34E +038.58E +013.42E +031.54E +023.23E +031.57E +01F283.15E +036.44E +013.15E +036.08E +013.22E +035.23E +013.18E +035.19E +013.44E +031.35E +023.29E +033.51E +01F293.99E +032.77E +023.89E +032.58E +023.67E +032.01E +024.35E +033.05E +024.29E +033.01E +023.68E +031.37E +02F306.68E +031.33E +037.87E +032.44E +034.58E +045.15E +043.94E +052.34E +054.90E +061.15E +071.36E +068.88E +05W/T/L—8/20/115/2/1221/3/525/3/114/1/14


## 5.2. Convergence Graphs and Box-and-Whisker Graphs

Except for table data, we will employ a variety of illustrations to demonstrate the advantages of CGBO. Several convergence figures and boxplot graphs for functionsF11,F12, and F15 are summarized in Figures 5 and 6.Figure 5
Convergence figures of five algorithms for F11, F12, and F15.Figure 6
Boxplot graphs of five algorithms for F11, F12, and F15.The horizontal axis and vertical axis of convergence figures, respectively, denote the number of iterations and average value of optimization. From Figures5 and 6, we can visibly find out the overall slope of CGBO is significantly larger than that of other algorithms. Meanwhile, the final convergence results of CGBO are lower than of competitor algorithms. These characteristics reveal that CGBO can not only converge quickly but also effectively keep from trapping into local optima and seek out the global optimal solution for solving some hybrid functions.Figure6 exhibits three boxplot graphs, where the red plus sign represents the outlier which is defined as a measured value that deviates from the mean by more than two times the standard deviation. The top (bottom) line is the maximum (minimum) value except for outliers, the upper and lower border lines, respectively, denote the upper and lower quantiles, and the red line in the middle is the median. These boxplots demonstrate that most assessment criteria of CGBO are better than those of other algorithms. It proves that CGBO has stability and robustness for a part of function optimization problems.


## 5.3. Population Diversity

In order to prove that the proposed algorithm has better efficiency in global exploration capability, we calculate the population diversity of the two algorithms to compare the improved algorithm CGBO with GBO. The population diversity is an important evaluation for metaheuristic algorithms. An excellent improvement mechanism will keep a high-level population diversity throughout the whole iterative process. Population diversity can be obtained by calculating the sum of standard deviations between each individual and the mean value of the entire population. Therefore, population diversity oftth iteration Divt can be formulated as follows:(38)Divt=∑i=1NNormxi−xmeanN,where Norm returns the Euclidean norm, xmean represents the mean value of each dimension in the whole population, and N is the number of individuals. Several functions are selected to draw up the variation curves of population diversity during the whole convergence process, and the diagram is demonstrated in Figure 7.Figure 7
Diagram of population diversity about CGBO by comparison with GBO for F13, F15, and F30.The population-based stochastic algorithms need to maintain a sophisticated population diversity during the inchoate iterative process to avoid falling into local optima. However, if population diversity keeps a high level all the time, its convergence speed is bound to be affected. From three diagrams, we can discover that CGBO has high population diversity before 1500 iterations and then steadily decline during the later optimization process in order to speed up convergence. This further demonstrates the efficiency of keeping the balance between exploration and exploitation for our proposed mechanism.


## 5.4. Contour Map

In order to more intuitively reflect the changing situation of individuals in the population, we utilized a two-dimensional contour map to exhibit the entire optimization process, as shown in Figure8. We select three kinds of individual distribution under the same number of iterations (iteration = 1, 50, 3000), where red dot represents the value of the first two dimensions for the individual. At the end of the optimization, all the individuals are clustered into a local optima (global optima), which is represented by a red five-pointed star. From the contour map of F28, we can find that all points assemble between Dim1∈−50,−25 and Dim2∈60,100 in the beginning of iteration, but in the end, population converges to a global optimum near point −30,−38. This indicates that CGBO has the probability to jump out of the local optimum when it is trapped into a local optimum; in other words, CGBO has excellent exploration ability.Figure 8
The 2-dimensional contour map of CGBO for F5, F22, and F28.


## 5.5. Parameter Optimization of DNM

The earliest single-layer perceptron only considered the function and structure of a single neuron [57], and it could only solve linear problems, whereas synapses were nonlinear. According to these theories, a single neuron model DNM was proposed and successfully applied to many real-world problems, for instance, liver disorder analysis, breast cancer classification, and financial time series prediction [58, 59].DNM has four layers which include synaptic layer, dendrite layer, membrane layer, and soma layer. The detailed structure of DNM is shown in Figure9. The input data are calculated by a modified sigmoid function in the synaptic layer, which normalizes the original data. The dendrite layer and membrane layer are, respectively, the multiplication operator and accumulation operator. It is worth noting that the multiplication operator can be equivalent to logic AND and the accumulation operator approximates logic OR when input data correspond to 0 or 1. Finally, the soma layer includes a sigmoid function to receive the entire output of the neuron.Figure 9
Dendritic neuron model.DNM is a neural network with a simple structure, which fully embodies the characteristics of a real neuron [60, 61]. A usable neural network structure cannot be achieved without an excellent training algorithm, and we can find that a simple DNM has only two parameters that need to be trained including weight W and threshold θ. In allusion to different classification or prediction problems, the input data are distinct and the structure of DNM also changes accordingly. The number of inputs on dendrite layer corresponds to features number, and the number of dendrites equals the number of individuals in metaheuristic algorithms.According to the above mechanism, we can take the advantage of our proposed CGBO algorithm to train DNM. Data sets involve Iris, Liver, Mackey Glass, Cancer, and Wine are selected to test the performance of the posttraining DNM. Table3 lists the best accuracy of DNM optimized by CGBO, GBO, and other three well-known training algorithms [62, 63] under 30 running times.Table 3
Experimental results for five classification data.
IrisLiverGlassCancerWineAlgorithmTrain set (%)Test set (%)Train set (%)Test set (%)Train set (%)Test set (%)Train set (%)Test set (%)Train set (%)Test set (%)CGBO97.3396.0075.0063.0097.1995.3399.4295.1498.8792.13GBO98.6788.0072.0958.3898.1390.6599.4294.0093.2586.51BP94.6796.0062.7953.1710094.3996.8494.2864.0456.17PSO93.3389.3372.6758.9597.1988.7898.5694.5782.0267.41JADE97.3394.6774.4159.5398.1395.3299.1494.8597.7586.51Here, each data set is divided into 50 percent test set and 50 percent train set, and each bold numerical value represents the best accuracy of them. CGBO apparently performs superior to other competitive algorithms. From the tables and graphs above, we can conclude that CGBO has excellent performance for function optimization or parameter training in terms of effectiveness and fault tolerance.


## 5.6. Time Complexity

Time complexity qualitatively describes the running time of the algorithm. An exponential time complexity will explode as the number of input data increases, which requires enormous computer resources, so the time complexity is also a significant indicator to evaluate the performance of the algorithm.Here, we compare the original algorithm GBO with the improved algorithm CGBO. Firstly, we take into account what these two algorithms have in common. The time complexity for procedure of initialization isON, and the GSR strategy traverses each dimension, whose time complexity is OM∗NM is generally much less than N. In addition, LEO costs ON and the CLS contains two parts; the time to generate chaotic sequences needs O12∗N and chaotic search process takes OM∗N. Obviously, in neither GBO nor CGBO, the total time complexity can be summed up as ON, which is rare and commendable for an algorithm with excellent computational performance. In the real machine demo, CGBO has precious less time consumption except for function evaluation. It is worth noting that the time complexity of competitor algorithms GLPSO, OLBSO, DNLGSA, JADE, and CGWO are all ON2 in the worst situation.


## 6. Conclusions

In this paper, we incorporate an improved CLS into a metaheuristic algorithm GBO and propose an upgraded version CGBO. Different from a single chaotic map adopted in the past, we randomly selected different chaotic mapping functions in order to allow full play to the randomness and ergodicity of chaotic sequences. Moreover, this mechanism produces more biased data, which are more regular than pseudo-random numbers and more random than density functions such as Gaussian functions. To comprehensively testify the robustness and effectiveness of CGBO, we choose thirty single-objective CEC2017 benchmark optimization function and parameter optimization of DNM to test performance of CGBO. We selected several state-of-the-art algorithms that performed well for comparison. Two experimental results demonstrate that our proposed CGBO performs well for optimization problem. In our future plan, we attempt to establish a brand new and perfect adaptive chaotic search mechanism, and then, we develop a more competitive algorithm for solving some burning engineering optimization problems.



---

*Source: 3353926-2021-12-02.xml*