# A Novel Self-Adaptive Harmony Search Algorithm

Authors: Kaiping Luo
Journal: Journal of Applied Mathematics
 (2013)

---

## Abstract

The harmony search algorithm is a music-inspired optimization technology and has been successfully applied to diverse scientific and engineering problems. However, like other metaheuristic algorithms, it still faces two difficulties: parameter setting and finding the optimal balance between diversity and intensity in searching. This paper proposes a novel, self-adaptive search mechanism for optimization problems with continuous variables. This new variant can automatically configure the evolutionary parameters in accordance with problem characteristics, such as the scale and the boundaries, and dynamically select evolutionary strategies in accordance with its search performance. The new variant simplifies the parameter setting and efficiently solves all types of optimization problems with continuous variables. Statistical test results show that this variant is considerably robust and outperforms the original harmony search (HS), improved harmony search (IHS), and other self-adaptive variants for large-scale optimization problems and constrained problems.


---

## Body

## 1. Introduction

Harmony search (HS) is a relatively new metaheuristic optimization algorithm [1] that imitates the improvisation process of Jazz musicians as shown in Figure 1. Each musician is analogous to each decision variable; the audience’s aesthetics are analogous to the objective function; the pitch range of a musical instrument corresponds to the value range of the decision variables; the musicians’ improvisations correspond to local and global search schemes; musical harmony at certain time corresponds to a solution vector at certain iteration. In general, a musician’s improvisation follows three rules [2]: (1) playing any one pitch from his/her memory, (2) playing an adjacent pitch to one pitch from his/her memory, and (3) playing a random pitch from the possible sound range. The HS algorithm mimics these rules and converts them into corresponding evolutionary strategies: (1) choosing any one value from the HS memory (defined as memory considerations), (2) choosing a value adjacent to one from the HS memory (defined as pitch adjustments), and (3) choosing a random value from the possible value range (defined as randomisation). The evolutionary rules in the HS algorithm are effectively directed using the following parameters: (1) harmony memory size (HMS, i.e., the number of solutions in harmony memory), (2) harmony memory considering rate (HMCR, distinctly HMCR∈[0,1]), (3) pitch adjusting rate (PAR, distinctly PAR∈[0,1]), (4) fine-tuning bandwidth (Bw, i.e., the adjustment range of a variable), and (5) the number of improvisations (NI, i.e., the maximum number of iterations).Figure 1
The Analogy between Improvisation and Optimization (from Geem [3]).The values of these parameters, particularly three key parameters, HMCR, PAR, and Bw, affect the performances of HS. However, parameter setting is problem dependent and difficult, like other evolutionary computational methods, such as genetic algorithms (GA), simulated annealing (SA), particle swarm optimization (PSO), and ant colony optimization (ACO). To alleviate the burden of manually finding the best parameter values, this work proposes a natural and novel approach to parameter setting based on the improvisation practice of musicians.For any algorithm that involves the operation of searching randomly, the most difficult is to find an appropriate balance between diversity and intensity in the process of searching. when emphasising diversity, it is easy to conduct a slower convergence and even divergence; when emphasising intensity, it is easy to conduct premature convergence and become trapped in a local optimal solution. The HS algorithm is no exception. Thus, this paper also proposes a novel self-adaptive search mechanism.The rest of this paper is organized as follows. Section2 presents a concise review of related previous work. Section 3 describes the novel self-adaptive harmony search (NSHS) algorithm. Section 4 introduces a test suite composed of twelve benchmark functions. Section 5 analyzes the robustness of the proposed variant. Section 6 confirms the superiority of the novel variant by the comparison test. The last section makes a conclusion.


## 2. Review of Related Previous Work

### 2.1. Original Harmony Search Algorithm

The original HS algorithm developed by [1] is carried out as follows.Step 1 (initialise the problem and algorithm parameters).
First, the optimization problem is specified as follows:(1)minxf(x)s.t.xi∈[lbi,ubi](i=1,2,…,n),
wheref(x) is the objective function, x is a vector consisting of n decision variables, and lbi and ubi are the lower and upper bounds of variable xi, respectively. The original HS algorithm basically considers the objective function only. If the problem has equality and/or inequality constraints and a solution vector violates any of them, either the resulting infeasible solution will be abandoned or its objective function value will have a certain penalty score added.
Second, the parameters HMS, HMCR, PAR, Bw, and NI must be valued, and the initial harmony memory (HM) must be generated. Consider(2)HM=[x11x21⋯xn1f(x1)x12x22⋯xn2f(x2)⋮⋮⋮⋮⋮x1HMSx2HMS⋯xnHMSf(xHMS)].Step 2 (improve a new harmony vector by improvisation).
A new solutionxnew=(x1new,x2new,…,xnnew), termed the harmony vector, is generated based on memory considerations, pitch adjustments, and randomisation. Consider
(3)xinew⟵{xi∈{xi1,xi2,…,xiHMS};withprobabilityHMCR*(1-PAR)xi·(1+r*Bw);withprobabilityr~U[-1,1]HMCR*PARxi∈[lbi,ubi];withprobability1-HMCR.Step 3 (assess the new harmony vector and update the harmony memory).
In this step, the merit of the new harmony vector is assessed based on its objective function value. If the new vector is better than the worst harmony vector in the current HM, the new vector is included in the HM and the existing worst vector is excluded from the HM.Step 4 (check the stopping criterion).
Repeat Steps2~3 until the termination criterion is satisfied; that is, the number of iterations reaches NI.
The structure of the HS algorithm is much easier. Its core evolutionary process includes only three operations: choosing, adjusting, and randomising. However, the meta-heuristic algorithm handles intensification and diversification. Its intensification and diversification are represented by HMCR and PAR, respectively, and its superiority is confirmed by the existence of a large number of successful applications to diverse scientific and engineering problems, including timetabling [4], the 0-1 knapsack problem [5], power system design [6], transmission network planning [7], flow-shop scheduling [8], structural design [9], water network design [10], soil stability analysis [11], ecological conservation [12], and vehicle routing [13].
However, the major disadvantage of the HS algorithm is that its performance depends on the parameter setting and static parameter values are not conducive to balancing intensification and diversification.


### 2.2. Variants Based on Parameter Setting

To improve the performance of the original HS algorithm or alleviate the burden of manually finding the best parameter values, some variants based on parameter setting have been proposed.


#### 2.2.1. Improved Harmony Search (IHS) Proposed by Mahdavi et al. [14]

In this variant, PAR and Bw are dynamically adjusted by the following formulae:(4)PAR(k)=PARmin+(PARmax-PARmin)*kNI,(5)Bw(k)=Bwmax*exp(kNI*ln(BwminBwmax)),wherePARmin and PARmax indicate the minimum and maximum pitch adjusting rates, respectively. k and NI denote the kth-iteration and the maximum number of iterations, respectively. Bwmin, and Bwmax indicate the minimum and maximum bandwidths.In fact, this variant does not alleviate the difficulty of parameter setting, because users have to provide suitable values ofPARmin, PARmax, Bwmin and Bwmax. Furthermore, using (4), the pitch adjusting rate gradually increases from PARmin to PARmax through an iterative process, which is rather inconsistent with the observation that a larger iteration number usually generates a more perfect harmony and requires a smaller tuning probability.Later, Omran and Mahdavi [15] abandoned the Bw parameter by replacing direct adjustment with the use of a random variable for the best harmony vector as shown in (6), calling this new variant the global-best harmony search (GHS). Consider
(6)xinew⟵xkbest(i=1,2,…,n;k∈{1,2,…,n})ccccccccccccccwithprobabilityHMCR·PAR.As Wang and Huang [16] noted, the name of this variant is misused and the method can easily cause premature convergence. In fact, the most serious defect in the GHS variant is that if the domain of each variable is different, that is, [lbi,ubi]≠[lbk,ubk],(i,k=1,2,…,n;i≠k), then the infeasible solution will frequently be generated. Thus, Pan et al. [17] corrected this serious drawback using the following equation:
(7)xinew⟵xibest(i=1,2,…,n)withprobabilityHMCR·PAR.


#### 2.2.2. The Self-Adaptive Harmony Search Proposed by Wang and Huang [16]

Wang and Huang recognised the shortcomings of IHS and GHS. They used the same modification to dynamically adapt PAR values during the search process, but the value of PAR is decreased linearly with time. Thus, their variant needs are still the values ofPARmin and PARmax. The innovation of the variant is to adjust the value of each variable according to the maximal and minimal values in the harmony memory (HM) and without the Bw parameter again. Consider
(8)xinew⟵{xi+r·(maxh∈[1,HMS]HM(h,i)-xi);ifr≥0.5xi-r·(xi-minh∈[1,HMS]HM(h,i));ifr<0.5ccccccccccccccccccccccccccccccccccccccccr~U[0,1].Although this adjustment enables each variable to avoid violating its boundary constraint, the bandwidth is larger; therefore, the convergence is very slow and can even stop.


#### 2.2.3. The Self-Adaptive Harmony Search Proposed by Pan et al

Pan et al. [18] proposed a local-best harmony search algorithm with dynamic subpopulations (DLHS). In the variant, the whole harmony memory is equally divided into m subgroups. Moreover, to keep population diversity, the small-sized sub-HMs are regrouped once the period R is reached. In the variant, HMCR and PAR are from a parameter set list (PSL) with length Len, which is randomly generated by the initial value with a uniform distribution in the range PAR∈[0,1] and HMCR∈[0.9,1] and then updated once the list is empty. In the variant, the value of the parameter Bw is also dynamically changed:
(9)Bw(k)={Bwmax-Bwmax-BwminNI*2k;if2k<NI,Bwmin;otherwise.Although the variant takes the advantage of each local optimum, it consumes too much time as a cost. Thus, Pan et al. [17] proposed a self-adaptive global best harmony search algorithm (SGHS) based on the GHS algorithm. In the new variant, the value of HMCR (PAR) is normally distributed in the range [0.9,1]([0,1]) with mean HMCRm (PARm) and standard deviation 0.01 (0.05). The initial values of HMCRm and PARm are given by the user. In a fixed learning period (LP), the values of HMCRm and PARm do not change; after LP, the values are recalculated by averaging all HMCR and PAR values recorded during the previous LP. In the variant, Bw is still determined by (9).Obviously, the two variants require extra jobs to set the values of more parameters than the basic HS algorithm and other variants. Moreover, it is irrational that the value of PAR still varies in the interval[0,1] regardless of the number of iterations because a successful search should gradually converge and the value of PAR should be decreased with time to prevent oscillation.These defects inspired me to investigate how the key parameters of the HS algorithm do not depend on the problem’s feature again.


### 2.3. Variants Based on Hybridisation with Other Metaheuristics

To improve the performance of the original HS, many researchers have hybridised the algorithm with other meta-heuristics, such as GA, PSO, and ACO [19]. This hybridisation can be categorised into two approaches [20]: (1) integrating some components of other meta-heuristic algorithms into the HS structure, and conversely, (2) integrating some HS components into other meta-heuristic algorithm structures. Perhaps hybrid methods are better than single meta-heuristics.However, in my opinion, hybridisation will depart from the essential nature of the original heuristic and the resulting mimicry will also be nondescript. Thus, the correct way to investigate this should be by refining the essence of the imitated objects. This idea inspired me to improve the search mechanism of the original HS algorithm by capturing the deep essence of musical improvisation.


## 2.1. Original Harmony Search Algorithm

The original HS algorithm developed by [1] is carried out as follows.Step 1 (initialise the problem and algorithm parameters).
First, the optimization problem is specified as follows:(1)minxf(x)s.t.xi∈[lbi,ubi](i=1,2,…,n),
wheref(x) is the objective function, x is a vector consisting of n decision variables, and lbi and ubi are the lower and upper bounds of variable xi, respectively. The original HS algorithm basically considers the objective function only. If the problem has equality and/or inequality constraints and a solution vector violates any of them, either the resulting infeasible solution will be abandoned or its objective function value will have a certain penalty score added.
Second, the parameters HMS, HMCR, PAR, Bw, and NI must be valued, and the initial harmony memory (HM) must be generated. Consider(2)HM=[x11x21⋯xn1f(x1)x12x22⋯xn2f(x2)⋮⋮⋮⋮⋮x1HMSx2HMS⋯xnHMSf(xHMS)].Step 2 (improve a new harmony vector by improvisation).
A new solutionxnew=(x1new,x2new,…,xnnew), termed the harmony vector, is generated based on memory considerations, pitch adjustments, and randomisation. Consider
(3)xinew⟵{xi∈{xi1,xi2,…,xiHMS};withprobabilityHMCR*(1-PAR)xi·(1+r*Bw);withprobabilityr~U[-1,1]HMCR*PARxi∈[lbi,ubi];withprobability1-HMCR.Step 3 (assess the new harmony vector and update the harmony memory).
In this step, the merit of the new harmony vector is assessed based on its objective function value. If the new vector is better than the worst harmony vector in the current HM, the new vector is included in the HM and the existing worst vector is excluded from the HM.Step 4 (check the stopping criterion).
Repeat Steps2~3 until the termination criterion is satisfied; that is, the number of iterations reaches NI.
The structure of the HS algorithm is much easier. Its core evolutionary process includes only three operations: choosing, adjusting, and randomising. However, the meta-heuristic algorithm handles intensification and diversification. Its intensification and diversification are represented by HMCR and PAR, respectively, and its superiority is confirmed by the existence of a large number of successful applications to diverse scientific and engineering problems, including timetabling [4], the 0-1 knapsack problem [5], power system design [6], transmission network planning [7], flow-shop scheduling [8], structural design [9], water network design [10], soil stability analysis [11], ecological conservation [12], and vehicle routing [13].
However, the major disadvantage of the HS algorithm is that its performance depends on the parameter setting and static parameter values are not conducive to balancing intensification and diversification.


## 2.2. Variants Based on Parameter Setting

To improve the performance of the original HS algorithm or alleviate the burden of manually finding the best parameter values, some variants based on parameter setting have been proposed.


### 2.2.1. Improved Harmony Search (IHS) Proposed by Mahdavi et al. [14]

In this variant, PAR and Bw are dynamically adjusted by the following formulae:(4)PAR(k)=PARmin+(PARmax-PARmin)*kNI,(5)Bw(k)=Bwmax*exp(kNI*ln(BwminBwmax)),wherePARmin and PARmax indicate the minimum and maximum pitch adjusting rates, respectively. k and NI denote the kth-iteration and the maximum number of iterations, respectively. Bwmin, and Bwmax indicate the minimum and maximum bandwidths.In fact, this variant does not alleviate the difficulty of parameter setting, because users have to provide suitable values ofPARmin, PARmax, Bwmin and Bwmax. Furthermore, using (4), the pitch adjusting rate gradually increases from PARmin to PARmax through an iterative process, which is rather inconsistent with the observation that a larger iteration number usually generates a more perfect harmony and requires a smaller tuning probability.Later, Omran and Mahdavi [15] abandoned the Bw parameter by replacing direct adjustment with the use of a random variable for the best harmony vector as shown in (6), calling this new variant the global-best harmony search (GHS). Consider
(6)xinew⟵xkbest(i=1,2,…,n;k∈{1,2,…,n})ccccccccccccccwithprobabilityHMCR·PAR.As Wang and Huang [16] noted, the name of this variant is misused and the method can easily cause premature convergence. In fact, the most serious defect in the GHS variant is that if the domain of each variable is different, that is, [lbi,ubi]≠[lbk,ubk],(i,k=1,2,…,n;i≠k), then the infeasible solution will frequently be generated. Thus, Pan et al. [17] corrected this serious drawback using the following equation:
(7)xinew⟵xibest(i=1,2,…,n)withprobabilityHMCR·PAR.


### 2.2.2. The Self-Adaptive Harmony Search Proposed by Wang and Huang [16]

Wang and Huang recognised the shortcomings of IHS and GHS. They used the same modification to dynamically adapt PAR values during the search process, but the value of PAR is decreased linearly with time. Thus, their variant needs are still the values ofPARmin and PARmax. The innovation of the variant is to adjust the value of each variable according to the maximal and minimal values in the harmony memory (HM) and without the Bw parameter again. Consider
(8)xinew⟵{xi+r·(maxh∈[1,HMS]HM(h,i)-xi);ifr≥0.5xi-r·(xi-minh∈[1,HMS]HM(h,i));ifr<0.5ccccccccccccccccccccccccccccccccccccccccr~U[0,1].Although this adjustment enables each variable to avoid violating its boundary constraint, the bandwidth is larger; therefore, the convergence is very slow and can even stop.


### 2.2.3. The Self-Adaptive Harmony Search Proposed by Pan et al

Pan et al. [18] proposed a local-best harmony search algorithm with dynamic subpopulations (DLHS). In the variant, the whole harmony memory is equally divided into m subgroups. Moreover, to keep population diversity, the small-sized sub-HMs are regrouped once the period R is reached. In the variant, HMCR and PAR are from a parameter set list (PSL) with length Len, which is randomly generated by the initial value with a uniform distribution in the range PAR∈[0,1] and HMCR∈[0.9,1] and then updated once the list is empty. In the variant, the value of the parameter Bw is also dynamically changed:
(9)Bw(k)={Bwmax-Bwmax-BwminNI*2k;if2k<NI,Bwmin;otherwise.Although the variant takes the advantage of each local optimum, it consumes too much time as a cost. Thus, Pan et al. [17] proposed a self-adaptive global best harmony search algorithm (SGHS) based on the GHS algorithm. In the new variant, the value of HMCR (PAR) is normally distributed in the range [0.9,1]([0,1]) with mean HMCRm (PARm) and standard deviation 0.01 (0.05). The initial values of HMCRm and PARm are given by the user. In a fixed learning period (LP), the values of HMCRm and PARm do not change; after LP, the values are recalculated by averaging all HMCR and PAR values recorded during the previous LP. In the variant, Bw is still determined by (9).Obviously, the two variants require extra jobs to set the values of more parameters than the basic HS algorithm and other variants. Moreover, it is irrational that the value of PAR still varies in the interval[0,1] regardless of the number of iterations because a successful search should gradually converge and the value of PAR should be decreased with time to prevent oscillation.These defects inspired me to investigate how the key parameters of the HS algorithm do not depend on the problem’s feature again.


## 2.2.1. Improved Harmony Search (IHS) Proposed by Mahdavi et al. [14]

In this variant, PAR and Bw are dynamically adjusted by the following formulae:(4)PAR(k)=PARmin+(PARmax-PARmin)*kNI,(5)Bw(k)=Bwmax*exp(kNI*ln(BwminBwmax)),wherePARmin and PARmax indicate the minimum and maximum pitch adjusting rates, respectively. k and NI denote the kth-iteration and the maximum number of iterations, respectively. Bwmin, and Bwmax indicate the minimum and maximum bandwidths.In fact, this variant does not alleviate the difficulty of parameter setting, because users have to provide suitable values ofPARmin, PARmax, Bwmin and Bwmax. Furthermore, using (4), the pitch adjusting rate gradually increases from PARmin to PARmax through an iterative process, which is rather inconsistent with the observation that a larger iteration number usually generates a more perfect harmony and requires a smaller tuning probability.Later, Omran and Mahdavi [15] abandoned the Bw parameter by replacing direct adjustment with the use of a random variable for the best harmony vector as shown in (6), calling this new variant the global-best harmony search (GHS). Consider
(6)xinew⟵xkbest(i=1,2,…,n;k∈{1,2,…,n})ccccccccccccccwithprobabilityHMCR·PAR.As Wang and Huang [16] noted, the name of this variant is misused and the method can easily cause premature convergence. In fact, the most serious defect in the GHS variant is that if the domain of each variable is different, that is, [lbi,ubi]≠[lbk,ubk],(i,k=1,2,…,n;i≠k), then the infeasible solution will frequently be generated. Thus, Pan et al. [17] corrected this serious drawback using the following equation:
(7)xinew⟵xibest(i=1,2,…,n)withprobabilityHMCR·PAR.


## 2.2.2. The Self-Adaptive Harmony Search Proposed by Wang and Huang [16]

Wang and Huang recognised the shortcomings of IHS and GHS. They used the same modification to dynamically adapt PAR values during the search process, but the value of PAR is decreased linearly with time. Thus, their variant needs are still the values ofPARmin and PARmax. The innovation of the variant is to adjust the value of each variable according to the maximal and minimal values in the harmony memory (HM) and without the Bw parameter again. Consider
(8)xinew⟵{xi+r·(maxh∈[1,HMS]HM(h,i)-xi);ifr≥0.5xi-r·(xi-minh∈[1,HMS]HM(h,i));ifr<0.5ccccccccccccccccccccccccccccccccccccccccr~U[0,1].Although this adjustment enables each variable to avoid violating its boundary constraint, the bandwidth is larger; therefore, the convergence is very slow and can even stop.


## 2.2.3. The Self-Adaptive Harmony Search Proposed by Pan et al

Pan et al. [18] proposed a local-best harmony search algorithm with dynamic subpopulations (DLHS). In the variant, the whole harmony memory is equally divided into m subgroups. Moreover, to keep population diversity, the small-sized sub-HMs are regrouped once the period R is reached. In the variant, HMCR and PAR are from a parameter set list (PSL) with length Len, which is randomly generated by the initial value with a uniform distribution in the range PAR∈[0,1] and HMCR∈[0.9,1] and then updated once the list is empty. In the variant, the value of the parameter Bw is also dynamically changed:
(9)Bw(k)={Bwmax-Bwmax-BwminNI*2k;if2k<NI,Bwmin;otherwise.Although the variant takes the advantage of each local optimum, it consumes too much time as a cost. Thus, Pan et al. [17] proposed a self-adaptive global best harmony search algorithm (SGHS) based on the GHS algorithm. In the new variant, the value of HMCR (PAR) is normally distributed in the range [0.9,1]([0,1]) with mean HMCRm (PARm) and standard deviation 0.01 (0.05). The initial values of HMCRm and PARm are given by the user. In a fixed learning period (LP), the values of HMCRm and PARm do not change; after LP, the values are recalculated by averaging all HMCR and PAR values recorded during the previous LP. In the variant, Bw is still determined by (9).Obviously, the two variants require extra jobs to set the values of more parameters than the basic HS algorithm and other variants. Moreover, it is irrational that the value of PAR still varies in the interval[0,1] regardless of the number of iterations because a successful search should gradually converge and the value of PAR should be decreased with time to prevent oscillation.These defects inspired me to investigate how the key parameters of the HS algorithm do not depend on the problem’s feature again.


## 2.3. Variants Based on Hybridisation with Other Metaheuristics

To improve the performance of the original HS, many researchers have hybridised the algorithm with other meta-heuristics, such as GA, PSO, and ACO [19]. This hybridisation can be categorised into two approaches [20]: (1) integrating some components of other meta-heuristic algorithms into the HS structure, and conversely, (2) integrating some HS components into other meta-heuristic algorithm structures. Perhaps hybrid methods are better than single meta-heuristics.However, in my opinion, hybridisation will depart from the essential nature of the original heuristic and the resulting mimicry will also be nondescript. Thus, the correct way to investigate this should be by refining the essence of the imitated objects. This idea inspired me to improve the search mechanism of the original HS algorithm by capturing the deep essence of musical improvisation.


## 3. A Novel Self-Adaptive Harmony Search Algorithm

### 3.1. Design of the Control Parameters

The HS algorithm includes five parameters: harmony memory size (HMS), harmony memory considering rate (HMCR), pitch adjusting rate (PAR), fine-tuning bandwidth (Bw), and the number of improvisations (NI). Among these parameters, HMCR, PAR, and Bw are crucial for the convergence of HS.It should be noted that each improvisation of a musician is always accompanied by fine-tuning until the improvisation is finished. That is, it is necessary to adjust each variable value before the HS algorithm is ended. Therefore, it is reasonable that the value of PAR ought to equal one and senseless to consider this parameter again. For this reason, the proposed self-adaptive variant does not use the PAR parameter.The value of HMCR in the basic HS algorithm is fixed at its initialisation and does not change until the search ends. As the value of HMCR decreases, harmony memory is used less efficiently, the algorithm converges much more slowly, and more time is consumed. As the value of HMCR increases, harmony memory is used more efficiently, the method converges more rapidly, less time is consumed, and the HS is easily trapped in a local optimum. To enhance the power of the HS algorithm, HMCR is usually valued in the interval[0.9,0.99]. However, the reason for this is not revealed [21]. In fact, musicians depend on their memory more for intricate than simple music. This fact inspired me to construct the HMCR parameter based on the dimension of the problem to be solved. That is, the dimension of the problem is analogous to the complexity of the music,
(10)HMCR=1-1n+1,wheren denotes the dimension of the problem to be solved. Clearly, 0.5 < HMCR < 1; as the number of dimensions of the problem increases, the value of HMCR increases and harmony memory is used more frequently, and vice versa.It should be noted that musicians fine-tune their playing throughout their improvisation, and the tuning range is wider at the beginning and narrower as the music is completed. This phenomenon inspired me to construct a dynamic fine-tuning bandwidthbwi for each variable xi, with bounds lbi and ubi and iteration number k:
(11)bwi(k)={ubi-lbi100*(1-kNI)(k=1,2,…,NI);iffstd>0.00010.0001;otherwise,wherefstd=std(f(x1),f(x2),…,f(xHMS)) denotes the standard deviation for the objective function values in the current HM. The value of bwi decreases gradually as the iteration number increases and increases as the range of each decision variable increases. The domain of each decision variable corresponds to the pitch range of each musical instrument; the gradual decrease in the adjustment range as the iteration number increases corresponds to the gradual decrease of fine-tuning bandwidth with increasing improvisation number. (ub-lb)/100 is analogous to the unit of fine-tuning bandwidth, and 0.0001 represents the minimal adjustment. fstd≤0.0001 represents the slight difference between the merit of each harmony vector in HM. In fact, at the late stages of musical improvisation, the music is gradually mellowed and the fine-tuning range used is very narrow.


### 3.2. Design of the Self-Adaptive Mechanism

For every meta-heuristic algorithm, accelerating the convergence and preventing the premature convergence appear contradictory. To resolve this contradiction, the evolutionary mechanism of the HS algorithm has to be properly revised. It should be again noted that fine-tuning occurs throughout the whole process of musician improvisation and the fine-tuning range is wider at the early stage and narrower at later stages. Therefore, this phenomenon can be mimicked by the following evolutionary operations:(12)xi′⟵{xij+r2*bwi;(j∈{1,2,…,HMS})ifr<HMCRlbi+r1*(ubi-lbi)ifr≥HMCR&+r2*bwi;fstd>0.0001minjxij+r1*(maxjxij-minjxij)ifr≥HMCR&+r2*bwi;fstd≤0.0001,where random numbersr,r1~U[0,1];r2~U[-1,1]. The standard deviation for the objective function values fstd represents the whole merit of the new harmony vectors in the current HM. If the standard deviation is less than a given threshold 0.0001, that is, if fstd≤0.0001, then the difference between the function values is very small. If the iteration number is sufficiently large, then the distribution of the harmony vectors is centralised; thus, a new random value should be generated within the narrow range [minjxij,maxjxij] for rapid convergence.


### 3.3. Evolutionary Process of the NSHS Algorithm

On the basis of the novel parameter-setting method and self-adaptive evolutionary mechanism, the computational procedure of the proposed variant can be given, as shown in Algorithm1.Algorithm 1:Pseudo code of the NSHS algorithm.
NSHS Algorithm
Initialise the problem and algorithm parameters:
Define a standard fitness function f(x),xi∈[lbi,ubi](i=1,2,…,n) by (1)
Define the parameters HMCR by (10), Bw by (11), HMS and NI
Generate randomly an initial HM and calculate fstd
Search the best solution:
While (k≤NI) do
Generate a new harmony vector by the following loop
While (i≤n) do
First, generate a new value for every variable by the following rules:
If (r≤HMCR,r~U[0,1]), then choose uniformly a value from the current memory for variable
xi′=xij;j∈{1,2,…,HMS}
Else, generate randomly
If fstd>0.0001, then generate randomly in the range of [lbi,ubi]
xi′=lbi+r1*(ubi-lbi);r1~U[0,1]
Else, generate randomly in the range of [minjxij,maxjxij]
xi′=minjxij+r1*(maxjxij-minjxij);r1~U[0,1]
End if
End if
Second, adjust the new value of every variable by the following rules:
Iffstd>0.0001, then
xi′′=x′+ubi-lbi100*(1-kNI)*r2;r2~U[-1,1]
Else, generate randomly in the range of[minjxij,maxjxij]
xi′′=xi′+0.0001*r2;r2~U[-1,1]
End if
End while
Assess the merit of the new harmony vector and update the current harmony memory
Iff(xnew)<f(xworst)=maxjf(xj), then
Accept the new harmony vector and replace the worst vector in theHM with it.
Recalculate the standard deviation of the fitness function valuesfstd.
End if
End while
Determine the optimal solution in theHM by the minimum fitness.


## 3.1. Design of the Control Parameters

The HS algorithm includes five parameters: harmony memory size (HMS), harmony memory considering rate (HMCR), pitch adjusting rate (PAR), fine-tuning bandwidth (Bw), and the number of improvisations (NI). Among these parameters, HMCR, PAR, and Bw are crucial for the convergence of HS.It should be noted that each improvisation of a musician is always accompanied by fine-tuning until the improvisation is finished. That is, it is necessary to adjust each variable value before the HS algorithm is ended. Therefore, it is reasonable that the value of PAR ought to equal one and senseless to consider this parameter again. For this reason, the proposed self-adaptive variant does not use the PAR parameter.The value of HMCR in the basic HS algorithm is fixed at its initialisation and does not change until the search ends. As the value of HMCR decreases, harmony memory is used less efficiently, the algorithm converges much more slowly, and more time is consumed. As the value of HMCR increases, harmony memory is used more efficiently, the method converges more rapidly, less time is consumed, and the HS is easily trapped in a local optimum. To enhance the power of the HS algorithm, HMCR is usually valued in the interval[0.9,0.99]. However, the reason for this is not revealed [21]. In fact, musicians depend on their memory more for intricate than simple music. This fact inspired me to construct the HMCR parameter based on the dimension of the problem to be solved. That is, the dimension of the problem is analogous to the complexity of the music,
(10)HMCR=1-1n+1,wheren denotes the dimension of the problem to be solved. Clearly, 0.5 < HMCR < 1; as the number of dimensions of the problem increases, the value of HMCR increases and harmony memory is used more frequently, and vice versa.It should be noted that musicians fine-tune their playing throughout their improvisation, and the tuning range is wider at the beginning and narrower as the music is completed. This phenomenon inspired me to construct a dynamic fine-tuning bandwidthbwi for each variable xi, with bounds lbi and ubi and iteration number k:
(11)bwi(k)={ubi-lbi100*(1-kNI)(k=1,2,…,NI);iffstd>0.00010.0001;otherwise,wherefstd=std(f(x1),f(x2),…,f(xHMS)) denotes the standard deviation for the objective function values in the current HM. The value of bwi decreases gradually as the iteration number increases and increases as the range of each decision variable increases. The domain of each decision variable corresponds to the pitch range of each musical instrument; the gradual decrease in the adjustment range as the iteration number increases corresponds to the gradual decrease of fine-tuning bandwidth with increasing improvisation number. (ub-lb)/100 is analogous to the unit of fine-tuning bandwidth, and 0.0001 represents the minimal adjustment. fstd≤0.0001 represents the slight difference between the merit of each harmony vector in HM. In fact, at the late stages of musical improvisation, the music is gradually mellowed and the fine-tuning range used is very narrow.


## 3.2. Design of the Self-Adaptive Mechanism

For every meta-heuristic algorithm, accelerating the convergence and preventing the premature convergence appear contradictory. To resolve this contradiction, the evolutionary mechanism of the HS algorithm has to be properly revised. It should be again noted that fine-tuning occurs throughout the whole process of musician improvisation and the fine-tuning range is wider at the early stage and narrower at later stages. Therefore, this phenomenon can be mimicked by the following evolutionary operations:(12)xi′⟵{xij+r2*bwi;(j∈{1,2,…,HMS})ifr<HMCRlbi+r1*(ubi-lbi)ifr≥HMCR&+r2*bwi;fstd>0.0001minjxij+r1*(maxjxij-minjxij)ifr≥HMCR&+r2*bwi;fstd≤0.0001,where random numbersr,r1~U[0,1];r2~U[-1,1]. The standard deviation for the objective function values fstd represents the whole merit of the new harmony vectors in the current HM. If the standard deviation is less than a given threshold 0.0001, that is, if fstd≤0.0001, then the difference between the function values is very small. If the iteration number is sufficiently large, then the distribution of the harmony vectors is centralised; thus, a new random value should be generated within the narrow range [minjxij,maxjxij] for rapid convergence.


## 3.3. Evolutionary Process of the NSHS Algorithm

On the basis of the novel parameter-setting method and self-adaptive evolutionary mechanism, the computational procedure of the proposed variant can be given, as shown in Algorithm1.Algorithm 1:Pseudo code of the NSHS algorithm.
NSHS Algorithm
Initialise the problem and algorithm parameters:
Define a standard fitness function f(x),xi∈[lbi,ubi](i=1,2,…,n) by (1)
Define the parameters HMCR by (10), Bw by (11), HMS and NI
Generate randomly an initial HM and calculate fstd
Search the best solution:
While (k≤NI) do
Generate a new harmony vector by the following loop
While (i≤n) do
First, generate a new value for every variable by the following rules:
If (r≤HMCR,r~U[0,1]), then choose uniformly a value from the current memory for variable
xi′=xij;j∈{1,2,…,HMS}
Else, generate randomly
If fstd>0.0001, then generate randomly in the range of [lbi,ubi]
xi′=lbi+r1*(ubi-lbi);r1~U[0,1]
Else, generate randomly in the range of [minjxij,maxjxij]
xi′=minjxij+r1*(maxjxij-minjxij);r1~U[0,1]
End if
End if
Second, adjust the new value of every variable by the following rules:
Iffstd>0.0001, then
xi′′=x′+ubi-lbi100*(1-kNI)*r2;r2~U[-1,1]
Else, generate randomly in the range of[minjxij,maxjxij]
xi′′=xi′+0.0001*r2;r2~U[-1,1]
End if
End while
Assess the merit of the new harmony vector and update the current harmony memory
Iff(xnew)<f(xworst)=maxjf(xj), then
Accept the new harmony vector and replace the worst vector in theHM with it.
Recalculate the standard deviation of the fitness function valuesfstd.
End if
End while
Determine the optimal solution in theHM by the minimum fitness.


## 4. Benchmark Functions

This section lists the global minimization benchmark functions used to evaluate the performances of the proposed algorithm. The test suite comprises six well-known unimodal benchmark functions, six well-known multimodal benchmark functions, and six constrained optimization functions. To evaluate the robustness of a stochastic search algorithm, the twelve unconstrained benchmark functions are randomly shifted and orthogonally rotated, and four benchmark functions are with normal noise in fitness.


### 4.1. Unimodal Benchmark Functions

Table1 summarizes the shifted rotated unimodal benchmark functions. Figure 2 depicts their 3D shapes if the dimension is equal to two.Table 1
Six shifted rotated unimodal benchmark functions.
Name
Expression
Shifted rotated Cigar-Tablet's function
f
1
=
z
1
2
+
ln
8
·
z
n
2
+
ln
4
·
∑
i
=
2
n
-
1
z
i
2
+
bias
Shifted rotated Rosenbrock's function
f
2
=
∑
i
=
1
n
-
1
(
100
·
(
z
i
+
1
-
z
i
2
)
2
+
(
z
i
-
1
)
2
)
+
bias
Shifted rotated Schwefel's problem 1.2 with noise in fitness
f
3
=
∑
i
=
1
n
(
∑
j
=
1
i
z
j
)
2
(
1
+
0.4
*
|
N
(
0,1
)
|
)
+
bias
Shifted rotated Schwefel’s problem 2.21 with noise in fitness
f
4
=
max
i
{
|
z
i
|
}
(
1
+
0.4
*
|
N
(
0,1
)
|
)
+
bias
Shifted rotated Schwefel’s problem 2.22 with noise in fitness
f
5
=
(
∑
i
=
1
n
|
z
i
|
+
∏
i
=
1
n
|
z
i
|
)
(
1
+
0.4
*
|
N
(
0,1
)
|
)
+
bias
Shifted rotated sphere function with noise in fitness
f
6
=
∑
i
=
1
n
z
i
2
(
1
+
0.4
*
|
N
(
0,1
)
|
)
+
bias
*
z
=
(
x
-
o
)
M. x={x1,x2,…,xn} is a solution in the search range of [−100, 100]. o={o1,o2,…,on} is a random vector in the range of [−100, 100]. M is an identity orthogonal matrix. bias∈ℝ is also a random number. In this study, these biases are randomly given. They are equal to −140, 390, −450, −310, −180, and −450, respectively. For the second benchmark function, global optimum x*=o+1 and f*=bias; for other functions, global optimum x*=o and f*=bias.Six well-known unimodal benchmark function landscapes.
(a)
2D Cigar-Tablet's function
(b)
2D Rosenbrock's function
(c)
2D Schwefel's problem 1.2
(d)
2D Schwefel’s problem 2.21
(e)
2D Schwefel’s problem 2.22
(f)
2D Sphere function


### 4.2. Multimodal Benchmark Functions

Table2 summarizes the shifted rotated multimodal benchmark functions. Figure 3 depicts their 3D shapes if the dimension is equal to two.Table 2
Six well-known multimodal benchmark functions.
Name
Expression
Shifted rotated Ackley's function
f
7
=
-
20
exp
(
-
0.2
·
1
n
∑
i
=
1
n
z
i
2
)
-
exp
(
1
n
∑
i
=
1
n
cos
(
2
π
z
i
)
)
+
20
+
e
+
bias
Shifted rotated Bohachevsky's function
f
8
=
∑
i
=
1
n
-
1
(
z
i
2
+
2
z
i
+
1
2
-
0.3
cos
(
3
π
z
i
)
-
0.4
cos
(
4
π
z
i
+
1
)
+
0.7
)
+
bias
Shifted rotated Griewank's function
f
9
=
1
4000
∑
i
=
1
n
z
i
2
-
∏
i
=
1
n
cos
(
z
i
i
)
+
1
+
bias
Shifted rotated Rastrigin's function
f
10
=
∑
i
=
1
n
(
z
i
2
-
10
cos
(
2
π
z
i
)
+
10
)
+
bias
Shifted rotated Schaffer's function
f
11
=
∑
i
=
1
n
-
1
(
z
i
2
+
2
z
i
+
1
2
)
0.25
(
1
+
si
n
2
(
50
(
z
i
2
+
2
z
i
+
1
2
)
0.1
)
)
+
bias
Shifted rotated Schwefel’s problem 2.13
f
12
=
∑
i
=
1
n
(
∑
j
=
1
n
(
a
i
j
sin
α
j
+
b
i
j
cos
α
j
)
-
∑
j
=
1
n
(
a
i
j
sin
z
j
+
b
i
j
cos
z
j
)
)
+
bias
*
z
=
(
x
-
o
)
M. x={x1,x2,…,xn} is a solution in the search range of [−10, 10]. o={o1,o2,…,on} is a random vector in the range of [−10, 10]. M is an identity orthogonal matrix. bias∈ℝ is also a random number. In this study, these biases are randomly given. They are equal to −140, −310, −180, −330, 390, and −460, respectively. For the twelfth benchmark function, global optimum x*=α and f*=bias; for other functions, global optimum x*=o and f*=bias.Six well-known multimodal benchmark function landscapes.
(a)
2D Ackley's function
(b)
2D Bohachevsky's function
(c)
2D Griewank's function
(d)
2D Rastrigin's function
(e)
2D Schaffer's function
(f)
2D Schwefel’s problem 2.13


### 4.3. Constrained Benchmark Functions

The six constrained optimization functions have also been described in the literature [2]. Thus, we only give their corresponding penalty functions defined in this study. It should be noted that these penalty functions are composed of many penalty terms in order to precisely control every violation:
(13)f13=min{(x1-2)2+(x2-1)2},x1,x2∈[-10,10].Ifmax(abs(x)-10)>0, then f13+max(abs(x)-10)↦f13.Ifx1-2x2+1=g1(x)≠0, then f13+10abs(g1)↦f13.If-x12/4-x22+1=g2(x)<0, then f13-10g2↦f13.Forf13, the known best solution x*=(0.8229,0.9114) and f*=1.3935. Consider
(14)f14=min{(x12+x2-11)2+(x1+x22-7)2},cccccccccccccccccccccccccccccx1,x2∈[0,6].Ifmax(x-6)>0, then f14+max(x-6)↦f14.Ifmax(-x)>0, then f14+max(-x)↦f14.If4.84-(x1-0.05)2-(x2-2.5)2=g1(x)<0, then f14-100g1↦f14.Ifx12+(x2-2.5)2-4.84=g2(x)<0, then f14-g2↦f14.Forf14, the known best solution x*=(2.2468,2.3819) and f*=13.5909. Consider
(15)f15=min{5.357847x32+0.8356891x1x2cccccccc+37.293239x1-40792.141},x1∈[78,102],x2∈[33,45],x3,x4,x5∈[27,45].Ifx1<78, then abs(f15)-x1+78↦f15.Ifx1>102, then abs(f15)+x1-102↦f15.Ifx2<33, then abs(f15)-x2+33↦f15.Ifx2>45, then abs(f15)+x2-45↦f15.If27-max(x3,x4,x5)>0, then abs(f15)-max(x3,x4,x5)+27↦f15.Ifmax(x3,x4,x5)-45>0, then abs(f15)+max(x3,x4,x5)-45↦f15.If85.334407+0.0056858x2x5+0.0006262x1x4-0.002205x3x5=g1(x)<0, then abs(f15)-g1↦f15.If6.66593-0.0056858x2x5-0.0006262x1x4+0.002205x3x5=g2(x)<0, then abs(f15)-10g2↦f15.If-9.48751+0.0071317x2x5+0.0029955x1x2+0.0021813x32=g3(x)<0, then abs(f15)-g3↦f15.If29.48751-0.0071317x2x5-0.0029955x1x2-0.0021813x32=g4(x)<0, then abs(f15)-g4↦f15.If-10.699039+0.0047026x3x5+0.0012547x1x3+0.0019085x3x4=g5(x)<0 then abs(f15)-10000g5↦f15.If15.699039-0.0047026x3x5-0.0012547x1x3-0.0019085x3x4=g6(x)<0, then abs(f15)-g6↦f15.Forf15, the known best solution x*=(78,33,29.995,45,36.7764) and f*=-30665.5. Consider
(16)f15=min{(x1-10)2+5(x2-12)2+x34ccccccc+3(x4-11)2+10x56+7x62ccccccc+x74-4x6x7-10x6-8x7},ccccccccccccccccccccccxi∈[-10,10].Ifmax(abs(x)-10)>0, then f16+max(abs(x)-10)↦f16.If127-2x12-3x24-x3-4x42-5x5=g1(x)<0, then f16-100g1↦f16.If282-7x1-3x2-10x32-x4+x5=g2(x)<0, then f16-g2↦f16.If196-23x1-x22-6x62+8x7=g3(x)<0, then f16-g3↦f16.If-4x12-x22+3x1x2-2x32-5x6+11x7=g4(x)<0, then f16-100g4↦f16.Forf16, the known best solution x*=(2.3305,1.9513,-0.4775,4.3657, -0.6245,1.0381,1.5942) and f*=680.6301. Consider
(17)cccf17=min{x1+x2+x3},x1∈[100,10000],x2,x3∈[1000,10000],x4,x5,x6,x7,x8∈[10,1000].Ifx1<100, then f17+100(100-x1)↦f17.Ifx1>10000, then f17+x1-10000↦f17.If1000-max(x2,x3)>0, then f17+100(1000-max(x2,x3))↦f17.Ifmax(x2,x3)-10000>0, then f17+max(x2,x3)-10000↦f17.If10-max(x4,x5,x6,x7,x8)>0, then f17+10-max(x4,x5,x6,x7,x8)↦f17.Ifmax(x4,x5,x6,x7,x8)-1000>0, then f17+max(x4,x5,x6,x7,x8)-1000↦f17.If1-0.0025(x4+x6)=g1(x)<0, then f17-3500g1↦f17.If1-0.0025(x5+x7-x4)=g2(x)<0, then f17-6000g2↦f17.If1-0.01(x8-x5)=g3(x)<0, then f17-4000g3↦f17.Ifx1x6-833.33252x4-100x1+83333.333=g4(x)<0, then f17-g4↦f17.Ifx2x7-1250x5-x2x4+1250x4=g5(x)<0, then f17-0.1g5↦f17.Ifx3x8-x3x5+2500x5-1250000=g6(x)<0, then f17-g6↦f17.Forf17, the known best solution x*=(579.3167,1359.943,5110.071,182.0174,295.5985, 217.9799,286.4162,395.5979) and f*=7049.3309. Consider
(18)f18=min{x12+x22+x1x2-14x1-16x2ccccccccc+(x3-10)2+4(x4-5)2+(x5-3)2ccccccccc+2(x6-1)2+5x72+7(x8-11)2+2(x9-10)2cccccccccc+(x10-7)2+45},ccccccccccccccccccccccccccccxi∈[-10,10](i=1,2,…,10).Ifmax(abs(x)-10)>0, then f18+max(abs(x)-10)↦f18.If105-4x1-5x2+3x7-9x8=g1(x)<0, then f18-10g1↦f18.If-10x1+8x2+17x7-2x8=g2(x)<0, then f18-10g2↦f18.If8x1-2x2-5x9+2x10+12=g3(x)<0, then f18-10g3↦f18.If-3(x1-2)2-4(x2-3)2-2x32+7x4+120=g4(x)<0, then f18-10g4↦f18.If-5x12-8x2-(x3-6)2+2x4+40=g5(x)<0, then f18-10g5↦f18.If-x12-2(x2-2)2+2x1x2-14x5+6x6=g6(x)<0, then f18-10g6↦f18.If-0.5(x1-8)2+2(x2-4)2-3x52+x6+30=g7(x)<0, then f18-10g7↦f18.If3x1-6x2-12(x9-8)2+7x10=g8(x)<0, then f18-g8↦f18.Forf18, the known best solution x*=(2.1720,2.3637,8.7739,5.0960,0.9907,1.4306, 1.3216, 9.8287,8.2801,8.3760)and f*=24.3062.


## 4.1. Unimodal Benchmark Functions

Table1 summarizes the shifted rotated unimodal benchmark functions. Figure 2 depicts their 3D shapes if the dimension is equal to two.Table 1
Six shifted rotated unimodal benchmark functions.
Name
Expression
Shifted rotated Cigar-Tablet's function
f
1
=
z
1
2
+
ln
8
·
z
n
2
+
ln
4
·
∑
i
=
2
n
-
1
z
i
2
+
bias
Shifted rotated Rosenbrock's function
f
2
=
∑
i
=
1
n
-
1
(
100
·
(
z
i
+
1
-
z
i
2
)
2
+
(
z
i
-
1
)
2
)
+
bias
Shifted rotated Schwefel's problem 1.2 with noise in fitness
f
3
=
∑
i
=
1
n
(
∑
j
=
1
i
z
j
)
2
(
1
+
0.4
*
|
N
(
0,1
)
|
)
+
bias
Shifted rotated Schwefel’s problem 2.21 with noise in fitness
f
4
=
max
i
{
|
z
i
|
}
(
1
+
0.4
*
|
N
(
0,1
)
|
)
+
bias
Shifted rotated Schwefel’s problem 2.22 with noise in fitness
f
5
=
(
∑
i
=
1
n
|
z
i
|
+
∏
i
=
1
n
|
z
i
|
)
(
1
+
0.4
*
|
N
(
0,1
)
|
)
+
bias
Shifted rotated sphere function with noise in fitness
f
6
=
∑
i
=
1
n
z
i
2
(
1
+
0.4
*
|
N
(
0,1
)
|
)
+
bias
*
z
=
(
x
-
o
)
M. x={x1,x2,…,xn} is a solution in the search range of [−100, 100]. o={o1,o2,…,on} is a random vector in the range of [−100, 100]. M is an identity orthogonal matrix. bias∈ℝ is also a random number. In this study, these biases are randomly given. They are equal to −140, 390, −450, −310, −180, and −450, respectively. For the second benchmark function, global optimum x*=o+1 and f*=bias; for other functions, global optimum x*=o and f*=bias.Six well-known unimodal benchmark function landscapes.
(a)
2D Cigar-Tablet's function
(b)
2D Rosenbrock's function
(c)
2D Schwefel's problem 1.2
(d)
2D Schwefel’s problem 2.21
(e)
2D Schwefel’s problem 2.22
(f)
2D Sphere function


## 4.2. Multimodal Benchmark Functions

Table2 summarizes the shifted rotated multimodal benchmark functions. Figure 3 depicts their 3D shapes if the dimension is equal to two.Table 2
Six well-known multimodal benchmark functions.
Name
Expression
Shifted rotated Ackley's function
f
7
=
-
20
exp
(
-
0.2
·
1
n
∑
i
=
1
n
z
i
2
)
-
exp
(
1
n
∑
i
=
1
n
cos
(
2
π
z
i
)
)
+
20
+
e
+
bias
Shifted rotated Bohachevsky's function
f
8
=
∑
i
=
1
n
-
1
(
z
i
2
+
2
z
i
+
1
2
-
0.3
cos
(
3
π
z
i
)
-
0.4
cos
(
4
π
z
i
+
1
)
+
0.7
)
+
bias
Shifted rotated Griewank's function
f
9
=
1
4000
∑
i
=
1
n
z
i
2
-
∏
i
=
1
n
cos
(
z
i
i
)
+
1
+
bias
Shifted rotated Rastrigin's function
f
10
=
∑
i
=
1
n
(
z
i
2
-
10
cos
(
2
π
z
i
)
+
10
)
+
bias
Shifted rotated Schaffer's function
f
11
=
∑
i
=
1
n
-
1
(
z
i
2
+
2
z
i
+
1
2
)
0.25
(
1
+
si
n
2
(
50
(
z
i
2
+
2
z
i
+
1
2
)
0.1
)
)
+
bias
Shifted rotated Schwefel’s problem 2.13
f
12
=
∑
i
=
1
n
(
∑
j
=
1
n
(
a
i
j
sin
α
j
+
b
i
j
cos
α
j
)
-
∑
j
=
1
n
(
a
i
j
sin
z
j
+
b
i
j
cos
z
j
)
)
+
bias
*
z
=
(
x
-
o
)
M. x={x1,x2,…,xn} is a solution in the search range of [−10, 10]. o={o1,o2,…,on} is a random vector in the range of [−10, 10]. M is an identity orthogonal matrix. bias∈ℝ is also a random number. In this study, these biases are randomly given. They are equal to −140, −310, −180, −330, 390, and −460, respectively. For the twelfth benchmark function, global optimum x*=α and f*=bias; for other functions, global optimum x*=o and f*=bias.Six well-known multimodal benchmark function landscapes.
(a)
2D Ackley's function
(b)
2D Bohachevsky's function
(c)
2D Griewank's function
(d)
2D Rastrigin's function
(e)
2D Schaffer's function
(f)
2D Schwefel’s problem 2.13


## 4.3. Constrained Benchmark Functions

The six constrained optimization functions have also been described in the literature [2]. Thus, we only give their corresponding penalty functions defined in this study. It should be noted that these penalty functions are composed of many penalty terms in order to precisely control every violation:
(13)f13=min{(x1-2)2+(x2-1)2},x1,x2∈[-10,10].Ifmax(abs(x)-10)>0, then f13+max(abs(x)-10)↦f13.Ifx1-2x2+1=g1(x)≠0, then f13+10abs(g1)↦f13.If-x12/4-x22+1=g2(x)<0, then f13-10g2↦f13.Forf13, the known best solution x*=(0.8229,0.9114) and f*=1.3935. Consider
(14)f14=min{(x12+x2-11)2+(x1+x22-7)2},cccccccccccccccccccccccccccccx1,x2∈[0,6].Ifmax(x-6)>0, then f14+max(x-6)↦f14.Ifmax(-x)>0, then f14+max(-x)↦f14.If4.84-(x1-0.05)2-(x2-2.5)2=g1(x)<0, then f14-100g1↦f14.Ifx12+(x2-2.5)2-4.84=g2(x)<0, then f14-g2↦f14.Forf14, the known best solution x*=(2.2468,2.3819) and f*=13.5909. Consider
(15)f15=min{5.357847x32+0.8356891x1x2cccccccc+37.293239x1-40792.141},x1∈[78,102],x2∈[33,45],x3,x4,x5∈[27,45].Ifx1<78, then abs(f15)-x1+78↦f15.Ifx1>102, then abs(f15)+x1-102↦f15.Ifx2<33, then abs(f15)-x2+33↦f15.Ifx2>45, then abs(f15)+x2-45↦f15.If27-max(x3,x4,x5)>0, then abs(f15)-max(x3,x4,x5)+27↦f15.Ifmax(x3,x4,x5)-45>0, then abs(f15)+max(x3,x4,x5)-45↦f15.If85.334407+0.0056858x2x5+0.0006262x1x4-0.002205x3x5=g1(x)<0, then abs(f15)-g1↦f15.If6.66593-0.0056858x2x5-0.0006262x1x4+0.002205x3x5=g2(x)<0, then abs(f15)-10g2↦f15.If-9.48751+0.0071317x2x5+0.0029955x1x2+0.0021813x32=g3(x)<0, then abs(f15)-g3↦f15.If29.48751-0.0071317x2x5-0.0029955x1x2-0.0021813x32=g4(x)<0, then abs(f15)-g4↦f15.If-10.699039+0.0047026x3x5+0.0012547x1x3+0.0019085x3x4=g5(x)<0 then abs(f15)-10000g5↦f15.If15.699039-0.0047026x3x5-0.0012547x1x3-0.0019085x3x4=g6(x)<0, then abs(f15)-g6↦f15.Forf15, the known best solution x*=(78,33,29.995,45,36.7764) and f*=-30665.5. Consider
(16)f15=min{(x1-10)2+5(x2-12)2+x34ccccccc+3(x4-11)2+10x56+7x62ccccccc+x74-4x6x7-10x6-8x7},ccccccccccccccccccccccxi∈[-10,10].Ifmax(abs(x)-10)>0, then f16+max(abs(x)-10)↦f16.If127-2x12-3x24-x3-4x42-5x5=g1(x)<0, then f16-100g1↦f16.If282-7x1-3x2-10x32-x4+x5=g2(x)<0, then f16-g2↦f16.If196-23x1-x22-6x62+8x7=g3(x)<0, then f16-g3↦f16.If-4x12-x22+3x1x2-2x32-5x6+11x7=g4(x)<0, then f16-100g4↦f16.Forf16, the known best solution x*=(2.3305,1.9513,-0.4775,4.3657, -0.6245,1.0381,1.5942) and f*=680.6301. Consider
(17)cccf17=min{x1+x2+x3},x1∈[100,10000],x2,x3∈[1000,10000],x4,x5,x6,x7,x8∈[10,1000].Ifx1<100, then f17+100(100-x1)↦f17.Ifx1>10000, then f17+x1-10000↦f17.If1000-max(x2,x3)>0, then f17+100(1000-max(x2,x3))↦f17.Ifmax(x2,x3)-10000>0, then f17+max(x2,x3)-10000↦f17.If10-max(x4,x5,x6,x7,x8)>0, then f17+10-max(x4,x5,x6,x7,x8)↦f17.Ifmax(x4,x5,x6,x7,x8)-1000>0, then f17+max(x4,x5,x6,x7,x8)-1000↦f17.If1-0.0025(x4+x6)=g1(x)<0, then f17-3500g1↦f17.If1-0.0025(x5+x7-x4)=g2(x)<0, then f17-6000g2↦f17.If1-0.01(x8-x5)=g3(x)<0, then f17-4000g3↦f17.Ifx1x6-833.33252x4-100x1+83333.333=g4(x)<0, then f17-g4↦f17.Ifx2x7-1250x5-x2x4+1250x4=g5(x)<0, then f17-0.1g5↦f17.Ifx3x8-x3x5+2500x5-1250000=g6(x)<0, then f17-g6↦f17.Forf17, the known best solution x*=(579.3167,1359.943,5110.071,182.0174,295.5985, 217.9799,286.4162,395.5979) and f*=7049.3309. Consider
(18)f18=min{x12+x22+x1x2-14x1-16x2ccccccccc+(x3-10)2+4(x4-5)2+(x5-3)2ccccccccc+2(x6-1)2+5x72+7(x8-11)2+2(x9-10)2cccccccccc+(x10-7)2+45},ccccccccccccccccccccccccccccxi∈[-10,10](i=1,2,…,10).Ifmax(abs(x)-10)>0, then f18+max(abs(x)-10)↦f18.If105-4x1-5x2+3x7-9x8=g1(x)<0, then f18-10g1↦f18.If-10x1+8x2+17x7-2x8=g2(x)<0, then f18-10g2↦f18.If8x1-2x2-5x9+2x10+12=g3(x)<0, then f18-10g3↦f18.If-3(x1-2)2-4(x2-3)2-2x32+7x4+120=g4(x)<0, then f18-10g4↦f18.If-5x12-8x2-(x3-6)2+2x4+40=g5(x)<0, then f18-10g5↦f18.If-x12-2(x2-2)2+2x1x2-14x5+6x6=g6(x)<0, then f18-10g6↦f18.If-0.5(x1-8)2+2(x2-4)2-3x52+x6+30=g7(x)<0, then f18-10g7↦f18.If3x1-6x2-12(x9-8)2+7x10=g8(x)<0, then f18-g8↦f18.Forf18, the known best solution x*=(2.1720,2.3637,8.7739,5.0960,0.9907,1.4306, 1.3216, 9.8287,8.2801,8.3760)and f*=24.3062.


## 5. Robustness of the Proposed NSHS Algorithm

The performance of a stochastic search algorithm often is affected by the initial populations. For the HS algorithm, the initial population distribution is determined by the value of HMS and the initial search ranges. Thus, this section investigates the robustness of the proposed variant from the two aspects.


### 5.1. Effect of HMS

Yang [21] declared that the value of HMS is insensitive to the performance of the original HS algorithm. To reveal the fact that the value of HMS does not also have a significant impact on the performance of the NSHS, a full-factorial experiment was conducted. In the experiment, the HMS is valued at four levels (5, 10, 20, and 50) as a fixed factor; above twelve unconstrained benchmark functions with 10-dimension and six constrained functions were employed; the simulation was repeated 30 times, and each simulation underwent 50,000 evaluations for the objective function. The relative error between the running result and the global or known optimum is tested as a dependent variable. The descriptive statistical result of these errors is shown in Table 3.Table 3
The descriptive statistical result of the relative errors of the NSHS algorithm in the case of HMS = 5, 10, and 50, respectively.
Function
HMS = 5
HMS = 10
HMS = 20
HMS = 50
Mean
Std.
Mean
Std.
Mean
Std.
Mean
Std.
f1
7.54
E
-
07
4.24
E
-
07
4.65
E
-
07
2.53
E
-
07
1.00
E
-
06
5.24
E
-
07
5.37
E
-
06
1.69
E
-
06
f2
2.96
E
-
01
5.36
E
-
01
1.56
E
-
01
1.45
E
-
01
1.22
E
-
01
1.36
E
-
01
1.09
E
-
01
1.32
E
-
01
f3
2.74
E
-
06
1.72
E
-
06
4.92
E
-
06
3.28
E
-
06
1.48
E
-
03
8.05
E
-
03
1.98
E
-
03
8.55
E
-
03
f4
3.76
E
-
05
1.76
E
-
05
4.71
E
-
05
1.32
E
-
05
7.23
E
-
05
1.64
E
-
05
1.34
E
-
04
2.53
E
-
05
f5
1.32
E
-
04
5.49
E
-
05
1.41
E
-
04
3.03
E
-
05
2.09
E
-
04
4.82
E
-
05
4.17
E
-
04
8.23
E
-
05
f6
4.22
E
-
07
2.46
E
-
07
1.11
E
-
07
6.73
E
-
08
3.27
E
-
07
2.02
E
-
07
1.81
E
-
06
6.08
E
-
07
f7
3.38
E
-
06
1.46
E
-
06
6.21
E
-
06
2.01
E
-
06
1.07
E
-
05
3.03
E
-
06
2.07
E
-
05
4.22
E
-
06
f8
2.25
E
-
09
9.12
E
-
10
2.95
E
-
09
1.26
E
-
09
2.90
E
-
08
1.80
E
-
08
7.47
E
-
07
2.82
E
-
07
f9
8.00
E
-
05
8.44
E
-
05
1.07
E
-
04
7.34
E
-
05
9.62
E
-
05
6.45
E
-
05
3.74
E
-
05
3.95
E
-
05
f10
2.12
E
-
03
2.69
E
-
03
1.10
E
-
03
1.69
E
-
03
5.03
E
-
04
1.39
E
-
03
3.59
E
-
04
9.48
E
-
04
f11
3.73
E
-
04
4.75
E
-
05
3.81
E
-
04
4.29
E
-
05
4.76
E
-
04
3.89
E
-
05
6.59
E
-
04
5.76
E
-
05
f12
3.59
E
-
07
2.67
E
-
07
1.26
E
-
07
9.95
E
-
08
2.16
E
-
07
1.71
E
-
07
9.81
E
-
07
4.43
E
-
07
f13
2.08
E
-
04
1.61
E
-
04
2.77
E
-
04
2.07
E
-
04
3.24
E
-
04
1.79
E
-
04
5.51
E
-
04
3.38
E
-
04
f14
3.52
E
-
06
4.29
E
-
06
3.36
E
-
06
2.67
E
-
06
7.43
E
-
06
4.40
E
-
06
9.78
E
-
06
5.48
E
-
06
f15
2.47
E
-
05
6.67
E
-
05
4.60
E
-
05
1.91
E
-
04
7.40
E
-
05
1.95
E
-
04
1.86
E
-
03
1.56
E
-
03
f16
1.17
E
-
04
7.96
E
-
05
1.26
E
-
04
8.46
E
-
05
1.16
E
-
04
6.32
E
-
05
1.75
E
-
04
7.27
E
-
05
f17
2.89
E
-
02
2.92
E
-
02
2.09
E
-
02
2.72
E
-
02
2.63
E
-
02
2.48
E
-
02
2.96
E
-
02
3.21
E
-
02
f18
1.44
E
-
02
9.70
E
-
03
7.29
E
-
03
3.81
E
-
03
9.57
E
-
03
3.65
E
-
03
1.50
E
-
02
6.03
E
-
03
Total
1.90
E
-
02
1.42
E
-
01
1.04
E
-
02
4.95
E
-
02
8.96
E
-
03
4.28
E
-
02
8.85
E
-
03
4.05
E
-
02
*Each group has 30 samples and the experiment has totally 4 * 18 * 30 = 2160 samples.Clearly, there exists the tiny difference. However, we can claim that the difference is due to the function tested rather than the HMS by the test result of univariate analysis of variance, seeing the report of SPSS shown in  Table4. Therefore, the performance of the proposed NSHS algorithm is not affected by the value of HMS. To save storage space, it is encouraged to use a small HMS, as Pan et al. [17] explained that “since HM is emulating musician’s short-term memory and the short-term memory of the human is known to be small, it is logical.”Table 4
The report of SPSS on the univariate (relative error) analysis of variance with two fixed factors (problem/function and HMS).
(a)
Tests of between-subjects effects (dependent variable: relative error).
Source
Type III sum of squares
df
Mean square
F
Sig.
Corrected
3.958a
71
.056
11.537
.000
Model
.301
1
.301
62.211
.000
HMS
.038
3
.013
2.632
.048
Problem
3.293
17
.194
40.092
.000
HMS * Problem
.627
51
.012
2.542
.000
Error
10.089
2088
.005
Total
14.347
2160
CorrectedTotal
14.046
2159
aR squared = .282 (adjusted R squared = .257).
(b)
Hochberg's multiple comparisons (dependent variable: relative error).
(I) HMS
(J) HMS
Mean difference (I − J)
Std.
Sig.
5
10
.0086
.0042
.225
20
.0100
.0042
.101
50
.0102
.0042
.095
10
5
−.0086
.0042
.225
20
.0014
.0042
1.000
50
.0015
.0042
1.000
20
5
−.0100
.0042
.101
10
−.0014
.0042
1.000
50
.0001
.0042
1.000
50
5
−.0102
.0042
.095
10
−.0015
.0042
1.000
20
−.0001
.0042
1.000
Based on observed means.


### 5.2. Effect of Initial Search Ranges

To identify whether the performance of the proposed variant depends on the initial HM, a full-factorial experiment was conducted using the parameters HMS = 10 and NI = 50,000. In the experiment, the initial HM was randomly generated at three types of the initial search ranges, that is, the whole domain, the left part, and the right part. The experiment was repeated 30 times and 1620 observations were collected. The descriptive statistical result of the relative errors of the proposed variant is shown in Table5.Table 5
The descriptive statistical result of the relative errors of the NSHS algorithm using three types of the initial search ranges.
Function
The whole domain
The left part
The right part
Mean
Std.
Mean
Std.
Mean
Std.
f1
4.10
E
-
07
2.00
E
-
07
3.97
E
-
07
2.35
E
-
07
4.67
E
-
07
4.34
E
-
07
f2
7.96
E
-
01
2.34
E
+
00
6.64
E
-
01
2.30
E
+
00
1.20
E
-
01
1.90
E
-
01
f3
4.88
E
-
06
2.72
E
-
06
4.87
E
-
06
3.20
E
-
06
4.73
E
-
06
2.64
E
-
06
f4
4.62
E
-
05
1.16
E
-
05
4.75
E
-
05
1.20
E
-
05
4.77
E
-
05
1.40
E
-
05
f5
1.46
E
-
04
3.73
E
-
05
1.39
E
-
04
4.14
E
-
05
1.29
E
-
04
3.29
E
-
05
f6
1.12
E
-
07
7.03
E
-
08
1.48
E
-
07
1.68
E
-
07
1.21
E
-
07
6.96
E
-
08
f7
6.70
E
-
06
1.45
E
-
06
7.06
E
-
06
1.59
E
-
06
6.47
E
-
06
1.82
E
-
06
f8
4.08
E
-
09
4.51
E
-
09
4.22
E
-
09
6.34
E
-
09
9.48
E
-
09
1.96
E
-
08
f9
1.18
E
-
04
8.47
E
-
05
1.23
E
-
04
6.71
E
-
05
1.16
E
-
04
7.40
E
-
05
f10
1.49
E
-
03
2.44
E
-
03
1.44
E
-
03
2.08
E
-
03
5.38
E
-
04
1.14
E
-
03
f11
3.87
E
-
04
5.90
E
-
05
3.98
E
-
04
5.71
E
-
05
3.90
E
-
04
4.95
E
-
05
f12
1.84
E
-
07
1.64
E
-
07
1.49
E
-
07
1.39
E
-
07
1.90
E
-
07
2.35
E
-
07
f13
2.86
E
-
04
2.09
E
-
04
2.99
E
-
04
1.81
E
-
04
2.82
E
-
04
1.88
E
-
04
f14
6.86
E
-
06
5.36
E
-
06
5.06
E
-
06
3.60
E
-
06
5.53
E
-
06
5.02
E
-
06
f15
1.43
E
-
05
9.29
E
-
06
1.14
E
-
05
3.25
E
-
06
1.24
E
-
05
3.99
E
-
06
f16
1.01
E
-
04
5.61
E
-
05
9.12
E
-
05
3.53
E
-
05
1.24
E
-
04
8.51
E
-
05
f17
2.86
E
-
02
2.43
E
-
02
1.73
E
-
02
2.47
E
-
02
3.06
E
-
02
1.37
E
-
02
f18
8.42
E
-
03
4.12
E
-
03
9.03
E
-
03
3.92
E
-
03
8.45
E
-
03
3.72
E
-
03
Total
4.64
E
-
02
5.72
E
-
01
3.85
E
-
02
5.54
E
-
01
8.91
E
-
03
5.23
E
-
02
*Each group has 30 samples and the experiment has totally 3 * 18 * 30 = 1620 samples.Obviously, there still exists the small difference. However, the test report of SPSS shown in Table6 still indicates that the difference is due to the function tested rather than the initial search ranges.Table 6
The report of SPSS on the univariate (relative error) analysis of variance with two fixed factors (problem/function and initial range).
(a)
Tests of between-subjects effects (dependent variable: relative error).
Source
Sum of squares
df
Mean square
F
Sig.
Corrected
31.122a
53
.587
2.939
.000
Model
1.584
1
1.584
7.927
.005
Initial range
.422
2
.211
1.055
.348
Problem
23.416
17
1.377
6.895
.000
Initial range * problem
7.285
34
.214
1.073
.357
Error
312.834
1566
.200
Total
345.540
1620
Corrected
343.956
1619
aR squared = .090 (adjusted R squared = .060).
(b)
Hochberg's multiple comparisons (dependent variable: relative error).
(I) Initial range
(J) Initial range
Mean difference (I − J)
Std.
Sig.
Left
Right
.0296
.0272
.622
Whole
−.0079
.0272
.988
Right
Left
−.0296
.0272
.622
Whole
−.0375
.0272
.424
Whole
Left
.0079
.0272
.988
Right
.0375
.0272
.424
Based on observed means.By the results of analysis of variance as shown in Tables4 and 6, the performance of the proposed NSHS algorithm does not depend on the initial HM. By the descriptive statistical results as shown in Tables 3 and 5, moreover, no matter how the initial HM is, the NSHS algorithm can stability search for the global optima of these benchmark functions with a tiny error, except for function f2. Therefore, the novel variant is considerably robust.


## 5.1. Effect of HMS

Yang [21] declared that the value of HMS is insensitive to the performance of the original HS algorithm. To reveal the fact that the value of HMS does not also have a significant impact on the performance of the NSHS, a full-factorial experiment was conducted. In the experiment, the HMS is valued at four levels (5, 10, 20, and 50) as a fixed factor; above twelve unconstrained benchmark functions with 10-dimension and six constrained functions were employed; the simulation was repeated 30 times, and each simulation underwent 50,000 evaluations for the objective function. The relative error between the running result and the global or known optimum is tested as a dependent variable. The descriptive statistical result of these errors is shown in Table 3.Table 3
The descriptive statistical result of the relative errors of the NSHS algorithm in the case of HMS = 5, 10, and 50, respectively.
Function
HMS = 5
HMS = 10
HMS = 20
HMS = 50
Mean
Std.
Mean
Std.
Mean
Std.
Mean
Std.
f1
7.54
E
-
07
4.24
E
-
07
4.65
E
-
07
2.53
E
-
07
1.00
E
-
06
5.24
E
-
07
5.37
E
-
06
1.69
E
-
06
f2
2.96
E
-
01
5.36
E
-
01
1.56
E
-
01
1.45
E
-
01
1.22
E
-
01
1.36
E
-
01
1.09
E
-
01
1.32
E
-
01
f3
2.74
E
-
06
1.72
E
-
06
4.92
E
-
06
3.28
E
-
06
1.48
E
-
03
8.05
E
-
03
1.98
E
-
03
8.55
E
-
03
f4
3.76
E
-
05
1.76
E
-
05
4.71
E
-
05
1.32
E
-
05
7.23
E
-
05
1.64
E
-
05
1.34
E
-
04
2.53
E
-
05
f5
1.32
E
-
04
5.49
E
-
05
1.41
E
-
04
3.03
E
-
05
2.09
E
-
04
4.82
E
-
05
4.17
E
-
04
8.23
E
-
05
f6
4.22
E
-
07
2.46
E
-
07
1.11
E
-
07
6.73
E
-
08
3.27
E
-
07
2.02
E
-
07
1.81
E
-
06
6.08
E
-
07
f7
3.38
E
-
06
1.46
E
-
06
6.21
E
-
06
2.01
E
-
06
1.07
E
-
05
3.03
E
-
06
2.07
E
-
05
4.22
E
-
06
f8
2.25
E
-
09
9.12
E
-
10
2.95
E
-
09
1.26
E
-
09
2.90
E
-
08
1.80
E
-
08
7.47
E
-
07
2.82
E
-
07
f9
8.00
E
-
05
8.44
E
-
05
1.07
E
-
04
7.34
E
-
05
9.62
E
-
05
6.45
E
-
05
3.74
E
-
05
3.95
E
-
05
f10
2.12
E
-
03
2.69
E
-
03
1.10
E
-
03
1.69
E
-
03
5.03
E
-
04
1.39
E
-
03
3.59
E
-
04
9.48
E
-
04
f11
3.73
E
-
04
4.75
E
-
05
3.81
E
-
04
4.29
E
-
05
4.76
E
-
04
3.89
E
-
05
6.59
E
-
04
5.76
E
-
05
f12
3.59
E
-
07
2.67
E
-
07
1.26
E
-
07
9.95
E
-
08
2.16
E
-
07
1.71
E
-
07
9.81
E
-
07
4.43
E
-
07
f13
2.08
E
-
04
1.61
E
-
04
2.77
E
-
04
2.07
E
-
04
3.24
E
-
04
1.79
E
-
04
5.51
E
-
04
3.38
E
-
04
f14
3.52
E
-
06
4.29
E
-
06
3.36
E
-
06
2.67
E
-
06
7.43
E
-
06
4.40
E
-
06
9.78
E
-
06
5.48
E
-
06
f15
2.47
E
-
05
6.67
E
-
05
4.60
E
-
05
1.91
E
-
04
7.40
E
-
05
1.95
E
-
04
1.86
E
-
03
1.56
E
-
03
f16
1.17
E
-
04
7.96
E
-
05
1.26
E
-
04
8.46
E
-
05
1.16
E
-
04
6.32
E
-
05
1.75
E
-
04
7.27
E
-
05
f17
2.89
E
-
02
2.92
E
-
02
2.09
E
-
02
2.72
E
-
02
2.63
E
-
02
2.48
E
-
02
2.96
E
-
02
3.21
E
-
02
f18
1.44
E
-
02
9.70
E
-
03
7.29
E
-
03
3.81
E
-
03
9.57
E
-
03
3.65
E
-
03
1.50
E
-
02
6.03
E
-
03
Total
1.90
E
-
02
1.42
E
-
01
1.04
E
-
02
4.95
E
-
02
8.96
E
-
03
4.28
E
-
02
8.85
E
-
03
4.05
E
-
02
*Each group has 30 samples and the experiment has totally 4 * 18 * 30 = 2160 samples.Clearly, there exists the tiny difference. However, we can claim that the difference is due to the function tested rather than the HMS by the test result of univariate analysis of variance, seeing the report of SPSS shown in  Table4. Therefore, the performance of the proposed NSHS algorithm is not affected by the value of HMS. To save storage space, it is encouraged to use a small HMS, as Pan et al. [17] explained that “since HM is emulating musician’s short-term memory and the short-term memory of the human is known to be small, it is logical.”Table 4
The report of SPSS on the univariate (relative error) analysis of variance with two fixed factors (problem/function and HMS).
(a)
Tests of between-subjects effects (dependent variable: relative error).
Source
Type III sum of squares
df
Mean square
F
Sig.
Corrected
3.958a
71
.056
11.537
.000
Model
.301
1
.301
62.211
.000
HMS
.038
3
.013
2.632
.048
Problem
3.293
17
.194
40.092
.000
HMS * Problem
.627
51
.012
2.542
.000
Error
10.089
2088
.005
Total
14.347
2160
CorrectedTotal
14.046
2159
aR squared = .282 (adjusted R squared = .257).
(b)
Hochberg's multiple comparisons (dependent variable: relative error).
(I) HMS
(J) HMS
Mean difference (I − J)
Std.
Sig.
5
10
.0086
.0042
.225
20
.0100
.0042
.101
50
.0102
.0042
.095
10
5
−.0086
.0042
.225
20
.0014
.0042
1.000
50
.0015
.0042
1.000
20
5
−.0100
.0042
.101
10
−.0014
.0042
1.000
50
.0001
.0042
1.000
50
5
−.0102
.0042
.095
10
−.0015
.0042
1.000
20
−.0001
.0042
1.000
Based on observed means.


## 5.2. Effect of Initial Search Ranges

To identify whether the performance of the proposed variant depends on the initial HM, a full-factorial experiment was conducted using the parameters HMS = 10 and NI = 50,000. In the experiment, the initial HM was randomly generated at three types of the initial search ranges, that is, the whole domain, the left part, and the right part. The experiment was repeated 30 times and 1620 observations were collected. The descriptive statistical result of the relative errors of the proposed variant is shown in Table5.Table 5
The descriptive statistical result of the relative errors of the NSHS algorithm using three types of the initial search ranges.
Function
The whole domain
The left part
The right part
Mean
Std.
Mean
Std.
Mean
Std.
f1
4.10
E
-
07
2.00
E
-
07
3.97
E
-
07
2.35
E
-
07
4.67
E
-
07
4.34
E
-
07
f2
7.96
E
-
01
2.34
E
+
00
6.64
E
-
01
2.30
E
+
00
1.20
E
-
01
1.90
E
-
01
f3
4.88
E
-
06
2.72
E
-
06
4.87
E
-
06
3.20
E
-
06
4.73
E
-
06
2.64
E
-
06
f4
4.62
E
-
05
1.16
E
-
05
4.75
E
-
05
1.20
E
-
05
4.77
E
-
05
1.40
E
-
05
f5
1.46
E
-
04
3.73
E
-
05
1.39
E
-
04
4.14
E
-
05
1.29
E
-
04
3.29
E
-
05
f6
1.12
E
-
07
7.03
E
-
08
1.48
E
-
07
1.68
E
-
07
1.21
E
-
07
6.96
E
-
08
f7
6.70
E
-
06
1.45
E
-
06
7.06
E
-
06
1.59
E
-
06
6.47
E
-
06
1.82
E
-
06
f8
4.08
E
-
09
4.51
E
-
09
4.22
E
-
09
6.34
E
-
09
9.48
E
-
09
1.96
E
-
08
f9
1.18
E
-
04
8.47
E
-
05
1.23
E
-
04
6.71
E
-
05
1.16
E
-
04
7.40
E
-
05
f10
1.49
E
-
03
2.44
E
-
03
1.44
E
-
03
2.08
E
-
03
5.38
E
-
04
1.14
E
-
03
f11
3.87
E
-
04
5.90
E
-
05
3.98
E
-
04
5.71
E
-
05
3.90
E
-
04
4.95
E
-
05
f12
1.84
E
-
07
1.64
E
-
07
1.49
E
-
07
1.39
E
-
07
1.90
E
-
07
2.35
E
-
07
f13
2.86
E
-
04
2.09
E
-
04
2.99
E
-
04
1.81
E
-
04
2.82
E
-
04
1.88
E
-
04
f14
6.86
E
-
06
5.36
E
-
06
5.06
E
-
06
3.60
E
-
06
5.53
E
-
06
5.02
E
-
06
f15
1.43
E
-
05
9.29
E
-
06
1.14
E
-
05
3.25
E
-
06
1.24
E
-
05
3.99
E
-
06
f16
1.01
E
-
04
5.61
E
-
05
9.12
E
-
05
3.53
E
-
05
1.24
E
-
04
8.51
E
-
05
f17
2.86
E
-
02
2.43
E
-
02
1.73
E
-
02
2.47
E
-
02
3.06
E
-
02
1.37
E
-
02
f18
8.42
E
-
03
4.12
E
-
03
9.03
E
-
03
3.92
E
-
03
8.45
E
-
03
3.72
E
-
03
Total
4.64
E
-
02
5.72
E
-
01
3.85
E
-
02
5.54
E
-
01
8.91
E
-
03
5.23
E
-
02
*Each group has 30 samples and the experiment has totally 3 * 18 * 30 = 1620 samples.Obviously, there still exists the small difference. However, the test report of SPSS shown in Table6 still indicates that the difference is due to the function tested rather than the initial search ranges.Table 6
The report of SPSS on the univariate (relative error) analysis of variance with two fixed factors (problem/function and initial range).
(a)
Tests of between-subjects effects (dependent variable: relative error).
Source
Sum of squares
df
Mean square
F
Sig.
Corrected
31.122a
53
.587
2.939
.000
Model
1.584
1
1.584
7.927
.005
Initial range
.422
2
.211
1.055
.348
Problem
23.416
17
1.377
6.895
.000
Initial range * problem
7.285
34
.214
1.073
.357
Error
312.834
1566
.200
Total
345.540
1620
Corrected
343.956
1619
aR squared = .090 (adjusted R squared = .060).
(b)
Hochberg's multiple comparisons (dependent variable: relative error).
(I) Initial range
(J) Initial range
Mean difference (I − J)
Std.
Sig.
Left
Right
.0296
.0272
.622
Whole
−.0079
.0272
.988
Right
Left
−.0296
.0272
.622
Whole
−.0375
.0272
.424
Whole
Left
.0079
.0272
.988
Right
.0375
.0272
.424
Based on observed means.By the results of analysis of variance as shown in Tables4 and 6, the performance of the proposed NSHS algorithm does not depend on the initial HM. By the descriptive statistical results as shown in Tables 3 and 5, moreover, no matter how the initial HM is, the NSHS algorithm can stability search for the global optima of these benchmark functions with a tiny error, except for function f2. Therefore, the novel variant is considerably robust.


## 6. Superiority of the Proposed NSHS Algorithm

To confirm the significant superiority of the proposed variant, a comparison test was conducted. The test compared the performance of the NSHS with the original HS algorithm [1] and the other variants mentioned in Section 2, including Mahdavi’s IHS [14], Wang’s self-adaptive variant represented by HSw for convenience [16], Pan’s DLHS [18], and SGHS [17]. For fair comparison, the parameter settings are those proposed by their developers, as shown in Table 7.Table 7
Algorithm parameters.
Algorithm
HMS
HMCR
PAR
Bw
NI
Others
HS
9
0.9
0.3
0.01
50,000
None
IHS
9
0.9
Min: 0.01; max: 0.99
Min: 0.0001; max: 1/(20(ub − lb))
50,000
None
HSw
9
0.9
Min: 0.01; max: 0.99
Needless
50,000
None
DLHS
9
Initial: 0.91
Initail: 0.5
Min: 0.0001; max: (ub − lb)/200
50,000
m
=
3; R=50: len = 200
SGHS
9
Initial: 0.98
Initial: 0.9
Min: 0.0005; max: (ub − lb)/10
50,000
LP = 100
NSHS*
9
1 − 1/(n+1)
Needless
No initial value, using (11)
50,000
None
*NSHS has only two key parameters HMCR and Bw, but they do not require the user's inputs.For sufficiently persuasive comparative results, every algorithm was run independently 30 times to solve the aforementioned 18 benchmark functions on the same computer (CPU: AMD 4 × 3.5 Ghz; RAM: 4 GB), and, at the beginning of solving a problem, each algorithm used the same initial HM. The conclusion is made based on an analysis of variance rather than a pairwiset-test because Garc’s study yet indicated that the latter method is suitable for a single-problem analysis and a multiple-problem analysis should employ a nonparametric test [22]. Next, the experimental results are elaborated from two aspects, respectively.


### 6.1. Unconstrained Benchmark Function

The performances of these algorithms regarding twelve unconstrained benchmark functions were investigated at different scales, that is, 10, 100, and 1000. Table8, Table 10 and Table 12 respectively summarize the descriptive statistical results at different scales. For 10-dimension benchmark functions, the superiority of the NSHS is not significant except for functions f3 and f4, as shown in Table 8. The test result shown in Table 9 indicates that the performance of the NSHS is significant better than the IHS at the 95% confidence level and the average error of the NSHS is less than other algorithms except for the DLHS. For 100-dimension benchmark functions, the average error of the NSHS is much less than other algorithms except for the SGHS, as shown in Table 11. Based on the data in Tables 12 and 13, we can see that the superiority of the proposed variant is extremely significant for 1000-dimension benchmark functions.Table 8
The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 10 dimensions.
Function
HS
IHS
HSw
DLHS
SGHS
NSHS
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
f1
1.02
E
-
09 (7.27E-10)
4.77
E
-
04 (3.90E-04)
1.10
E
-
04 (1.39E-04)
1.96
E
-
12 (1.20E-12)
6.40
E
-
13 (4.98E-13)
4.49
E
-
07 (3.42E-07)
f2
2.17
E
-
01 (7.90E-01)
4.47
E
-
01 (5.54E-01)
1.77
E
-
01 (1.77E-01)
2.36
E
-
01 (3.18E-01)
4.50
E
-
01 (1.50E+00)
4.94
E
-
01 (9.18E-01)
f3
7.08
E
-
01 (4.88E-01)
8.61
E
-
01 (5.77E-01)
8.14
E
-
01 (5.33E-01)
1.64
E
-
02 (8.96E-02)
2.59
E
-
01 (3.38E-01)
5.11
E
-
06 (2.91E-06)
f4
1.49
E
-
02 (7.17E-03)
1.37
E
-
02 (9.75E-03)
6.97
E
-
03 (2.31E-03)
1.12
E
-
03 (3.48E-03)
1.94
E
-
02 (2.14E-02)
4.22
E
-
05 (1.22E-05)
f5
6.15
E
-
03  (2.77E-03)
1.13
E
-
02 (3.65E-03)
2.50
E
-
03 (1.17E-03)
2.05
E
-
07 (6.51E-08)
1.87
E
-
03 (1.14E-03)
1.19
E
-
04 (4.72E-05)
f6
2.13
E
-
04 (2.84E-04)
8.78
E
-
04 (6.27E-04)
1.61
E
-
04 (1.69E-04)
4.25
E
-
13 (2.80E-13)
1.31
E
-
05 (1.48E-05)
1.39
E
-
07 (7.43E-08)
f7
1.74
E
-
06 (6.23E-07)
7.56
E
-
07 (9.01E-08)
8.43
E
-
05 (3.82E-05)
8.71
E
-
08 (4.24E-08)
5.24
E
-
08 (1.25E-08)
6.00
E
-
06 (1.43E-06)
f8
9.05
E
-
09 (5.01E-09)
9.92
E
-
10 (3.60E-10)
1.89
E
-
05 (2.00E-05)
1.78
E
-
11 (1.47E-11)
5.61
E
-
12 (4.12E-12)
3.10
E
-
09 (2.39E-09)
f9
7.64
E
-
05 (6.26E-05)
7.37
E
-
05 (6.07E-05)
5.74
E
-
05 (4.79E-05)
5.06
E
-
05 (3.89E-05)
2.01
E
-
04 (1.08E-04)
1.13
E
-
04 (9.19E-05)
f10
5.38
E
-
08 (4.63E-08)
4.37
E
-
09 (1.28E-09)
1.15
E
-
04 (9.04E-05)
6.86
E
-
11 (5.53E-11)
2.63
E
-
11 (1.73E-11)
5.47
E
-
04 (1.37E-03)
f11
2.92
E
-
03 (3.01E-03)
2.61
E
-
03 (3.48E-03)
1.13
E
-
03 (4.56E-04)
1.97
E
-
04 (7.40E-04)
3.94
E
-
05 (4.99E-06)
3.99
E
-
04 (5.66E-05)
f12
1.04
E
-
10 (6.02E-11)
2.21
E
-
12 (2.58E-12)
4.51
E
-
08 (3.87E-08)
4.54
E
-
13 (1.06E-12)
8.38
E
-
14 (6.21E-14)
1.58
E
-
07 (1.37E-07)
Total
7.91
E
-
02 (3.30E-01)
1.11
E
-
01 (3.43E-01)
8.36
E
-
02 (2.77E-01)
2.12
E
-
02 (1.14E-01)
6.08
E
-
02 (4.57E-01)
4.12
E
-
02 (2.94E-01)
*Each group has 30 samples and the experiment has totally 12 * 6 * 30 = 2160 samples. The bold represents the significant best.Table 9
Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 10 dimensions.
(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.
HS
NSHS
.03782732242345
.0199
.103
IHS
NSHS
.07017638607941*
.0199
.001
HSw
NSHS
.04233406887345
.0199
.064
DLHS
NSHS
−.02008968262494
.0199
.986
SGHS
NSHS
.01957666996786
.0199
.421
*The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).Table 10
The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 100 dimensions.
Function
HS
IHS
HSw
DLHS
SGHS
NSHS
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
f1
2.30
E
+
02 (2.34E+01)
2.38
E
+
02 (3.29E+01)
2.06
E
+
02 (2.57E+01)
1.30
E
+
02 (3.60E+01)
5.23
E
-
02 (8.03E-03)
1.56
E
-
03 (3.58E-04)
f2
8.24
E
+
06 (1.82E+06)
7.97
E
+
06 (1.77E+06)
5.60
E
+
06 (1.14E+06)
2.21
E
+
06 (1.08E+06)
4.91
E
+
00 (2.32E+00)
2.44
E
+
00 (7.03E+00)
f3
8.94
E
+
02 (1.30E+02)
9.46
E
+
02 (1.04E+02)
8.68
E
+
02 (1.29E+02)
5.79
E
+
02 (7.58E+01)
7.99
E
+
02 (1.66E+02)
8.04
E
+
02 (1.83E+02)
f4
3.27
E
-
01 (2.30E-02)
3.22
E
-
01 (2.08E-02)
2.43
E
-
01 (1.91E-02)
2.69
E
-
01 (2.65E-02)
4.22
E
-
01 (4.47E-02)
4.24
E
-
01 (3.39E-02)
f5
2.09
E
+
10 (7.98E+10)
1.50
E
+
12 (8.23E+12)
6.56
E
+
00 (7.28E-01)
3.84
E
+
00 (6.54E-01)
1.38
E
+
00 (2.12E-01)
4.78
E
+
00 (1.05E+00)
f6
9.32
E
+
01 (1.32E+01)
8.88
E
+
01 (9.45E+00)
8.59
E
+
01 (1.28E+01)
6.79
E
+
01 (1.38E+01)
2.40
E
+
01 (4.48E+00)
1.19
E
+
00 (2.37E+00)
f7
4.93
E
-
02 (1.90E-03)
4.88
E
-
02 (2.15E-03)
4.79
E
-
02 (1.87E-03)
4.21
E
-
02 (4.51E-03)
3.60
E
-
04 (2.36E-04)
5.01
E
-
04 (1.20E-03)
f8
2.63
E
+
00 (2.85E-01)
2.66
E
+
00 (2.34E-01)
2.38
E
+
00 (2.72E-01)
1.50
E
+
00 (4.14E-01)
1.23
E
-
02 (3.98E-03)
1.10
E
-
02 (5.63E-03)
f9
5.86
E
-
03 (8.71E-05)
5.87
E
-
03 (5.18E-05)
5.74
E
-
03 (1.10E-04)
4.46
E
-
03 (9.65E-04)
3.07
E
-
05 (5.04E-05)
1.77
E
-
05 (2.40E-05)
f10
2.40
E
+
00 (1.83E-01)
2.41
E
+
00 (1.66E-01)
2.69
E
+
00 (1.59E-01)
1.52
E
+
00 (4.14E-01)
1.54
E
-
01 (2.06E-02)
3.35
E
-
01 (3.89E-02)
f11
4.62
E
-
01 (3.03E-02)
4.26
E
-
01 (2.40E-02)
4.59
E
-
01 (1.79E-02)
3.89
E
-
01 (3.40E-02)
1.24
E
-
01 (1.64E-02)
5.57
E
-
01 (1.08E-01)
f12
1.32
E
-
02 (1.59E-03)
1.26
E
-
02 (1.73E-03)
3.86
E
-
02 (3.18E-03)
7.20
E
-
03 (2.15E-03)
4.73
E
-
05 (2.23E-05)
2.93
E
-
04 (6.85E-05)
Total
1.74
E
+
09 (2.34E+10)
1.25
E
+
11 (2.37E+12)
4.67
E
+
05 (1.58E+06)
1.85
E
+
05 (6.86E+05)
6.92
E
+
01 (2.25E+02)
6.78
E
+
01 (2.28E+02)
*Each group has 30 samples and the experiment has totally 12 * 6 * 30 = 2160 samples. The bold represents the significant best.Table 11
Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 100 dimensions.
(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.
HS
NSHS
1743032079.82376
72262361658.159
.826
IHS
NSHS
125173553098.6880
72262361658.159
.143
HSw
NSHS
466806.998460054
72262361658.159
.833
DLHS
NSHS
184496.101783522
72262361658.159
.833
SGHS
NSHS
1.39707526033672
72262361658.159
.833
*The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).Table 12
The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 1000 dimensions.
Function
HS
IHS
HSw
DLHS
SGHS
NSHS
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
f1
2.65
E
+
04 (5.35E+02)
2.63
E
+
04 (5.15E+02)
2.85
E
+
04 (6.46E+02)
2.63
E
+
04 (1.76E+03)
1.77
E
+
03 (1.33E+02)
7.93
E
+
00 (5.36E-01)
f2
6.34
E
+
09 (3.62E+08)
6.41
E
+
09 (2.85E+08)
6.43
E
+
09 (3.01E+08)
6.31
E
+
09 (6.22E+08)
5.04
E
+
07 (3.26E+06)
6.91
E
+
03 (1.40E+03)
f3
1.16
E
+
05 (1.38E+04)
1.17
E
+
05 (1.34E+04)
1.10
E
+
05 (9.12E+03)
8.30
E
+
04 (6.59E+03)
7.01
E
+
04 (1.02E+04)
6.86
E
+
04 (7.45E+03)
f4
5.58
E
-
01 (7.02E-03)
5.56
E
-
01 (9.08E-03)
5.18
E
-
01 (6.71E-03)
5.43
E
-
01 (7.79E-03)
5.86
E
-
01 (1.19E-02)
5.72
E
-
01 (1.16E-02)
f5
2.61
E
+
00 (3.17E-02)
2.61
E
+
00 (5.05E-02)
2.69
E
+
00 (5.58E-02)
2.60
E
+
00 (5.30E-02)
2.69
E
+
00 (4.96E-02)
3.15
E
+
00 (6.74E-02)
f6
7.83
E
+
03 (2.54E+02)
7.97
E
+
03 (2.55E+02)
8.08
E
+
03 (2.23E+02)
7.78
E
+
03 (3.95E+02)
7.72
E
+
03 (2.70E+02)
1.02
E
+
04 (4.90E+02)
f7
1.04
E
-
01 (5.97E-04)
1.04
E
-
01 (5.02E-04)
1.06
E
-
01 (9.92E-04)
1.04
E
-
01 (1.23E-03)
5.69
E
-
02 (9.70E-04)
4.47
E
-
02 (1.37E-03)
f8
2.61
E
+
02 (5.43E+00)
2.60
E
+
02 (6.21E+00)
2.79
E
+
02 (7.31E+00)
2.60
E
+
02 (1.43E+01)
1.95
E
+
01 (1.32E+00)
2.07
E
+
00 (5.15E-02)
f9
4.28
E
-
02 (1.01E-03)
4.26
E
-
02 (1.18E-03)
4.51
E
-
02 (9.60E-04)
4.26
E
-
02 (2.36E-03)
7.97
E
-
03 (1.54E-04)
5.12
E
-
04 (3.80E-05)
f10
1.10
E
+
02 (1.95E+00)
1.10
E
+
02 (2.46E+00)
1.17
E
+
02 (3.02E+00)
1.12
E
+
02 (5.36E+00)
3.26
E
+
01 (1.09E+00)
2.54
E
+
01 (8.27E-01)
f11
1.02
E
+
01 (1.06E-01)
9.92
E
+
00 (9.23E-02)
1.04
E
+
01 (9.89E-02)
1.00
E
+
01 (1.54E-01)
6.68
E
+
00 (2.17E-01)
1.19
E
+
01 (1.03E-01)
f12
1.70
E
+
00 (3.57E-02)
1.68
E
+
00 (3.93E-02)
2.27
E
+
00 (5.14E-02)
1.73
E
+
00 (1.12E-01)
3.21
E
-
01 (2.46E-02)
1.59
E
-
01 (7.18E-03)
Total
5.29
E
+
08 (1.76E+09)
5.34
E
+
08 (1.77E+09)
5.36
E
+
08 (1.78E+09)
5.26
E
+
08 (1.76E+09)
4.21
E
+
06 (1.40E+07)
7.14
E
+
03 (1.90E+04)
*Each group has 30 samples and the experiment has totally 12 * 6 * 30 = 2160 samples. For a solvability, the search ranges of function f5 are renewed in [−1, 1]because 100∧1000 reaches an infinity even 10∧1000. The bold represents the significant best.Table 13
Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 1000 dimensions.
(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.
HS
NSHS
528495067.3354575
*
7295157.814
.000
IHS
NSHS
533757007.9854518
*
7295157.814
.000
HSw
NSHS
536111617.63627493
*
7295157.814
.000
DLHS
NSHS
525985111.0408457
*
7295157.814
.000
SGHS
NSHS
4198894.56625653
7295157.814
.611
*The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).


### 6.2. Constrained Benchmark Function

The performances of these algorithms with respect to the aforementioned optimization problems with constraints were investigated. The descriptive statistical result and the report on multiple comparisons are, respectively, shown in Tables14 and 15. Clearly, the performance of the proposed variant is vastly superior to the HS, the IHS, the HSw, and the SGHS. Although the performance of the NSHS is slightly better than the DLHS, the evolutionary mechanism of the former is simpler and the NSHS does not require too much key-parameters setting. Moreover, the performance of the NSHS is significantly better than the DLHS for functions f15 and f18, seeing Table 14.Table 14
The descriptive statistical result of the relative errors of the six algorithms to solve six constrained benchmark functions.
Function
HS
IHS
HSw
DLHS
SGHS
NSHS
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
f13
3.22
E
-
01 (5.19E-01)
7.81
E
-
01 (1.04E+00)
7.68
E
-
01 (8.69E-01)
-
2.22
E
-
05 (2.46E-06)
7.09
E
-
02 (1.10E-01)
2.44
E
-
04 (1.87E-04)
f14
1.91
E
-
05 (2.45E-05)
1.71
E
-
04 (1.72E-04)
4.59
E
-
03 (2.51E-03)
1.84
E
-
06 (2.72E-06)
2.59
E
-
03 (2.38E-03)
4.49
E
-
06 (4.70E-06)
f15
8.72
E
-
03 (4.86E-03)
8.72
E
-
03 (5.13E-03)
1.06
E
-
02 (4.14E-03)
3.35
E
-
03 (2.98E-03)
1.16
E
-
02 (5.63E-03)
1.16
E
-
05 (3.20E-06)
f16
8.44
E
-
03 (9.40E-03)
2.84
E
-
02 (2.04E-02)
1.80
E
-
02 (1.23E-02)
1.34
E
-
04 (1.29E-04)
6.31
E
-
03 (6.17E-03)
1.0
0
E
-
04 (6.43E-05)
f17
6.81
E
-
02 (7.64E-02)
7.68
E
-
02 (8.20E-02)
8.24
E
-
02 (4.25E-02)
3.93
E
-
02 (3.71E-02)
1.02
E
-
01 (7.23E-02)
2.48
E
-
02 (2.14E-02)
f18
2.81
E
-
01 (1.81E-01)
5.93
E
-
01 (4.37E-01)
3.76
E
-
01 (2.15E-01)
7.49
E
-
02 (5.50E-02)
2.55
E
-
01 (1.82E-01)
7.98
E
-
03 (3.76E-03)
Total
1.16
E
-
01 (2.63E-01)
2.53
E
-
01 (5.62E-01)
2.12
E
-
01 (4.62E-01)
1.95
E
-
02 (3.91E-02)
7.47
E
-
02 (1.29E-01)
5.20
E
-
03 (1.22E-02)
*The experiment collected 6 * 6 * 30 = 1080 observations, 23 out of whom are invalid. For function f15, the DLHS got 2 infeasible solutions. For function f17, the HS got 4 infeasible solutions and the IHS 5, the HSw 3, the DLHS 1, the SGHS 5, and the NSHS 3, respectively. The bold represents the significant best.Table 15
Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving six constrained benchmark functions.
(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.
HS
NSHS
.11043867790257
*
.028
.000
IHS
NSHS
.24772610041475
*
.028
.000
HSw
NSHS
.20691985495254
*
.028
.000
DLHS
NSHS
.01430413405475
.028
.640
SGHS
NSHS
.06953695639486
*
.028
.028
*The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).


## 6.1. Unconstrained Benchmark Function

The performances of these algorithms regarding twelve unconstrained benchmark functions were investigated at different scales, that is, 10, 100, and 1000. Table8, Table 10 and Table 12 respectively summarize the descriptive statistical results at different scales. For 10-dimension benchmark functions, the superiority of the NSHS is not significant except for functions f3 and f4, as shown in Table 8. The test result shown in Table 9 indicates that the performance of the NSHS is significant better than the IHS at the 95% confidence level and the average error of the NSHS is less than other algorithms except for the DLHS. For 100-dimension benchmark functions, the average error of the NSHS is much less than other algorithms except for the SGHS, as shown in Table 11. Based on the data in Tables 12 and 13, we can see that the superiority of the proposed variant is extremely significant for 1000-dimension benchmark functions.Table 8
The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 10 dimensions.
Function
HS
IHS
HSw
DLHS
SGHS
NSHS
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
f1
1.02
E
-
09 (7.27E-10)
4.77
E
-
04 (3.90E-04)
1.10
E
-
04 (1.39E-04)
1.96
E
-
12 (1.20E-12)
6.40
E
-
13 (4.98E-13)
4.49
E
-
07 (3.42E-07)
f2
2.17
E
-
01 (7.90E-01)
4.47
E
-
01 (5.54E-01)
1.77
E
-
01 (1.77E-01)
2.36
E
-
01 (3.18E-01)
4.50
E
-
01 (1.50E+00)
4.94
E
-
01 (9.18E-01)
f3
7.08
E
-
01 (4.88E-01)
8.61
E
-
01 (5.77E-01)
8.14
E
-
01 (5.33E-01)
1.64
E
-
02 (8.96E-02)
2.59
E
-
01 (3.38E-01)
5.11
E
-
06 (2.91E-06)
f4
1.49
E
-
02 (7.17E-03)
1.37
E
-
02 (9.75E-03)
6.97
E
-
03 (2.31E-03)
1.12
E
-
03 (3.48E-03)
1.94
E
-
02 (2.14E-02)
4.22
E
-
05 (1.22E-05)
f5
6.15
E
-
03  (2.77E-03)
1.13
E
-
02 (3.65E-03)
2.50
E
-
03 (1.17E-03)
2.05
E
-
07 (6.51E-08)
1.87
E
-
03 (1.14E-03)
1.19
E
-
04 (4.72E-05)
f6
2.13
E
-
04 (2.84E-04)
8.78
E
-
04 (6.27E-04)
1.61
E
-
04 (1.69E-04)
4.25
E
-
13 (2.80E-13)
1.31
E
-
05 (1.48E-05)
1.39
E
-
07 (7.43E-08)
f7
1.74
E
-
06 (6.23E-07)
7.56
E
-
07 (9.01E-08)
8.43
E
-
05 (3.82E-05)
8.71
E
-
08 (4.24E-08)
5.24
E
-
08 (1.25E-08)
6.00
E
-
06 (1.43E-06)
f8
9.05
E
-
09 (5.01E-09)
9.92
E
-
10 (3.60E-10)
1.89
E
-
05 (2.00E-05)
1.78
E
-
11 (1.47E-11)
5.61
E
-
12 (4.12E-12)
3.10
E
-
09 (2.39E-09)
f9
7.64
E
-
05 (6.26E-05)
7.37
E
-
05 (6.07E-05)
5.74
E
-
05 (4.79E-05)
5.06
E
-
05 (3.89E-05)
2.01
E
-
04 (1.08E-04)
1.13
E
-
04 (9.19E-05)
f10
5.38
E
-
08 (4.63E-08)
4.37
E
-
09 (1.28E-09)
1.15
E
-
04 (9.04E-05)
6.86
E
-
11 (5.53E-11)
2.63
E
-
11 (1.73E-11)
5.47
E
-
04 (1.37E-03)
f11
2.92
E
-
03 (3.01E-03)
2.61
E
-
03 (3.48E-03)
1.13
E
-
03 (4.56E-04)
1.97
E
-
04 (7.40E-04)
3.94
E
-
05 (4.99E-06)
3.99
E
-
04 (5.66E-05)
f12
1.04
E
-
10 (6.02E-11)
2.21
E
-
12 (2.58E-12)
4.51
E
-
08 (3.87E-08)
4.54
E
-
13 (1.06E-12)
8.38
E
-
14 (6.21E-14)
1.58
E
-
07 (1.37E-07)
Total
7.91
E
-
02 (3.30E-01)
1.11
E
-
01 (3.43E-01)
8.36
E
-
02 (2.77E-01)
2.12
E
-
02 (1.14E-01)
6.08
E
-
02 (4.57E-01)
4.12
E
-
02 (2.94E-01)
*Each group has 30 samples and the experiment has totally 12 * 6 * 30 = 2160 samples. The bold represents the significant best.Table 9
Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 10 dimensions.
(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.
HS
NSHS
.03782732242345
.0199
.103
IHS
NSHS
.07017638607941*
.0199
.001
HSw
NSHS
.04233406887345
.0199
.064
DLHS
NSHS
−.02008968262494
.0199
.986
SGHS
NSHS
.01957666996786
.0199
.421
*The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).Table 10
The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 100 dimensions.
Function
HS
IHS
HSw
DLHS
SGHS
NSHS
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
f1
2.30
E
+
02 (2.34E+01)
2.38
E
+
02 (3.29E+01)
2.06
E
+
02 (2.57E+01)
1.30
E
+
02 (3.60E+01)
5.23
E
-
02 (8.03E-03)
1.56
E
-
03 (3.58E-04)
f2
8.24
E
+
06 (1.82E+06)
7.97
E
+
06 (1.77E+06)
5.60
E
+
06 (1.14E+06)
2.21
E
+
06 (1.08E+06)
4.91
E
+
00 (2.32E+00)
2.44
E
+
00 (7.03E+00)
f3
8.94
E
+
02 (1.30E+02)
9.46
E
+
02 (1.04E+02)
8.68
E
+
02 (1.29E+02)
5.79
E
+
02 (7.58E+01)
7.99
E
+
02 (1.66E+02)
8.04
E
+
02 (1.83E+02)
f4
3.27
E
-
01 (2.30E-02)
3.22
E
-
01 (2.08E-02)
2.43
E
-
01 (1.91E-02)
2.69
E
-
01 (2.65E-02)
4.22
E
-
01 (4.47E-02)
4.24
E
-
01 (3.39E-02)
f5
2.09
E
+
10 (7.98E+10)
1.50
E
+
12 (8.23E+12)
6.56
E
+
00 (7.28E-01)
3.84
E
+
00 (6.54E-01)
1.38
E
+
00 (2.12E-01)
4.78
E
+
00 (1.05E+00)
f6
9.32
E
+
01 (1.32E+01)
8.88
E
+
01 (9.45E+00)
8.59
E
+
01 (1.28E+01)
6.79
E
+
01 (1.38E+01)
2.40
E
+
01 (4.48E+00)
1.19
E
+
00 (2.37E+00)
f7
4.93
E
-
02 (1.90E-03)
4.88
E
-
02 (2.15E-03)
4.79
E
-
02 (1.87E-03)
4.21
E
-
02 (4.51E-03)
3.60
E
-
04 (2.36E-04)
5.01
E
-
04 (1.20E-03)
f8
2.63
E
+
00 (2.85E-01)
2.66
E
+
00 (2.34E-01)
2.38
E
+
00 (2.72E-01)
1.50
E
+
00 (4.14E-01)
1.23
E
-
02 (3.98E-03)
1.10
E
-
02 (5.63E-03)
f9
5.86
E
-
03 (8.71E-05)
5.87
E
-
03 (5.18E-05)
5.74
E
-
03 (1.10E-04)
4.46
E
-
03 (9.65E-04)
3.07
E
-
05 (5.04E-05)
1.77
E
-
05 (2.40E-05)
f10
2.40
E
+
00 (1.83E-01)
2.41
E
+
00 (1.66E-01)
2.69
E
+
00 (1.59E-01)
1.52
E
+
00 (4.14E-01)
1.54
E
-
01 (2.06E-02)
3.35
E
-
01 (3.89E-02)
f11
4.62
E
-
01 (3.03E-02)
4.26
E
-
01 (2.40E-02)
4.59
E
-
01 (1.79E-02)
3.89
E
-
01 (3.40E-02)
1.24
E
-
01 (1.64E-02)
5.57
E
-
01 (1.08E-01)
f12
1.32
E
-
02 (1.59E-03)
1.26
E
-
02 (1.73E-03)
3.86
E
-
02 (3.18E-03)
7.20
E
-
03 (2.15E-03)
4.73
E
-
05 (2.23E-05)
2.93
E
-
04 (6.85E-05)
Total
1.74
E
+
09 (2.34E+10)
1.25
E
+
11 (2.37E+12)
4.67
E
+
05 (1.58E+06)
1.85
E
+
05 (6.86E+05)
6.92
E
+
01 (2.25E+02)
6.78
E
+
01 (2.28E+02)
*Each group has 30 samples and the experiment has totally 12 * 6 * 30 = 2160 samples. The bold represents the significant best.Table 11
Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 100 dimensions.
(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.
HS
NSHS
1743032079.82376
72262361658.159
.826
IHS
NSHS
125173553098.6880
72262361658.159
.143
HSw
NSHS
466806.998460054
72262361658.159
.833
DLHS
NSHS
184496.101783522
72262361658.159
.833
SGHS
NSHS
1.39707526033672
72262361658.159
.833
*The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).Table 12
The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 1000 dimensions.
Function
HS
IHS
HSw
DLHS
SGHS
NSHS
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
f1
2.65
E
+
04 (5.35E+02)
2.63
E
+
04 (5.15E+02)
2.85
E
+
04 (6.46E+02)
2.63
E
+
04 (1.76E+03)
1.77
E
+
03 (1.33E+02)
7.93
E
+
00 (5.36E-01)
f2
6.34
E
+
09 (3.62E+08)
6.41
E
+
09 (2.85E+08)
6.43
E
+
09 (3.01E+08)
6.31
E
+
09 (6.22E+08)
5.04
E
+
07 (3.26E+06)
6.91
E
+
03 (1.40E+03)
f3
1.16
E
+
05 (1.38E+04)
1.17
E
+
05 (1.34E+04)
1.10
E
+
05 (9.12E+03)
8.30
E
+
04 (6.59E+03)
7.01
E
+
04 (1.02E+04)
6.86
E
+
04 (7.45E+03)
f4
5.58
E
-
01 (7.02E-03)
5.56
E
-
01 (9.08E-03)
5.18
E
-
01 (6.71E-03)
5.43
E
-
01 (7.79E-03)
5.86
E
-
01 (1.19E-02)
5.72
E
-
01 (1.16E-02)
f5
2.61
E
+
00 (3.17E-02)
2.61
E
+
00 (5.05E-02)
2.69
E
+
00 (5.58E-02)
2.60
E
+
00 (5.30E-02)
2.69
E
+
00 (4.96E-02)
3.15
E
+
00 (6.74E-02)
f6
7.83
E
+
03 (2.54E+02)
7.97
E
+
03 (2.55E+02)
8.08
E
+
03 (2.23E+02)
7.78
E
+
03 (3.95E+02)
7.72
E
+
03 (2.70E+02)
1.02
E
+
04 (4.90E+02)
f7
1.04
E
-
01 (5.97E-04)
1.04
E
-
01 (5.02E-04)
1.06
E
-
01 (9.92E-04)
1.04
E
-
01 (1.23E-03)
5.69
E
-
02 (9.70E-04)
4.47
E
-
02 (1.37E-03)
f8
2.61
E
+
02 (5.43E+00)
2.60
E
+
02 (6.21E+00)
2.79
E
+
02 (7.31E+00)
2.60
E
+
02 (1.43E+01)
1.95
E
+
01 (1.32E+00)
2.07
E
+
00 (5.15E-02)
f9
4.28
E
-
02 (1.01E-03)
4.26
E
-
02 (1.18E-03)
4.51
E
-
02 (9.60E-04)
4.26
E
-
02 (2.36E-03)
7.97
E
-
03 (1.54E-04)
5.12
E
-
04 (3.80E-05)
f10
1.10
E
+
02 (1.95E+00)
1.10
E
+
02 (2.46E+00)
1.17
E
+
02 (3.02E+00)
1.12
E
+
02 (5.36E+00)
3.26
E
+
01 (1.09E+00)
2.54
E
+
01 (8.27E-01)
f11
1.02
E
+
01 (1.06E-01)
9.92
E
+
00 (9.23E-02)
1.04
E
+
01 (9.89E-02)
1.00
E
+
01 (1.54E-01)
6.68
E
+
00 (2.17E-01)
1.19
E
+
01 (1.03E-01)
f12
1.70
E
+
00 (3.57E-02)
1.68
E
+
00 (3.93E-02)
2.27
E
+
00 (5.14E-02)
1.73
E
+
00 (1.12E-01)
3.21
E
-
01 (2.46E-02)
1.59
E
-
01 (7.18E-03)
Total
5.29
E
+
08 (1.76E+09)
5.34
E
+
08 (1.77E+09)
5.36
E
+
08 (1.78E+09)
5.26
E
+
08 (1.76E+09)
4.21
E
+
06 (1.40E+07)
7.14
E
+
03 (1.90E+04)
*Each group has 30 samples and the experiment has totally 12 * 6 * 30 = 2160 samples. For a solvability, the search ranges of function f5 are renewed in [−1, 1]because 100∧1000 reaches an infinity even 10∧1000. The bold represents the significant best.Table 13
Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 1000 dimensions.
(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.
HS
NSHS
528495067.3354575
*
7295157.814
.000
IHS
NSHS
533757007.9854518
*
7295157.814
.000
HSw
NSHS
536111617.63627493
*
7295157.814
.000
DLHS
NSHS
525985111.0408457
*
7295157.814
.000
SGHS
NSHS
4198894.56625653
7295157.814
.611
*The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).


## 6.2. Constrained Benchmark Function

The performances of these algorithms with respect to the aforementioned optimization problems with constraints were investigated. The descriptive statistical result and the report on multiple comparisons are, respectively, shown in Tables14 and 15. Clearly, the performance of the proposed variant is vastly superior to the HS, the IHS, the HSw, and the SGHS. Although the performance of the NSHS is slightly better than the DLHS, the evolutionary mechanism of the former is simpler and the NSHS does not require too much key-parameters setting. Moreover, the performance of the NSHS is significantly better than the DLHS for functions f15 and f18, seeing Table 14.Table 14
The descriptive statistical result of the relative errors of the six algorithms to solve six constrained benchmark functions.
Function
HS
IHS
HSw
DLHS
SGHS
NSHS
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
f13
3.22
E
-
01 (5.19E-01)
7.81
E
-
01 (1.04E+00)
7.68
E
-
01 (8.69E-01)
-
2.22
E
-
05 (2.46E-06)
7.09
E
-
02 (1.10E-01)
2.44
E
-
04 (1.87E-04)
f14
1.91
E
-
05 (2.45E-05)
1.71
E
-
04 (1.72E-04)
4.59
E
-
03 (2.51E-03)
1.84
E
-
06 (2.72E-06)
2.59
E
-
03 (2.38E-03)
4.49
E
-
06 (4.70E-06)
f15
8.72
E
-
03 (4.86E-03)
8.72
E
-
03 (5.13E-03)
1.06
E
-
02 (4.14E-03)
3.35
E
-
03 (2.98E-03)
1.16
E
-
02 (5.63E-03)
1.16
E
-
05 (3.20E-06)
f16
8.44
E
-
03 (9.40E-03)
2.84
E
-
02 (2.04E-02)
1.80
E
-
02 (1.23E-02)
1.34
E
-
04 (1.29E-04)
6.31
E
-
03 (6.17E-03)
1.0
0
E
-
04 (6.43E-05)
f17
6.81
E
-
02 (7.64E-02)
7.68
E
-
02 (8.20E-02)
8.24
E
-
02 (4.25E-02)
3.93
E
-
02 (3.71E-02)
1.02
E
-
01 (7.23E-02)
2.48
E
-
02 (2.14E-02)
f18
2.81
E
-
01 (1.81E-01)
5.93
E
-
01 (4.37E-01)
3.76
E
-
01 (2.15E-01)
7.49
E
-
02 (5.50E-02)
2.55
E
-
01 (1.82E-01)
7.98
E
-
03 (3.76E-03)
Total
1.16
E
-
01 (2.63E-01)
2.53
E
-
01 (5.62E-01)
2.12
E
-
01 (4.62E-01)
1.95
E
-
02 (3.91E-02)
7.47
E
-
02 (1.29E-01)
5.20
E
-
03 (1.22E-02)
*The experiment collected 6 * 6 * 30 = 1080 observations, 23 out of whom are invalid. For function f15, the DLHS got 2 infeasible solutions. For function f17, the HS got 4 infeasible solutions and the IHS 5, the HSw 3, the DLHS 1, the SGHS 5, and the NSHS 3, respectively. The bold represents the significant best.Table 15
Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving six constrained benchmark functions.
(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.
HS
NSHS
.11043867790257
*
.028
.000
IHS
NSHS
.24772610041475
*
.028
.000
HSw
NSHS
.20691985495254
*
.028
.000
DLHS
NSHS
.01430413405475
.028
.640
SGHS
NSHS
.06953695639486
*
.028
.028
*The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).


## 7. Conclusion

This paper presented a novel self-adaptive harmony search algorithm (NSHS) for optimization problems with continuous variables with the aim of improving the performance of the harmony search. To investigate the performance of the novel variant, many comparison experiments were conducted based on a test suite comprising six unimodal benchmark functions, six multimodal benchmark functions, and six constrained benchmark functions. Using nonparametric statistical tests, the NSHS’s robustness and superiority were detected. Experimental results on the effect of HMS and initial search ranges indicated that the performance of the NSHS did not depend on the initial HM. The results, when compared with related previous work, manifested that the NSHS performed dramatically better than other six algorithms for large-scale problems and constrained problems.The originality of this work is mainly reflected in two aspects.(i)
Proposed a simpler approach to the setting of key parameters based on musical practices. The proposed variant has only two key parameters HMCR and Bw, but they do not require the user’s inputs. Moreover, the parameters settings are in accordance with the features of the optimization problem to be solved, such as the scale and the boundaries. Moreover, this approach discards the PAR parameter and further simplifies the parameter setting. Musical improvisation is usually accompanied by fine-tuning; thus, it is senseless to consider this parameter and fine-tuning bandwidth at the same time.(ii)
Developed a novel self-adaptive evolutionary scheme based on the standard deviation of the fitness function values in the memory to accelerate algorithm convergence and assure the reaching of a global optimum. This scheme automatically narrows the search space based on a suitable threshold and, meanwhile, solves the problem of the optimal balance between search diversity and intensity.



---

*Source: 653749-2013-12-22.xml*