# Solving Unconstrained Global Optimization Problems via Hybrid Swarm Intelligence Approaches

Authors: Jui-Yu Wu
Journal: Mathematical Problems in Engineering
 (2013)

---

## Abstract

Stochastic global optimization (SGO) algorithms such as the particle swarm optimization (PSO) approach have become popular for solving unconstrained global optimization (UGO) problems. The PSO approach, which belongs to the swarm intelligence domain, does not require gradient information, enabling it to overcome this limitation of traditional nonlinear programming methods. Unfortunately, PSO algorithm implementation and performance depend on several parameters, such as cognitive parameter, social parameter, and constriction coefficient. These parameters are tuned by using trial and error. To reduce the parametrization of a PSO method, this work presents two efficient hybrid SGO approaches, namely, a real-coded genetic algorithm-based PSO (RGA-PSO) method and an artificial immune algorithm-based PSO (AIA-PSO) method. The specific parameters of the internal PSO algorithm are optimized using the external RGA and AIA approaches, and then the internal PSO algorithm is applied to solve UGO problems. The performances of the proposed RGA-PSO and AIA-PSO algorithms are then evaluated using a set of benchmark UGO problems. Numerical results indicate that, besides their ability to converge to a global minimum for each test UGO problem, the proposed RGA-PSO and AIA-PSO algorithms outperform many hybrid SGO algorithms. Thus, the RGA-PSO and AIA-PSO approaches can be considered alternative SGO approaches for solving standard-dimensional UGO problems.


---

## Body

## 1. Introduction

An unconstrained global optimization (UGO) problem can generally be formulated as follows:(1)Minimizef(x),x=[x1,x2,…,xN]T∈ℜN,
where f(x) is an objective function and x represents a decision variable vector. Additionally, x∈S, S⊆ℜN  denotes search space (S), which is N dimensional and bounded by parametric constraints as follows:
(2)xnl≤xn≤xnu,n=1,2,…,N,
where xnl and xnu are the lower and upper boundaries of the decision variables xn, respectively.Many conventional nonlinear programming (NLP) techniques, such as the golden search, quadratic approximation, Nelder-Mead, steepest descent, Newton, and conjugate gradient methods, have been used to solve UGO problems [1]. Unfortunately, such NLP methods have difficulty in solving UGO problems when an objective function of an UGO problem is nondifferential. Many stochastic global optimization (SGO) approaches developed to overcome this limitation of the traditional NLP methods include genetic algorithms (GAs), particle swarm optimization (PSO), ant colony optimization (ACO), and artificial immune algorithms (AIAs). For instance, Hamzaçebi [2] developed an enhanced GA incorporating a local random search algorithm for eight continuous functions. Furthermore, Chen [3] presented a two-layer PSO method to solve nine UGO problems. Zhao [4] presented a perturbed PSO approach for 12 UGO problems. Meanwhile, Toksari [5] developed an ACO algorithm for solving UGO problems. Finally, Kelsey and Timmis [6] presented an AIA method based on the clonal selection principle for solving 12 UGO problems.This work focuses on a PSO algorithm, based on it is being effective, robust and easy to use in the SGO methods. Research on the PSO method has considered many critical issues such as parameter selection, integration of the PSO algorithm with the approaches of self-adaptation, and integration with other intelligent optimizing methods [7]. This work surveys two issues: first is a PSO approach that integrates with other intelligent optimizing methods and second is parameter selection for use in a PSO approach.Regarding the first issue, the conventional PSO algorithm lacks evolution operators of GAs, such as crossover and mutation operations. Therefore, PSO has premature convergence, that is, a rapid loss of diversity during optimization [4]. To overcome this limitation, many hybrid SGO methods have been developed to create diverse candidate solutions to enhance the performance of a PSO approach. Hybrid algorithms have some advantages; for instance, hybrid algorithms outperform individual algorithms in solving certain problems and thus can solve general problems more efficiently [8]. Kao and Zahara [9] presented a hybrid GA and PSO algorithm to solve 17 multimodal test functions. Their study used the operations of GA and PSO methods to generate candidate solutions to improve solution quality and convergence rates. Furthermore, Shelokar et al. [10] presented a hybrid PSO and ACO algorithm to solve multimodal continuous optimization problems. Their study used an ACO algorithm to update the particle positions to enhance a PSO algorithm performance. Chen et al. [11] presented a hybrid PSO and external optimization based on the Bak–Sneppen model to solve unimodal and multimodal benchmark problems. Furthermore, Thangaraj et al. [12] surveyed many algorithms that combine the PSO algorithm with other search techniques and compared the performances obtained using hybrid differential evolution PSO (DE-PSO), adaptive mutation PSO (AMPSO), and hybrid GA and PSO (GA-PSO) approaches to solve nine conventional benchmark problems.Regarding the second issue, a PSO algorithm has numerous parameters that must be set, such as cognitive parameter, social parameter, inertia weight, and constriction coefficient. Traditionally, the optimal parameter settings of a PSO algorithm are tuned based on trial and error. The abilities of a PSO algorithm to explore and exploit are constrained to optimum parameter settings [13, 14]. Therefore, Jiang et al. [15] used a stochastic process theory to analyze the parameter settings (e.g., cognitive parameter, social parameter, and inertia weight) of a standard PSO algorithm.This work focuses on the second issue related to the application of a PSO method. Fortunately, the optimization of parameter settings for a PSO algorithm can be viewed as an UGO problem. Moreover, real-coded GA (RGA) and AIA are efficient SGO approaches for solving UGO problems. Based on the advantage of a hybrid algorithm [8], this work develops two hybrid SGO approaches. The first approach is a hybrid RGA and PSO (RGA-PSO) algorithm, while the second one is a hybrid AIA and PSO (AIA-PSO) algorithm. The proposed RGA-PSO and AIA-PSO algorithms are considered as a means of solving the two optimization problems simultaneously. The first UGO problem (optimization of cognitive parameter, social parameter, and constriction coefficient) is optimized using external RGA and AIA approaches, respectively. The second UGO problem is then solved using the internal PSO algorithm. The performances of the proposed RGA-PSO and AIA-PSO algorithms are evaluated using a set of benchmark UGO problems and compared with those of many hybrid algorithms [9, 10, 12].The rest of this paper is organized as follows. Section2 describes RGA, PSO, and AIA approaches. Section 3 then presents the proposed RGA-PSO and AIA-PSO methods. Next, Section 4 compares the experimental results of the proposed RGA-PSO and AIA-PSO approaches with those of many hybrid methods. Conclusions are finally drawn in Section 5.


## 2. Related Works

The SGO approaches such as RGA, PSO, and AIA [16] are described as follows.


### 2.1. Real-Coded Genetic Algorithm

GAs are based on the concepts of natural selection and use three genetic operations, that is, selection, crossover, and mutation, to explore and exploit the solution space. In solving continuous function optimization problems, RGA method outperforms binary-coded GA approach [17]. Therefore, this work describes operators of a RGA method [18].


#### 2.1.1. Selection Operation

A selection operation picks up strong individuals from a current population based on their fitness function values and then reproduces these individuals into a crossover pool. Many selection operations developed include the roulette wheel, the ranking, and the tournament methods [17, 18]. This work employs the normalized geometric ranking method as follows:
(3)pj=q′(1-q)rank-1,j=1,2,…,psRGA,
where pj = probability of selecting individual j, q = probability of choosing the best individual (here q=0.35), q′=q/(1-(1-q)psRGA), and rank = individual ranking based on fitness value, where 1 represents the best, rank=1,2,…,psRGA, and psRGA = population size of the RGA method.


#### 2.1.2. Crossover Operation

While exploring the solution space by creating new offspring, the crossover operation randomly chooses two parents from the crossover pool and then uses these two parents to create two new offspring. This operation is repeated until the psRGA/2 is satisfied. The whole arithmetic crossover is easily performed as follows:
(4)v1′=β×v1+(1-β)×v2,v2′=(1-β)×v1+β×v2,
where v1 and v2 = parents (decision variable vectors), v1′ and v2′ = offspring (decision variable vectors), and β = uniform random number in the interval [0,1.5].


#### 2.1.3. Mutation Operation

Mutation operation can improve the diversity of individuals (candidate solutions). Multi-non-uniform mutation is described as follows:(5)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gRGA)ifU1(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gRGA)ifU1(0,1)≥0.5,
where pert(gRGA)=[U2(0,1)(1-gRGA/gmax,RGA)]2, perturbed factor, U1(0,1) and U2(0,1) = uniform random variable in the interval [0,1], gmax,RGA = maximum generation of the RGA method, gRGA = current generation of the RGA method, xcurrent,n = current decision variable xn,   and  xtrial,n = trial decision variable (candidate solution) xn.


### 2.2. Particle Swarm Optimization

Kennedy and Eberhart [19] first presented a standard PSO algorithm, which is inspired by the social behavior of bird flocks or fish schools. Like GAs, a PSO method is a population-based algorithm. A population of candidate solutions is called a particle swarm. The particle velocities can be updated by (6) as follows:
(6)vj,n(gPSO+1)=vj,n(gPSO)+c1rj,1(gPSO)×[pj,nlb(gPSO)-xj,n(gPSO)]+c2rj,2(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]j=1,2,…,psPSO,n=1,2,…,N
where vj,n(gPSO+1) = particle velocity of decision variable xn of particle j at generation gPSO+1,  vj,n(gPSO) = particle velocity of decision variable xn of particle j at generation gPSO, c1 = cognitive parameter, c2 = social parameter, xj,n(gPSO) = particle position of decision variable xn of particle j at generation gPSO, rj,1(gPSO),rj,2(gPSO) = independent uniform random numbers in the interval [0,1] at generation gPSO,  pj,nlb(gPSO) = best local solution at generation gPSO,  pj,ngb(gPSO) = best global solution at generation gPSO, and psPSO = population size of the PSO algorithm.The particle positions can be obtained using (7) as follows:
(7)xj,n(gPSO+1)=xj,n(gPSO)+vj,n(gPSO+1),j=1,2,…,psPSO,n=1,2,…,N.Shi and Eberhart [20] introduced a modified PSO algorithm by incorporating an inertia weight (ωin) into (8) to control the exploration and exploitation capabilities of a PSO algorithm as follows:
(8)vj,n(gPSO+1)=ωinvj,n(gPSO)+c1rj,1(gPSO)×[pj,nlb(gPSO)-xj,n(gPSO)]+c2rj,2(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]j=1,2,…,psPSO,n=1,2,…,N.A constriction coefficient (χ) in (9) is used to balance the exploration and exploitation tradeoff [21–23] as follows:
(9)vj,n(gPSO+1)=χ{[pj,ngb(gPSO)-xj,n(gPSO)]vj,n(gPSO)+τ1(gPSO)×[pj,nlb(gPSO)-xj,n(gPSO)]+τ2(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]}j=1,2,…,psPSO,n=1,2,…,N,
where
(10)χ=2U3(0,1)|2-τ-τ(τ-4)|,U3(0,1) = uniform random variable in the interval [0,1], τ=τ1+τ2, τ1=c1rj,1,τ1=c2rj,2.This work considers parametersωin and χ to modify the particle velocities as follows:
(11)vj,n(gPSO+1)=χ{[pj,ngb(gPSO)-xj,n(gPSO)]ωinvj,n(gPSO)+c1rj,1(gPSO)×[pj,nlb(gPSO)-xj,n(gPSO)]+c2rj,2(gPSO)×[pj,ngb(gPSO)-xj,n(gPSO)]}j=1,2,…,psPSO,n=1,2,…,N,
where ωin=((gmax,PSO-gPSO)/gmax,PSO), increased gPSO value reduces the ωin, and  gmax,PSO = maximum generation of the PSO algorithm.According to (11), the optimal values of parameters c1, c2, and χ are difficult to obtain through trial and error. This work thus optimizes these parameter settings by using RGA and AIA approaches.


### 2.3. Artificial Immune Algorithm

Wu [24] presented an AIA approach based on clonal selection and immune network theories to solve constrained global optimization problems. The AIA method consists of selection, hypermutation, receptor editing, and bone marrow operations. The selection operation is performed to reproduce strong antibodies (Abs). Also, diverse Absare created using hypermutation, receptor editing, and bone marrow operations, as described in the following subsections.


#### 2.3.1.Ab and Ag Representation

In the human system, an antigen (Ag) has multiple epitopes (antigenic determinants), which can be recognized by many Abswith paratopes (recognizers), on its surface. In the AIA approach, an Ag represents known parameters of a solved problem. The Abs are the candidate solutions (i.e., decision variables xn, n=1,2,...,N) of the solved problem. The quality of a candidate solution is evaluated using an Ab-Ag affinity that is derived from the value of an objective function of the solved problem.


#### 2.3.2. Selection Operation

The selection operation controls the number of antigen-specificAbs. This operation is defined according to Ab-Ag and Ab-Ab recognition information as follows:
(12)pj,rec=1N∑n=1N1edj,n,dj,n=|xn*-xj,nxn*|,j=1,2,…,rsAIA,n=1,2,…,N,
where pj,rec = probability that Abj  recognizes Ab*  (the best solution), xn* = the best Ab* with the highest Ab-Ag affinity, xj,n = decision variables xn of Abj, and rsAIA = repertoire (population) size of the AIA approach.TheAb* is recognized by other Abj in a current Ab repertoire. Large pj,rec implies that Abj can effectively recognize Ab*. The Abj with pj,rec that is equivalent to or larger than the threshold degree of AIA approach prt,AIA is reproduced to generate an intermediate Ab repertoire.


#### 2.3.3. Hypermutation Operation

The somatic hypermutation operation can be expressed as follows:(13)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gAIA),ifU4(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gAIA),ifU4(0,1)≥0.5,
where pert(gAIA)={U5(0,1)(1-gAIA/gmax,AIA)}2 = perturbation factor, gAIA = current generation of the AIA method, gmax,AIA = maximum generation number of the AIA method, and U4(0,1) and U5(0,1) = uniform random number in the interval [0,1].This operation has two tasks, that is, a uniform search and local fine tuning.


#### 2.3.4. Receptor-Editing Operation

A receptor-editing operation is developed based on the standard Cauchy distributionC(0,1), in which the local parameter is zero and the scale parameter is one. Receptor editing is implemented using Cauchy random variables that are created from C(0,1), owing to their ability to provide a large jump in the Ab-Ag affinity landscape to increase the probability of escaping from the local Ab-Ag affinity landscape. Cauchy receptor editing can be defined by
(14)xtrial=xcurrent+U6(0,1)2×σ,
where σ=[σ1,σ2,…,σN]T, vector of Cauchy random variables, and U6(0,1) = uniform random number in the interval [0,1].This operation is used in local fine-tuning and large perturbation.


#### 2.3.5. Bone Marrow Operation

The paratope of anAb can be created by recombining gene segments VHDHJH and VLJL [25]. Therefore, based on this metaphor, diverse Abs are synthesized using a bone marrow operation. This operation randomly selects two Abs from the intermediate Ab repertoire and a recombination point from the gene segments of the paratope of the selected Abs. The selected gene segments (e.g., gene x1 of Ab1 and gene x1 of the Ab2) are reproduced to create a library of gene segments. The selected gene segments in the paratope are then deleted. The new Ab1 is formed by inserting the gene segment, which is gene x1 of the Ab2 in the library plus a random variable created from standard normal distribution N(0,1), at the recombination point. The details of the implementation of the bone marrow operation can be found [24].


## 2.1. Real-Coded Genetic Algorithm

GAs are based on the concepts of natural selection and use three genetic operations, that is, selection, crossover, and mutation, to explore and exploit the solution space. In solving continuous function optimization problems, RGA method outperforms binary-coded GA approach [17]. Therefore, this work describes operators of a RGA method [18].


### 2.1.1. Selection Operation

A selection operation picks up strong individuals from a current population based on their fitness function values and then reproduces these individuals into a crossover pool. Many selection operations developed include the roulette wheel, the ranking, and the tournament methods [17, 18]. This work employs the normalized geometric ranking method as follows:
(3)pj=q′(1-q)rank-1,j=1,2,…,psRGA,
where pj = probability of selecting individual j, q = probability of choosing the best individual (here q=0.35), q′=q/(1-(1-q)psRGA), and rank = individual ranking based on fitness value, where 1 represents the best, rank=1,2,…,psRGA, and psRGA = population size of the RGA method.


### 2.1.2. Crossover Operation

While exploring the solution space by creating new offspring, the crossover operation randomly chooses two parents from the crossover pool and then uses these two parents to create two new offspring. This operation is repeated until the psRGA/2 is satisfied. The whole arithmetic crossover is easily performed as follows:
(4)v1′=β×v1+(1-β)×v2,v2′=(1-β)×v1+β×v2,
where v1 and v2 = parents (decision variable vectors), v1′ and v2′ = offspring (decision variable vectors), and β = uniform random number in the interval [0,1.5].


### 2.1.3. Mutation Operation

Mutation operation can improve the diversity of individuals (candidate solutions). Multi-non-uniform mutation is described as follows:(5)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gRGA)ifU1(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gRGA)ifU1(0,1)≥0.5,
where pert(gRGA)=[U2(0,1)(1-gRGA/gmax,RGA)]2, perturbed factor, U1(0,1) and U2(0,1) = uniform random variable in the interval [0,1], gmax,RGA = maximum generation of the RGA method, gRGA = current generation of the RGA method, xcurrent,n = current decision variable xn,   and  xtrial,n = trial decision variable (candidate solution) xn.


## 2.1.1. Selection Operation

A selection operation picks up strong individuals from a current population based on their fitness function values and then reproduces these individuals into a crossover pool. Many selection operations developed include the roulette wheel, the ranking, and the tournament methods [17, 18]. This work employs the normalized geometric ranking method as follows:
(3)pj=q′(1-q)rank-1,j=1,2,…,psRGA,
where pj = probability of selecting individual j, q = probability of choosing the best individual (here q=0.35), q′=q/(1-(1-q)psRGA), and rank = individual ranking based on fitness value, where 1 represents the best, rank=1,2,…,psRGA, and psRGA = population size of the RGA method.


## 2.1.2. Crossover Operation

While exploring the solution space by creating new offspring, the crossover operation randomly chooses two parents from the crossover pool and then uses these two parents to create two new offspring. This operation is repeated until the psRGA/2 is satisfied. The whole arithmetic crossover is easily performed as follows:
(4)v1′=β×v1+(1-β)×v2,v2′=(1-β)×v1+β×v2,
where v1 and v2 = parents (decision variable vectors), v1′ and v2′ = offspring (decision variable vectors), and β = uniform random number in the interval [0,1.5].


## 2.1.3. Mutation Operation

Mutation operation can improve the diversity of individuals (candidate solutions). Multi-non-uniform mutation is described as follows:(5)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gRGA)ifU1(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gRGA)ifU1(0,1)≥0.5,
where pert(gRGA)=[U2(0,1)(1-gRGA/gmax,RGA)]2, perturbed factor, U1(0,1) and U2(0,1) = uniform random variable in the interval [0,1], gmax,RGA = maximum generation of the RGA method, gRGA = current generation of the RGA method, xcurrent,n = current decision variable xn,   and  xtrial,n = trial decision variable (candidate solution) xn.


## 2.2. Particle Swarm Optimization

Kennedy and Eberhart [19] first presented a standard PSO algorithm, which is inspired by the social behavior of bird flocks or fish schools. Like GAs, a PSO method is a population-based algorithm. A population of candidate solutions is called a particle swarm. The particle velocities can be updated by (6) as follows:
(6)vj,n(gPSO+1)=vj,n(gPSO)+c1rj,1(gPSO)×[pj,nlb(gPSO)-xj,n(gPSO)]+c2rj,2(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]j=1,2,…,psPSO,n=1,2,…,N
where vj,n(gPSO+1) = particle velocity of decision variable xn of particle j at generation gPSO+1,  vj,n(gPSO) = particle velocity of decision variable xn of particle j at generation gPSO, c1 = cognitive parameter, c2 = social parameter, xj,n(gPSO) = particle position of decision variable xn of particle j at generation gPSO, rj,1(gPSO),rj,2(gPSO) = independent uniform random numbers in the interval [0,1] at generation gPSO,  pj,nlb(gPSO) = best local solution at generation gPSO,  pj,ngb(gPSO) = best global solution at generation gPSO, and psPSO = population size of the PSO algorithm.The particle positions can be obtained using (7) as follows:
(7)xj,n(gPSO+1)=xj,n(gPSO)+vj,n(gPSO+1),j=1,2,…,psPSO,n=1,2,…,N.Shi and Eberhart [20] introduced a modified PSO algorithm by incorporating an inertia weight (ωin) into (8) to control the exploration and exploitation capabilities of a PSO algorithm as follows:
(8)vj,n(gPSO+1)=ωinvj,n(gPSO)+c1rj,1(gPSO)×[pj,nlb(gPSO)-xj,n(gPSO)]+c2rj,2(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]j=1,2,…,psPSO,n=1,2,…,N.A constriction coefficient (χ) in (9) is used to balance the exploration and exploitation tradeoff [21–23] as follows:
(9)vj,n(gPSO+1)=χ{[pj,ngb(gPSO)-xj,n(gPSO)]vj,n(gPSO)+τ1(gPSO)×[pj,nlb(gPSO)-xj,n(gPSO)]+τ2(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]}j=1,2,…,psPSO,n=1,2,…,N,
where
(10)χ=2U3(0,1)|2-τ-τ(τ-4)|,U3(0,1) = uniform random variable in the interval [0,1], τ=τ1+τ2, τ1=c1rj,1,τ1=c2rj,2.This work considers parametersωin and χ to modify the particle velocities as follows:
(11)vj,n(gPSO+1)=χ{[pj,ngb(gPSO)-xj,n(gPSO)]ωinvj,n(gPSO)+c1rj,1(gPSO)×[pj,nlb(gPSO)-xj,n(gPSO)]+c2rj,2(gPSO)×[pj,ngb(gPSO)-xj,n(gPSO)]}j=1,2,…,psPSO,n=1,2,…,N,
where ωin=((gmax,PSO-gPSO)/gmax,PSO), increased gPSO value reduces the ωin, and  gmax,PSO = maximum generation of the PSO algorithm.According to (11), the optimal values of parameters c1, c2, and χ are difficult to obtain through trial and error. This work thus optimizes these parameter settings by using RGA and AIA approaches.


## 2.3. Artificial Immune Algorithm

Wu [24] presented an AIA approach based on clonal selection and immune network theories to solve constrained global optimization problems. The AIA method consists of selection, hypermutation, receptor editing, and bone marrow operations. The selection operation is performed to reproduce strong antibodies (Abs). Also, diverse Absare created using hypermutation, receptor editing, and bone marrow operations, as described in the following subsections.


### 2.3.1.Ab and Ag Representation

In the human system, an antigen (Ag) has multiple epitopes (antigenic determinants), which can be recognized by many Abswith paratopes (recognizers), on its surface. In the AIA approach, an Ag represents known parameters of a solved problem. The Abs are the candidate solutions (i.e., decision variables xn, n=1,2,...,N) of the solved problem. The quality of a candidate solution is evaluated using an Ab-Ag affinity that is derived from the value of an objective function of the solved problem.


### 2.3.2. Selection Operation

The selection operation controls the number of antigen-specificAbs. This operation is defined according to Ab-Ag and Ab-Ab recognition information as follows:
(12)pj,rec=1N∑n=1N1edj,n,dj,n=|xn*-xj,nxn*|,j=1,2,…,rsAIA,n=1,2,…,N,
where pj,rec = probability that Abj  recognizes Ab*  (the best solution), xn* = the best Ab* with the highest Ab-Ag affinity, xj,n = decision variables xn of Abj, and rsAIA = repertoire (population) size of the AIA approach.TheAb* is recognized by other Abj in a current Ab repertoire. Large pj,rec implies that Abj can effectively recognize Ab*. The Abj with pj,rec that is equivalent to or larger than the threshold degree of AIA approach prt,AIA is reproduced to generate an intermediate Ab repertoire.


### 2.3.3. Hypermutation Operation

The somatic hypermutation operation can be expressed as follows:(13)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gAIA),ifU4(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gAIA),ifU4(0,1)≥0.5,
where pert(gAIA)={U5(0,1)(1-gAIA/gmax,AIA)}2 = perturbation factor, gAIA = current generation of the AIA method, gmax,AIA = maximum generation number of the AIA method, and U4(0,1) and U5(0,1) = uniform random number in the interval [0,1].This operation has two tasks, that is, a uniform search and local fine tuning.


### 2.3.4. Receptor-Editing Operation

A receptor-editing operation is developed based on the standard Cauchy distributionC(0,1), in which the local parameter is zero and the scale parameter is one. Receptor editing is implemented using Cauchy random variables that are created from C(0,1), owing to their ability to provide a large jump in the Ab-Ag affinity landscape to increase the probability of escaping from the local Ab-Ag affinity landscape. Cauchy receptor editing can be defined by
(14)xtrial=xcurrent+U6(0,1)2×σ,
where σ=[σ1,σ2,…,σN]T, vector of Cauchy random variables, and U6(0,1) = uniform random number in the interval [0,1].This operation is used in local fine-tuning and large perturbation.


### 2.3.5. Bone Marrow Operation

The paratope of anAb can be created by recombining gene segments VHDHJH and VLJL [25]. Therefore, based on this metaphor, diverse Abs are synthesized using a bone marrow operation. This operation randomly selects two Abs from the intermediate Ab repertoire and a recombination point from the gene segments of the paratope of the selected Abs. The selected gene segments (e.g., gene x1 of Ab1 and gene x1 of the Ab2) are reproduced to create a library of gene segments. The selected gene segments in the paratope are then deleted. The new Ab1 is formed by inserting the gene segment, which is gene x1 of the Ab2 in the library plus a random variable created from standard normal distribution N(0,1), at the recombination point. The details of the implementation of the bone marrow operation can be found [24].


## 2.3.1.Ab and Ag Representation

In the human system, an antigen (Ag) has multiple epitopes (antigenic determinants), which can be recognized by many Abswith paratopes (recognizers), on its surface. In the AIA approach, an Ag represents known parameters of a solved problem. The Abs are the candidate solutions (i.e., decision variables xn, n=1,2,...,N) of the solved problem. The quality of a candidate solution is evaluated using an Ab-Ag affinity that is derived from the value of an objective function of the solved problem.


## 2.3.2. Selection Operation

The selection operation controls the number of antigen-specificAbs. This operation is defined according to Ab-Ag and Ab-Ab recognition information as follows:
(12)pj,rec=1N∑n=1N1edj,n,dj,n=|xn*-xj,nxn*|,j=1,2,…,rsAIA,n=1,2,…,N,
where pj,rec = probability that Abj  recognizes Ab*  (the best solution), xn* = the best Ab* with the highest Ab-Ag affinity, xj,n = decision variables xn of Abj, and rsAIA = repertoire (population) size of the AIA approach.TheAb* is recognized by other Abj in a current Ab repertoire. Large pj,rec implies that Abj can effectively recognize Ab*. The Abj with pj,rec that is equivalent to or larger than the threshold degree of AIA approach prt,AIA is reproduced to generate an intermediate Ab repertoire.


## 2.3.3. Hypermutation Operation

The somatic hypermutation operation can be expressed as follows:(13)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gAIA),ifU4(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gAIA),ifU4(0,1)≥0.5,
where pert(gAIA)={U5(0,1)(1-gAIA/gmax,AIA)}2 = perturbation factor, gAIA = current generation of the AIA method, gmax,AIA = maximum generation number of the AIA method, and U4(0,1) and U5(0,1) = uniform random number in the interval [0,1].This operation has two tasks, that is, a uniform search and local fine tuning.


## 2.3.4. Receptor-Editing Operation

A receptor-editing operation is developed based on the standard Cauchy distributionC(0,1), in which the local parameter is zero and the scale parameter is one. Receptor editing is implemented using Cauchy random variables that are created from C(0,1), owing to their ability to provide a large jump in the Ab-Ag affinity landscape to increase the probability of escaping from the local Ab-Ag affinity landscape. Cauchy receptor editing can be defined by
(14)xtrial=xcurrent+U6(0,1)2×σ,
where σ=[σ1,σ2,…,σN]T, vector of Cauchy random variables, and U6(0,1) = uniform random number in the interval [0,1].This operation is used in local fine-tuning and large perturbation.


## 2.3.5. Bone Marrow Operation

The paratope of anAb can be created by recombining gene segments VHDHJH and VLJL [25]. Therefore, based on this metaphor, diverse Abs are synthesized using a bone marrow operation. This operation randomly selects two Abs from the intermediate Ab repertoire and a recombination point from the gene segments of the paratope of the selected Abs. The selected gene segments (e.g., gene x1 of Ab1 and gene x1 of the Ab2) are reproduced to create a library of gene segments. The selected gene segments in the paratope are then deleted. The new Ab1 is formed by inserting the gene segment, which is gene x1 of the Ab2 in the library plus a random variable created from standard normal distribution N(0,1), at the recombination point. The details of the implementation of the bone marrow operation can be found [24].


## 3. Methods

This work develops the RGA-PSO and AIA-PSO approaches for solving UGO problems. The implementation of the RGA-PSO and AIA-PSO methods is described as follows.


### 3.1. RGA-PSO Algorithm

Figure1 shows the pseudocode of the proposed RGA-PSO algorithm. The best parameter setting of the internal PSO algorithm is obtained by using the external RGA method. Benchmark UGO problems are solved by using the internal PSO algorithm.Figure 1
The pseudocode of the proposed RGA-PSO algorithm.External RGAStep1 (initialize the parameter settings). Parameter settings such as psRGA, crossover probability of the RGA method pc,RGA, mutation probability of the RGA approach pm,RGA, psPSO as well as lower and upper boundaries of the parameters (c1, c2, and χ) for a PSO algorithm are given. The candidate solution (individual) of a RGA method represents the optimized parameters of internal PSO algorithm. Figure 2 illustrates the candidate solution of the RGA method.Figure 2
Candidate solution of the RGA method.Step2 (calculate the fitness function value). The fitness function value fitnessj of the external RGA method is the best objective function value f(xPSO*) obtained from the best solution xPSO* of each internal PSO algorithm execution as follows:
(15)fitnessj=f(xPSO*),j=1,2,…,psRGA.The candidate solutionj of the external RGA method is incorporated into the internal PSO algorithm and, then, the internal PSO algorithm is used to solve an UGO problem. The internal PSO algorithm is executed as follows.Internal PSO Algorithm(1)
Generate an initial particle swarm. An initial particle swarm is created based on psPSO from [xnl,xnu] of an UGO problem. A particle represents a candidate solution of an UGO problem.(2)
Compute the objective function value. The objective function value of the internal PSO algorithm of particle jf(xj,PSO)j=1,2,…,psPSO is the objective function value of an UGO problem.(3)
Update the particle velocity and position. Equations (7) and (11) can be used to update the particle position and velocity.(4)
Perform an elitist strategy. A new particle swarm is generated from internal step (3). Notably, f(xj,PSO) of a candidate solution j (particle j) in the particle swarm is evaluated. This work makes a pairwise comparison between the f(xj,PSO) of candidate solutions in the new particle swarm and that in the current particle swarm. A situation in which the candidate solution j (j=1,2,…,psPSO) in the new particle swarm is better than candidate solution j in the current particle swarm implies that the strong candidate solution j in the new particle swarm replaces the candidate solution j in the current particle swarm. The elitist strategy guarantees that the best candidate solution is always preserved in the next generation. The current particle swarm is updated to the particle swarm of the next generation.Internal steps from(2) to (4) are repeated until the maximum generation number of the PSO method gmax,PSO of the internal PSO algorithm is satisfied.EndStep3 (perform a selection operation). Equation (3) is used to select the parents into a crossover pool.Step4 (implement a crossover operation). The crossover operation performs a global search. The candidate solutions are created by using (4).Step5 (conduct a mutation operation). The mutation operation implements a local search. A solution space is exploited using (5).Step6 (perform an elitist strategy). This work presents an elitist strategy to update the population. A situation in which the fittnessj of candidate solution j in the new population is larger than the fittnessj of candidate solution j in the current population suggests that a replacement of the weak candidate solution j takes place. Additionally, a situation in which the fittnessj of candidate solution j in the new population is equal to or worse than that in the current population implies that the candidate solution j in the current population survives. In addition to maintaining the strong candidate solutions, this strategy effectively eliminates weak candidate solutions.External steps 2 to 6 are repeated until thegmax,RGA of the external RGA method is met.


### 3.2. AIA-PSO Algorithm

Figure3 shows the pseudocode of the proposed AIA-PSO algorithm. The external AIA method is used to optimize the parameter settings of the internal PSO method, which is employed to solve benchmark UGO problems.Figure 3
The pseudocode of the proposed AIA-PSO algorithm.External AIAStep 1 (initialize the parameter settings). Several parameters must be predetermined. These include repertoire (population) size rsAIA and prt,AIA. An available Ab repertoire (population) is randomly generated using rsAIA from the lower and upper boundaries of parameters χ [χl,χu], c1 [c1l,c1u], and c2 [c2l,c2u]. Figure 4 shows the Ab and Ag representation.Figure 4
Ab and Ag representation.Step 2 (evaluate the Ab-Ag affinity).Internal PSO Algorithm. The external AIA approach offers parameter settings c1, c2, and χ for the internal PSO algorithm. Subsequently, internal steps (1)–(4) of the PSO algorithm are implemented. The internal PSO method returns the best fitness value of PSOf(xPSO*) to the external AIA method.(1)
Generate an initial particle swarm. An initial particle swarm is created based on psPSO from [xnl, xnu] of an UGO problem. A particle represents a candidate solution of an UGO problem.(2)
Compute the fitness value. The fitness value of the internal PSO algorithm f(xPSO,j)  j=1,2,…,psPSO is the objective function value of an UGO problem.(3)
Update the particle velocity and position. Equations (7) and (11) can be used to update the particle position and velocity.(4)
Perform an elitist strategy. A new particle swarm (population) is generated from internal step (3). Notably, f(xj,PSO) of a candidate solution j (particle j) in the particle swarm is evaluated. This work makes a pairwise comparison between the f(xj,PSO) of candidate solutions in the new particle swarm and that in the current particle swarm. The elitist strategy guarantees that the best candidate solution is always preserved in the next generation. The current particle swarm is updated to the particle swarm of the next generation.Internal steps from(2) to (4) are repeated until the gmax,PSO of the internal PSO algorithm is satisfied.EndConsistent with theAb-Ag affinity metaphor, an Ab-Ag affinity is determined using (16) as follows:
(16)max(affinityj)=-1×f(xPSO*)j=1,2,…,rsAIA.Following the evaluation of theAb-Ag affinities of Absin the current Ab repertoire, the Ab with the highest Ab-Ag affinity (Ab*) is chosen to undergo clonal selection in external Step 3.Step 3 (perform a clonal selection operation). To control the number of antigen-specific Abs, (12) is employed.Step 4 (implement an Ab-Ag affinity maturation operation). The intermediate Ab repertoire that is created in external Step 3 is divided into two subsets. These Abs undergo somatic hypermutation operation by using (13) when the random number is 0.5 or less. Notably, these Abs suffer receptor-editing operation using (14) when the random number exceeds 0.5.Step 5 (introduce diverse Abs). Based on the bone marrow operation, diverse Absare created to recruit the Abssuppressed in external Step 3.Step 6 (Update an Ab repertoire). A new Abrepertoire is generated from external Steps 3–5. The Ab-Ag affinities of the Abs in the generated Ab repertoire are evaluated. This work presents a strategy for updating the Ab repertoire. A situation in which the Ab-Ag affinity of Abj in the new Ab repertoire exceeds that in the current Ab repertoire implies that a strong Ab in the new Ab repertoire replaces the weak Ab in the current Ab repertoire. Additionally, a situation in which the Ab-Ag affinity of Abj in the new Ab repertoire equals to or is worse than that in the current Ab repertoire implies that the Abj in the current Ab repertoire survives. In addition to maintaining the strong Abs, this strategy eliminates nonfunctional AbsRepeat external Steps 2–6 until the termination criteriongmax,AIA is satisfied.


## 3.1. RGA-PSO Algorithm

Figure1 shows the pseudocode of the proposed RGA-PSO algorithm. The best parameter setting of the internal PSO algorithm is obtained by using the external RGA method. Benchmark UGO problems are solved by using the internal PSO algorithm.Figure 1
The pseudocode of the proposed RGA-PSO algorithm.External RGAStep1 (initialize the parameter settings). Parameter settings such as psRGA, crossover probability of the RGA method pc,RGA, mutation probability of the RGA approach pm,RGA, psPSO as well as lower and upper boundaries of the parameters (c1, c2, and χ) for a PSO algorithm are given. The candidate solution (individual) of a RGA method represents the optimized parameters of internal PSO algorithm. Figure 2 illustrates the candidate solution of the RGA method.Figure 2
Candidate solution of the RGA method.Step2 (calculate the fitness function value). The fitness function value fitnessj of the external RGA method is the best objective function value f(xPSO*) obtained from the best solution xPSO* of each internal PSO algorithm execution as follows:
(15)fitnessj=f(xPSO*),j=1,2,…,psRGA.The candidate solutionj of the external RGA method is incorporated into the internal PSO algorithm and, then, the internal PSO algorithm is used to solve an UGO problem. The internal PSO algorithm is executed as follows.Internal PSO Algorithm(1)
Generate an initial particle swarm. An initial particle swarm is created based on psPSO from [xnl,xnu] of an UGO problem. A particle represents a candidate solution of an UGO problem.(2)
Compute the objective function value. The objective function value of the internal PSO algorithm of particle jf(xj,PSO)j=1,2,…,psPSO is the objective function value of an UGO problem.(3)
Update the particle velocity and position. Equations (7) and (11) can be used to update the particle position and velocity.(4)
Perform an elitist strategy. A new particle swarm is generated from internal step (3). Notably, f(xj,PSO) of a candidate solution j (particle j) in the particle swarm is evaluated. This work makes a pairwise comparison between the f(xj,PSO) of candidate solutions in the new particle swarm and that in the current particle swarm. A situation in which the candidate solution j (j=1,2,…,psPSO) in the new particle swarm is better than candidate solution j in the current particle swarm implies that the strong candidate solution j in the new particle swarm replaces the candidate solution j in the current particle swarm. The elitist strategy guarantees that the best candidate solution is always preserved in the next generation. The current particle swarm is updated to the particle swarm of the next generation.Internal steps from(2) to (4) are repeated until the maximum generation number of the PSO method gmax,PSO of the internal PSO algorithm is satisfied.EndStep3 (perform a selection operation). Equation (3) is used to select the parents into a crossover pool.Step4 (implement a crossover operation). The crossover operation performs a global search. The candidate solutions are created by using (4).Step5 (conduct a mutation operation). The mutation operation implements a local search. A solution space is exploited using (5).Step6 (perform an elitist strategy). This work presents an elitist strategy to update the population. A situation in which the fittnessj of candidate solution j in the new population is larger than the fittnessj of candidate solution j in the current population suggests that a replacement of the weak candidate solution j takes place. Additionally, a situation in which the fittnessj of candidate solution j in the new population is equal to or worse than that in the current population implies that the candidate solution j in the current population survives. In addition to maintaining the strong candidate solutions, this strategy effectively eliminates weak candidate solutions.External steps 2 to 6 are repeated until thegmax,RGA of the external RGA method is met.


## 3.2. AIA-PSO Algorithm

Figure3 shows the pseudocode of the proposed AIA-PSO algorithm. The external AIA method is used to optimize the parameter settings of the internal PSO method, which is employed to solve benchmark UGO problems.Figure 3
The pseudocode of the proposed AIA-PSO algorithm.External AIAStep 1 (initialize the parameter settings). Several parameters must be predetermined. These include repertoire (population) size rsAIA and prt,AIA. An available Ab repertoire (population) is randomly generated using rsAIA from the lower and upper boundaries of parameters χ [χl,χu], c1 [c1l,c1u], and c2 [c2l,c2u]. Figure 4 shows the Ab and Ag representation.Figure 4
Ab and Ag representation.Step 2 (evaluate the Ab-Ag affinity).Internal PSO Algorithm. The external AIA approach offers parameter settings c1, c2, and χ for the internal PSO algorithm. Subsequently, internal steps (1)–(4) of the PSO algorithm are implemented. The internal PSO method returns the best fitness value of PSOf(xPSO*) to the external AIA method.(1)
Generate an initial particle swarm. An initial particle swarm is created based on psPSO from [xnl, xnu] of an UGO problem. A particle represents a candidate solution of an UGO problem.(2)
Compute the fitness value. The fitness value of the internal PSO algorithm f(xPSO,j)  j=1,2,…,psPSO is the objective function value of an UGO problem.(3)
Update the particle velocity and position. Equations (7) and (11) can be used to update the particle position and velocity.(4)
Perform an elitist strategy. A new particle swarm (population) is generated from internal step (3). Notably, f(xj,PSO) of a candidate solution j (particle j) in the particle swarm is evaluated. This work makes a pairwise comparison between the f(xj,PSO) of candidate solutions in the new particle swarm and that in the current particle swarm. The elitist strategy guarantees that the best candidate solution is always preserved in the next generation. The current particle swarm is updated to the particle swarm of the next generation.Internal steps from(2) to (4) are repeated until the gmax,PSO of the internal PSO algorithm is satisfied.EndConsistent with theAb-Ag affinity metaphor, an Ab-Ag affinity is determined using (16) as follows:
(16)max(affinityj)=-1×f(xPSO*)j=1,2,…,rsAIA.Following the evaluation of theAb-Ag affinities of Absin the current Ab repertoire, the Ab with the highest Ab-Ag affinity (Ab*) is chosen to undergo clonal selection in external Step 3.Step 3 (perform a clonal selection operation). To control the number of antigen-specific Abs, (12) is employed.Step 4 (implement an Ab-Ag affinity maturation operation). The intermediate Ab repertoire that is created in external Step 3 is divided into two subsets. These Abs undergo somatic hypermutation operation by using (13) when the random number is 0.5 or less. Notably, these Abs suffer receptor-editing operation using (14) when the random number exceeds 0.5.Step 5 (introduce diverse Abs). Based on the bone marrow operation, diverse Absare created to recruit the Abssuppressed in external Step 3.Step 6 (Update an Ab repertoire). A new Abrepertoire is generated from external Steps 3–5. The Ab-Ag affinities of the Abs in the generated Ab repertoire are evaluated. This work presents a strategy for updating the Ab repertoire. A situation in which the Ab-Ag affinity of Abj in the new Ab repertoire exceeds that in the current Ab repertoire implies that a strong Ab in the new Ab repertoire replaces the weak Ab in the current Ab repertoire. Additionally, a situation in which the Ab-Ag affinity of Abj in the new Ab repertoire equals to or is worse than that in the current Ab repertoire implies that the Abj in the current Ab repertoire survives. In addition to maintaining the strong Abs, this strategy eliminates nonfunctional AbsRepeat external Steps 2–6 until the termination criteriongmax,AIA is satisfied.


## 4. Results

The proposed RGA-PSO and AIA-PSO algorithms were applied to a set of benchmark UGO problems taken from other studies [9, 10, 17, 26], as detailed in the Appendix. The proposed RGA-PSO and AIA-PSO approaches were coded in MATLAB software and run on a Pentium D 3.0 (GHz) personal computer. One-hundred independent runs were conducted for each test problem (TP). To have comparable numerical results, the accuracy was chosen based on the numerical results reported in [9, 10, 17, 26]. Numerical results were summarized, including rate of successful minimizations (success rate %), best mean worst, mean computational CPU time (MCCT), and mean error ME (average value of the gap between the objective function values calculated using the AIA-PSO and RGA-PSO solutions and the known global minimum value). Table 1 lists the parameter settings for the proposed RGA-PSO and AIA-PSO approaches. The table shows 20,000 (20×1000) objective function evaluations of the internal PSO approach for an UGO problem with N≤3 decision variables and 60,000 (20×3000) objective function evaluations of the internal PSO for an UGO problem with 5≤N≤30 decision variables. Moreover, the external AIA and RGA methods stop when gmax,RGA= 20 and gmax,AIA= 20 are met or the best fitness value of the RGA approach (or the best Ab-Ag affinity of the AIA method) does not significantly change for the past five generations.Table 1
The parameter settings for the proposed RGA-PSO and AIA-PSO approaches.
Methods
Parameter settings
Search space
External RGA
p
c
,
RGA
= 1pm,RGA= 0.15psRGA= 20gmax,RGA= 20
[
χ
l
,
χ
u
] = [0.1,1] [c1l,c1u] = [0.1,5] [c2l,c2u] = [0.1,5]
External AIA
p
rt,AIA = 0.9 rsAIA = 20gmax,AIA = 20
Internal PSO
psPSO = 20 gmax,PSO = 1000 for N≤3 gmax,PSO = 3000 for 5≤N≤30
[
x
n
l, xnu] for a UGO problem


### 4.1. Numerical Results Obtained Using the RGA-PSO and AIA-PSO Algorithms for Low-Dimensional UGO Problems (2≤N≤10)

Table2 lists the numerical results obtained using the proposed RGA-PSO algorithm. The numerical results indicate that the RGA-PSO algorithm can obtain the global minimum for each test UGO problem since these MEs equal or closely approximate “0,” and the RGA-PSO algorithm has an acceptable MCCT for each TP. Table 3 lists the optimal parameter settings obtained using the proposed RGA-PSO algorithm to solve 14 UGO problems.Table 2
Numerical results obtained from the proposed RGA-PSO algorithm for solving 14 UGO problems.
TP number
Function name
Global minimum
Required accuracy
Success rate (%)
f
(
x
RGA-PSO
*
)
f
(
x
RGA-PSO
mean
)
f
(
x
RGA-PSO
worst
)
ME
MCCT (sec.)
1
SHCB
−1.0316
1
E
-
4
100
−1.0316
−1.0316
−1.0316
0.00000000
40.56
2
GP
3
1
E
-
3
100
3
3
3
0.00000000
38.29
3
ES
−1
1
E
-
3
100
−1
−1
−1
0.00000000
41.29
4
B2
0
1
E
-
2
100
0.00000000
0.00000000
0.00000000
0.00000000
40.26
5
DJ
0
1
E
-
4
100
0.00000000
5.533
E
-
226
5.533
E
-
224
5.533
E
-
226
40.50
6
Booth
0
1
E
-
4
100
0.00000000
0.00000000
0.00000000
0.00000000
37.57
7
RC
5/4π
1
E
-
3
100
0.39788735
0.39788735
0.39788735
0.00000000
40.97
8
RA
−2
1
E
-
4
100
−2
−2
−2
0.00000000
39.31
9
RS2
0
1
E
-
3
100
0.00000000
8.709
E
-
28
8.686
E
-
26
8.709
E
-
28
37.47
10
RS5
0
1
E
-
3
100
9.587
E
-
12
9.452
E
-
06
9.058
E
-
05
9.452
E
-
06
306.15
11
SH
−186.7309
1
E
-
2
100
−186.7309
−186.7309
−186.7309
0.00000000
54.11
12
ZA2
0
1
E
-
4
100
0.00000000
0.00000000
0.00000000
0.00000000
40.74
13
ZA5
0
1
E
-
4
100
0.00000000
2.660
E
-
117
2.660
E
-
115
2.660
E
-
117
157.09
14
ZA10
0
1
E
-
3
100
2.453
E
-
120
4.375
E
-
26
2.933
E
-
24
4.375
E
-
26
194.73
f
(
x
RGA-PSO
*
): the objective value obtained using the best RGA-PSO solution.
f
(
x
RGA-PSO
mean
): mean objective value obtained using the RGA-PSO solutions.
f
(
x
RGA-PSO
worst
): the worst objective value obtained using the worst RGA-PSO solution.Table 3
The optimal parameter settings obtained using the proposed RGA-PSO algorithm for solving 14 UGO problems.
TP number
Function name
χ
c
1
c
2
1
SHCB
0.36497798
1.79972890
2.36717639
2
GP
0.86138264
3.74964714
0.10000000
3
ES
0.36038653
0.42043804
4.78367612
4
B2
0.75486133
0.17875492
0.95144705
5
DJ
0.39357143
3.17898242
4.72512283
6
Booth
0.56703400
4.84740898
2.58109482
7
RC
0.44153938
2.71748076
3.81076900
8
RA
0.93533829
0.76530619
3.53402840
9
RS2
0.87003776
1.28577955
2.77423057
10
RS5
0.62553666
0.10000000
5.00000000
11
SH
0.45903159
1.07452631
4.15038556
12
ZA2
0.75044069
0.50883133
1.93860581
13
ZA5
0.52013863
1.05471011
4.61197390
14
ZA10
0.56126516
0.39456605
4.95735921Table4 lists the numerical results obtained using the proposed AIA-PSO algorithm. Numerical results indicate that the AIA-PSO algorithm can obtain the global minimum for each test UGO problem since these MEs equal or closely approximate “0,” and that the AIA-PSO algorithm has an acceptable MCCT for each TP. Table 5 lists the optimal parameter settings obtained using the proposed AIA-PSO algorithm for solving 14 UGO problems.Table 4
Numerical results obtained from the proposed AIA-PSO algorithm for solving 14 UGO problems.
TP number
Function name
Global minimum
Required accuracy
Success rate (%)
f
(
x
AIA-PSO
*
)
f
(
x
AIA-PSO
mean
)
f
(
x
AIA-PSO
worst
)
ME
MCCT (sec)
1
SHCB
−1.0316
1
E
-
4
100
−1.0316
−1.0316
−1.0316
0.00000000
39.69
2
GP
3
1
E
-
3
100
3
3
3
0.00000000
37.80
3
ES
−1
1
E
-
3
100
−1
−1
−1
0.00000000
42.08
4
B2
0
1
E
-
2
100
0.00000000
0.00000000
0.00000000
0.00000000
40.43
5
DJ
0
1
E
-
4
100
0.00000000
2.544
E
-
215
2.544
E
-
213
2.544
E
-
215
39.43
6
Booth
0
1
E
-
4
100
0.00000000
0.00000000
0.00000000
0.00000000
37.48
7
RC
5/4π
1
E
-
3
100
0.39788735
0.39788735
0.39788735
0.00000000
40.53
8
RA
−2
1
E
-
4
100
−2
−2
−2
0.00000000
39.17
9
RS2
0
1
E
-
3
100
0.00000000
2.341
E
-
27
2.129
E
-
25
2.341
E
-
27
37.49
10
RS5
0
1
E
-
3
100
4.342
E
-
10
7.291
E
-
06
9.313
E
-
05
7.291
E
-
06
304.23
11
SH
−186.7309
1
E
-
2
100
−186.7309
−186.7309
−186.7309
0.00000000
54.70
12
ZA2
0
1
E
-
4
100
0.00000000
0.00000000
0.00000000
0.00000000
40.31
13
ZA5
0
1
E
-
4
100
0.00000000
7.2591
E
-
184
7.259
E
-
182
7.259
E
-
184
144.38
14
ZA10
0
1
E
-
3
100
1.472
E
-
120
3.287
E
-
42
2.856
E
-
40
3.287
E
-
42
192.06
f
(
x
AIA-PSO
*
): the objective value obtained using the best AIA-PSO solution.
f
(
x
AIA-PSO
mean
): mean objective value obtained using the AIA-PSO solutions.
f
(
x
AIA-PSO
worst
): the worst objective value obtained using the worst AIA-PSO solution.Table 5
The optimal parameter settings obtained using the proposed AIA-PSO algorithm for solving 14 UGO problems.
TP number
Function name
χ
c
1
c
2
1
SHCB
0.59403239
1.75960560
4.12826475
2
GP
0.29584147
2.62468918
4.41787729
3
ES
0.39500323
0.84589484
2.64979988
4
B2
0.64806710
1.78727677
1.83995777
5
DJ
0.45630341
1.11829588
4.10382418
6
Booth
0.60889449
2.53158038
1.32827344
7
RC
1.00000000
1.48772085
0.43921335
8
RA
0.38157302
4.93248942
4.08732827
9
RS2
0.39872504
4.65992304
4.84228028
10
RS5
0.55519288
0.10000000
4.52483714
11
SH
0.54293231
2.10545153
4.52889414
12
ZA2
0.44530811
1.05474921
3.20092987
13
ZA5
0.51683760
1.17129205
4.94965004
14
ZA10
0.53688659
0.53752883
4.99881022


### 4.2. Numerical Results Obtained Using the RGA-PSO and AIA-PSO Algorithms for a Standard-Dimensional UGO Problem (N=30)

To investigate the effectiveness of the RGA-PSO and AIA-PSO methods for solving a standard-dimensional UGO problem, the Zakharov problem with 30 decision variables (ZA30), as described in the Appendix, has been solved using the RGA-PSO and AIA-PSO approaches. Fifty independent runs were performed to solve the UGO problem. To increase the diversity of candidate solutions for use in the external RGA method, the parameter pm,RGA was set from 0.15 to “1.” Table 6 lists the numerical results obtained using the RGA-PSO and AIA-PSO approaches. This table indicates that the two approaches converge to the global optimum value, since the MEs closely approximate “0,” and the MCCT of the RGA-PSO method is larger than that of the AIA-PSO method. Moreover, the success rates of the proposed RGA-PSO and AIA-PSO approaches are 100%. The Wilcoxon test is performed to the difference of median values of the MEs obtained using the RGA-PSO and AIA-PSO methods. The P value of the Wilcoxon test is 0.028, which is smaller than the significance level of 0.05, indicating that the performance of the RGA-PSO method is statistically different to that of the AIA-PSO method. Table 7 summarizes the optimal parameter settings obtained using the proposed RGA-PSO and AIA-PSO algorithms for the UGO problem ZA30.Table 6
Numerical results obtained from the proposed RGA-PSO and AIA-PSO algorithms for solving a standard-dimensional UGO problem.
Functionname
Global  minimum
Required  accuracy
Methods
Success rate (%)
f
(
x
RGA-PSO
*
)
f
(
x
RGA-PSO
mean
)
f
(
x
RGA-PSO
worst
)
ME
MCCT (sec)
ZA30
0
1
E
-
3
RGA-PSO
100
4.186
E
-
13
4.537
E
-
06
5.331
E
-
05
4.537
E
-
06
1753.86
AIA-PSO
100
6.520
E
-
12
8.099
E
-
06
9.175
E
-
05
8.099
E
-
06
1453.98Table 7
The optimal parameter settings obtained using the proposed RGA-PSO and AIA-PSO algorithms for solving a standard-dimensional UGO problem.
Function name
Methods
χ
c
1
c
2
ZA30
RGA-PSO
0.58721675
0.56183025
4.91401312
AIA-PSO
0.61082112
0.38556559
5.00000000The UGO problem ZA50 was solved using the RGA-PSO and AIA-PSO methods. The RGA-PSO and AIA-PSO methods fail to solve the UGO problem, since the diversity of the particle swarm in the internal PSO method cannot be maintained. Hence, future work will focus on improving the diversity of the particle swarm by applying mutation operations.


### 4.3. Comparison

Table8 lists the results of the Wilcoxon test for the MEs obtained using the proposed RGA-PSO and AIA-PSO methods for 14 UGO problems. In this table, the “**” represents the P value of Wilcoxon test which cannot be obtained, since the MEs obtained using the RGA-PSO and AIA-PSO methods for a TP are identical. Moreover, the median values of the MEs obtained using the RGA-PSO and AIA-PSO methods for TP 10 are not statistically different, since their P value is larger than the significance level of 0.05. Overall, the performances obtained using the RGA-PSO and AIA-PSO methods are statistically identical.Table 8
Results of Wilcoxon test for the MEs obtained using the proposed RGA-PSO and AIA-PSO methods for 14 UGO problems.
TP number
Function name
RGA-PSO versus AIA-PSO
P value
1
SHCB
**
2
GP
**
3
ES
**
4
B2
**
5
DJ
**
6
Booth
**
7
RC
**
8
RA
**
9
RS2
**
10
RS5
0.833
11
SH
**
12
ZA2
**
13
ZA5
**
14
ZA10
**Table9 compares the numerical results obtained using the RGA-PSO and AIA-PSO methods with those obtained using the hybrid algorithms for 11 TPs. Specifically, the table lists the numerical results obtained using the Nelder-Mead simplex search and PSO (NM-PSO) and GA-PSO taken from [9], those obtained using particle swarm ant colony optimization (PSACO), continuous hybrid algorithm (CHA) and continuous tabu simplex search (CTSS) taken from [10], and those obtained using DE-PSO, AMPSO1, and AMPSO2 taken from [12]. Table 9 indicates that the RGA-PSO and AIA-PSO methods yield superior accuracy of MEs obtained using the NM-PSO, GA-PSO, DE-PSO, AM-PSO1, AM-PSO2, CHA and CTSS approaches for TPs 2, 3, 4, 5, 7, 9, 11, and 12 and that RGA-PSO and AIA-PSO approaches yield superior accuracy of MEs obtained using the PSACO method for TPs 4, 5, 7, 9, 10, 11, 12, 13 and 14. Table 10 compares the percentage success rates of the proposed RGA-PSO and AIA-PSO approaches and those of the hybrid algorithms for 11 TPs, indicating that all algorithms except for the CHA and CTSS methods achieved identical performance (100% success rate) for all TPs.Table 9
Comparison of the results of the proposed RGA-PSO and AIA-PSO approaches and those of the hybrid algorithms for 11 TPs.
TP number
Functionname
M
Es
NM-PSO [9]
PSACO [10]
GA-PSO [9]
DE-PSO [12]
AM-PSO1 [12]
AM-PSO2 [12]
CHA [10]
CTSS [10]
AIA-PSO(this work)
RGA-PSO(this work)
2
GP
0.00003
0.00000000
0.00012
3.67
E
-
5
3.81
E
-
5
9.89
E
-
6
0.0010
0.001
0.00000000
0.00000000
3
ES
0.00004
0.00000000
0.00003
1.67
E
-
5
4.10
E
-
5
3.71
E
-
5
0.0010
0.005
0.00000000
0.00000000
4
B2
0.00003
5.5511
E
-
17
0.00001
1.11
E
-
5
1.17
E
-
6
2.64
E
-
5
0.0000002
0.000005
0.00000000
0.00000000
5
DJ
0.00003
7.6900
E
-
29
0.00004
1.67
E
-
5
4.23
E
-
5
3.34
E
-
5
0.0002
0.0002
2.544
E
-
215
5.5331
E
-
226
7
RC
0.00003
2.6185
E
-
13
0.00009
2.29
E
-
6
1.52
E
-
5
1.49
E
-
5
0.0001
0.005
0.00000000
0.00000000
9
RS2
0.00003
1.7152
E
-
10
0.00064
2.46
E
-
5
2.16
E
-
5
3.33
E
-
5
0.0040
0.0040
2.341
E
-
27
8.709
E
-
28
10
RS5
0.0056
1.8538
E
-
04
0.00013
7.74
E
-
5
9.73
E
-
5
9.93
E
-
5
—
—
7.291
E
-
06
9.452
E
-
06
11
SH
0.00002
1.09239
E
-
09
0.00007
3.03
E
-
6
1.72
E
-
5
2.54
E
-
7
0.0050
0.001
0.00000000
0.00000000
12
ZA2
0.00003
5.7061
E
-
27
0.00005
1.12
E
-
5
2.54
E
-
5
2.61
Z
E
-
6
3
E
-
6
3
E
-
7
0.00000000
0.00000000
13
ZA5
0.00026
3.6352
E
-
17
0.00000
3.49
E
-
5
3.44
E
-
5
5.07
E
-
5
—
—
7.259
E
-
184
2.660
E
-
117
14
ZA10
—
2
E
-
08
0.00000
6.91
E
-
5
7.41
E
-
5
9.11
E
-
5
1
E
-
6
—
3.287
E
-
42
4.3756
E
-
26
(“—” denotes unavailable information).Table 10
Comparison of the success rate % of the proposed RGA-PSO and AIA-PSO approaches and those of the hybrid algorithms for 11 TPs.
TP number
Function name
Success rate %
NM-PSO [9]
PSACO [10]
GA-PSO [9]
DE-PSO [12]
AM-PSO1 [12]
AM-PSO2 [12]
CHA [10]
CTSS [10]
AIA-PSO(this work)
RGA-PSO(this work)
2
GP
100
100
100
100
100
100
100
100
100
100
3
ES
100
100
100
100
100
100
100
100
100
100
4
B2
100
100
100
100
100
100
100
100
100
100
5
DJ
100
100
100
100
100
100
100
100
100
100
7
RC
100
100
100
100
100
100
100
100
100
100
9
RS2
100
100
100
100
100
100
100
100
100
100
10
RS5
100
100
100
100
100
100
—
—
100
100
11
SH
100
100
100
100
100
100
100
100
100
100
12
ZA2
100
100
100
100
100
100
100
100
100
100
13
ZA5
100
100
100
100
100
100
—
—
100
100
14
ZA10
100
100
100
100
100
100
100
—
100
100
(“—” denotes unavailable information).


### 4.4. Summary of Results

The proposed RGA-PSO and AIA-PSO algorithms have the following benefits.(1)
Parameter manipulation of the internal PSO algorithm is based on the solved UGO problems. Owing to their ability to efficiently solve UGO problems, the external RGA and AIA approaches are substituted for trial and error to manipulate the parameters (χ, c1, and c2).(2)
Besides obtaining the optimum parameter settings of the internal PSO algorithm, the RGA-PSO and AIA-PSO algorithms can yield a global minimum for an UGO problem.(3)
Beside, outperforming some published hybrid SGO methods, the proposed RGA-PSO and AIA-PSO approaches reduce the parametrization for the internal PSO algorithm, despite being more complex than individual SGO approaches.The proposed RGA-PSO and AIA-PSO algorithms are limited in that they cannot solve high-dimensional UGO problems (such asN=50). Future work will focus on increasing the diversity of the particle swarm of the internal PSO method by applying mutation to solve high-dimensional UGO problems.


## 4.1. Numerical Results Obtained Using the RGA-PSO and AIA-PSO Algorithms for Low-Dimensional UGO Problems (2≤N≤10)

Table2 lists the numerical results obtained using the proposed RGA-PSO algorithm. The numerical results indicate that the RGA-PSO algorithm can obtain the global minimum for each test UGO problem since these MEs equal or closely approximate “0,” and the RGA-PSO algorithm has an acceptable MCCT for each TP. Table 3 lists the optimal parameter settings obtained using the proposed RGA-PSO algorithm to solve 14 UGO problems.Table 2
Numerical results obtained from the proposed RGA-PSO algorithm for solving 14 UGO problems.
TP number
Function name
Global minimum
Required accuracy
Success rate (%)
f
(
x
RGA-PSO
*
)
f
(
x
RGA-PSO
mean
)
f
(
x
RGA-PSO
worst
)
ME
MCCT (sec.)
1
SHCB
−1.0316
1
E
-
4
100
−1.0316
−1.0316
−1.0316
0.00000000
40.56
2
GP
3
1
E
-
3
100
3
3
3
0.00000000
38.29
3
ES
−1
1
E
-
3
100
−1
−1
−1
0.00000000
41.29
4
B2
0
1
E
-
2
100
0.00000000
0.00000000
0.00000000
0.00000000
40.26
5
DJ
0
1
E
-
4
100
0.00000000
5.533
E
-
226
5.533
E
-
224
5.533
E
-
226
40.50
6
Booth
0
1
E
-
4
100
0.00000000
0.00000000
0.00000000
0.00000000
37.57
7
RC
5/4π
1
E
-
3
100
0.39788735
0.39788735
0.39788735
0.00000000
40.97
8
RA
−2
1
E
-
4
100
−2
−2
−2
0.00000000
39.31
9
RS2
0
1
E
-
3
100
0.00000000
8.709
E
-
28
8.686
E
-
26
8.709
E
-
28
37.47
10
RS5
0
1
E
-
3
100
9.587
E
-
12
9.452
E
-
06
9.058
E
-
05
9.452
E
-
06
306.15
11
SH
−186.7309
1
E
-
2
100
−186.7309
−186.7309
−186.7309
0.00000000
54.11
12
ZA2
0
1
E
-
4
100
0.00000000
0.00000000
0.00000000
0.00000000
40.74
13
ZA5
0
1
E
-
4
100
0.00000000
2.660
E
-
117
2.660
E
-
115
2.660
E
-
117
157.09
14
ZA10
0
1
E
-
3
100
2.453
E
-
120
4.375
E
-
26
2.933
E
-
24
4.375
E
-
26
194.73
f
(
x
RGA-PSO
*
): the objective value obtained using the best RGA-PSO solution.
f
(
x
RGA-PSO
mean
): mean objective value obtained using the RGA-PSO solutions.
f
(
x
RGA-PSO
worst
): the worst objective value obtained using the worst RGA-PSO solution.Table 3
The optimal parameter settings obtained using the proposed RGA-PSO algorithm for solving 14 UGO problems.
TP number
Function name
χ
c
1
c
2
1
SHCB
0.36497798
1.79972890
2.36717639
2
GP
0.86138264
3.74964714
0.10000000
3
ES
0.36038653
0.42043804
4.78367612
4
B2
0.75486133
0.17875492
0.95144705
5
DJ
0.39357143
3.17898242
4.72512283
6
Booth
0.56703400
4.84740898
2.58109482
7
RC
0.44153938
2.71748076
3.81076900
8
RA
0.93533829
0.76530619
3.53402840
9
RS2
0.87003776
1.28577955
2.77423057
10
RS5
0.62553666
0.10000000
5.00000000
11
SH
0.45903159
1.07452631
4.15038556
12
ZA2
0.75044069
0.50883133
1.93860581
13
ZA5
0.52013863
1.05471011
4.61197390
14
ZA10
0.56126516
0.39456605
4.95735921Table4 lists the numerical results obtained using the proposed AIA-PSO algorithm. Numerical results indicate that the AIA-PSO algorithm can obtain the global minimum for each test UGO problem since these MEs equal or closely approximate “0,” and that the AIA-PSO algorithm has an acceptable MCCT for each TP. Table 5 lists the optimal parameter settings obtained using the proposed AIA-PSO algorithm for solving 14 UGO problems.Table 4
Numerical results obtained from the proposed AIA-PSO algorithm for solving 14 UGO problems.
TP number
Function name
Global minimum
Required accuracy
Success rate (%)
f
(
x
AIA-PSO
*
)
f
(
x
AIA-PSO
mean
)
f
(
x
AIA-PSO
worst
)
ME
MCCT (sec)
1
SHCB
−1.0316
1
E
-
4
100
−1.0316
−1.0316
−1.0316
0.00000000
39.69
2
GP
3
1
E
-
3
100
3
3
3
0.00000000
37.80
3
ES
−1
1
E
-
3
100
−1
−1
−1
0.00000000
42.08
4
B2
0
1
E
-
2
100
0.00000000
0.00000000
0.00000000
0.00000000
40.43
5
DJ
0
1
E
-
4
100
0.00000000
2.544
E
-
215
2.544
E
-
213
2.544
E
-
215
39.43
6
Booth
0
1
E
-
4
100
0.00000000
0.00000000
0.00000000
0.00000000
37.48
7
RC
5/4π
1
E
-
3
100
0.39788735
0.39788735
0.39788735
0.00000000
40.53
8
RA
−2
1
E
-
4
100
−2
−2
−2
0.00000000
39.17
9
RS2
0
1
E
-
3
100
0.00000000
2.341
E
-
27
2.129
E
-
25
2.341
E
-
27
37.49
10
RS5
0
1
E
-
3
100
4.342
E
-
10
7.291
E
-
06
9.313
E
-
05
7.291
E
-
06
304.23
11
SH
−186.7309
1
E
-
2
100
−186.7309
−186.7309
−186.7309
0.00000000
54.70
12
ZA2
0
1
E
-
4
100
0.00000000
0.00000000
0.00000000
0.00000000
40.31
13
ZA5
0
1
E
-
4
100
0.00000000
7.2591
E
-
184
7.259
E
-
182
7.259
E
-
184
144.38
14
ZA10
0
1
E
-
3
100
1.472
E
-
120
3.287
E
-
42
2.856
E
-
40
3.287
E
-
42
192.06
f
(
x
AIA-PSO
*
): the objective value obtained using the best AIA-PSO solution.
f
(
x
AIA-PSO
mean
): mean objective value obtained using the AIA-PSO solutions.
f
(
x
AIA-PSO
worst
): the worst objective value obtained using the worst AIA-PSO solution.Table 5
The optimal parameter settings obtained using the proposed AIA-PSO algorithm for solving 14 UGO problems.
TP number
Function name
χ
c
1
c
2
1
SHCB
0.59403239
1.75960560
4.12826475
2
GP
0.29584147
2.62468918
4.41787729
3
ES
0.39500323
0.84589484
2.64979988
4
B2
0.64806710
1.78727677
1.83995777
5
DJ
0.45630341
1.11829588
4.10382418
6
Booth
0.60889449
2.53158038
1.32827344
7
RC
1.00000000
1.48772085
0.43921335
8
RA
0.38157302
4.93248942
4.08732827
9
RS2
0.39872504
4.65992304
4.84228028
10
RS5
0.55519288
0.10000000
4.52483714
11
SH
0.54293231
2.10545153
4.52889414
12
ZA2
0.44530811
1.05474921
3.20092987
13
ZA5
0.51683760
1.17129205
4.94965004
14
ZA10
0.53688659
0.53752883
4.99881022


## 4.2. Numerical Results Obtained Using the RGA-PSO and AIA-PSO Algorithms for a Standard-Dimensional UGO Problem (N=30)

To investigate the effectiveness of the RGA-PSO and AIA-PSO methods for solving a standard-dimensional UGO problem, the Zakharov problem with 30 decision variables (ZA30), as described in the Appendix, has been solved using the RGA-PSO and AIA-PSO approaches. Fifty independent runs were performed to solve the UGO problem. To increase the diversity of candidate solutions for use in the external RGA method, the parameter pm,RGA was set from 0.15 to “1.” Table 6 lists the numerical results obtained using the RGA-PSO and AIA-PSO approaches. This table indicates that the two approaches converge to the global optimum value, since the MEs closely approximate “0,” and the MCCT of the RGA-PSO method is larger than that of the AIA-PSO method. Moreover, the success rates of the proposed RGA-PSO and AIA-PSO approaches are 100%. The Wilcoxon test is performed to the difference of median values of the MEs obtained using the RGA-PSO and AIA-PSO methods. The P value of the Wilcoxon test is 0.028, which is smaller than the significance level of 0.05, indicating that the performance of the RGA-PSO method is statistically different to that of the AIA-PSO method. Table 7 summarizes the optimal parameter settings obtained using the proposed RGA-PSO and AIA-PSO algorithms for the UGO problem ZA30.Table 6
Numerical results obtained from the proposed RGA-PSO and AIA-PSO algorithms for solving a standard-dimensional UGO problem.
Functionname
Global  minimum
Required  accuracy
Methods
Success rate (%)
f
(
x
RGA-PSO
*
)
f
(
x
RGA-PSO
mean
)
f
(
x
RGA-PSO
worst
)
ME
MCCT (sec)
ZA30
0
1
E
-
3
RGA-PSO
100
4.186
E
-
13
4.537
E
-
06
5.331
E
-
05
4.537
E
-
06
1753.86
AIA-PSO
100
6.520
E
-
12
8.099
E
-
06
9.175
E
-
05
8.099
E
-
06
1453.98Table 7
The optimal parameter settings obtained using the proposed RGA-PSO and AIA-PSO algorithms for solving a standard-dimensional UGO problem.
Function name
Methods
χ
c
1
c
2
ZA30
RGA-PSO
0.58721675
0.56183025
4.91401312
AIA-PSO
0.61082112
0.38556559
5.00000000The UGO problem ZA50 was solved using the RGA-PSO and AIA-PSO methods. The RGA-PSO and AIA-PSO methods fail to solve the UGO problem, since the diversity of the particle swarm in the internal PSO method cannot be maintained. Hence, future work will focus on improving the diversity of the particle swarm by applying mutation operations.


## 4.3. Comparison

Table8 lists the results of the Wilcoxon test for the MEs obtained using the proposed RGA-PSO and AIA-PSO methods for 14 UGO problems. In this table, the “**” represents the P value of Wilcoxon test which cannot be obtained, since the MEs obtained using the RGA-PSO and AIA-PSO methods for a TP are identical. Moreover, the median values of the MEs obtained using the RGA-PSO and AIA-PSO methods for TP 10 are not statistically different, since their P value is larger than the significance level of 0.05. Overall, the performances obtained using the RGA-PSO and AIA-PSO methods are statistically identical.Table 8
Results of Wilcoxon test for the MEs obtained using the proposed RGA-PSO and AIA-PSO methods for 14 UGO problems.
TP number
Function name
RGA-PSO versus AIA-PSO
P value
1
SHCB
**
2
GP
**
3
ES
**
4
B2
**
5
DJ
**
6
Booth
**
7
RC
**
8
RA
**
9
RS2
**
10
RS5
0.833
11
SH
**
12
ZA2
**
13
ZA5
**
14
ZA10
**Table9 compares the numerical results obtained using the RGA-PSO and AIA-PSO methods with those obtained using the hybrid algorithms for 11 TPs. Specifically, the table lists the numerical results obtained using the Nelder-Mead simplex search and PSO (NM-PSO) and GA-PSO taken from [9], those obtained using particle swarm ant colony optimization (PSACO), continuous hybrid algorithm (CHA) and continuous tabu simplex search (CTSS) taken from [10], and those obtained using DE-PSO, AMPSO1, and AMPSO2 taken from [12]. Table 9 indicates that the RGA-PSO and AIA-PSO methods yield superior accuracy of MEs obtained using the NM-PSO, GA-PSO, DE-PSO, AM-PSO1, AM-PSO2, CHA and CTSS approaches for TPs 2, 3, 4, 5, 7, 9, 11, and 12 and that RGA-PSO and AIA-PSO approaches yield superior accuracy of MEs obtained using the PSACO method for TPs 4, 5, 7, 9, 10, 11, 12, 13 and 14. Table 10 compares the percentage success rates of the proposed RGA-PSO and AIA-PSO approaches and those of the hybrid algorithms for 11 TPs, indicating that all algorithms except for the CHA and CTSS methods achieved identical performance (100% success rate) for all TPs.Table 9
Comparison of the results of the proposed RGA-PSO and AIA-PSO approaches and those of the hybrid algorithms for 11 TPs.
TP number
Functionname
M
Es
NM-PSO [9]
PSACO [10]
GA-PSO [9]
DE-PSO [12]
AM-PSO1 [12]
AM-PSO2 [12]
CHA [10]
CTSS [10]
AIA-PSO(this work)
RGA-PSO(this work)
2
GP
0.00003
0.00000000
0.00012
3.67
E
-
5
3.81
E
-
5
9.89
E
-
6
0.0010
0.001
0.00000000
0.00000000
3
ES
0.00004
0.00000000
0.00003
1.67
E
-
5
4.10
E
-
5
3.71
E
-
5
0.0010
0.005
0.00000000
0.00000000
4
B2
0.00003
5.5511
E
-
17
0.00001
1.11
E
-
5
1.17
E
-
6
2.64
E
-
5
0.0000002
0.000005
0.00000000
0.00000000
5
DJ
0.00003
7.6900
E
-
29
0.00004
1.67
E
-
5
4.23
E
-
5
3.34
E
-
5
0.0002
0.0002
2.544
E
-
215
5.5331
E
-
226
7
RC
0.00003
2.6185
E
-
13
0.00009
2.29
E
-
6
1.52
E
-
5
1.49
E
-
5
0.0001
0.005
0.00000000
0.00000000
9
RS2
0.00003
1.7152
E
-
10
0.00064
2.46
E
-
5
2.16
E
-
5
3.33
E
-
5
0.0040
0.0040
2.341
E
-
27
8.709
E
-
28
10
RS5
0.0056
1.8538
E
-
04
0.00013
7.74
E
-
5
9.73
E
-
5
9.93
E
-
5
—
—
7.291
E
-
06
9.452
E
-
06
11
SH
0.00002
1.09239
E
-
09
0.00007
3.03
E
-
6
1.72
E
-
5
2.54
E
-
7
0.0050
0.001
0.00000000
0.00000000
12
ZA2
0.00003
5.7061
E
-
27
0.00005
1.12
E
-
5
2.54
E
-
5
2.61
Z
E
-
6
3
E
-
6
3
E
-
7
0.00000000
0.00000000
13
ZA5
0.00026
3.6352
E
-
17
0.00000
3.49
E
-
5
3.44
E
-
5
5.07
E
-
5
—
—
7.259
E
-
184
2.660
E
-
117
14
ZA10
—
2
E
-
08
0.00000
6.91
E
-
5
7.41
E
-
5
9.11
E
-
5
1
E
-
6
—
3.287
E
-
42
4.3756
E
-
26
(“—” denotes unavailable information).Table 10
Comparison of the success rate % of the proposed RGA-PSO and AIA-PSO approaches and those of the hybrid algorithms for 11 TPs.
TP number
Function name
Success rate %
NM-PSO [9]
PSACO [10]
GA-PSO [9]
DE-PSO [12]
AM-PSO1 [12]
AM-PSO2 [12]
CHA [10]
CTSS [10]
AIA-PSO(this work)
RGA-PSO(this work)
2
GP
100
100
100
100
100
100
100
100
100
100
3
ES
100
100
100
100
100
100
100
100
100
100
4
B2
100
100
100
100
100
100
100
100
100
100
5
DJ
100
100
100
100
100
100
100
100
100
100
7
RC
100
100
100
100
100
100
100
100
100
100
9
RS2
100
100
100
100
100
100
100
100
100
100
10
RS5
100
100
100
100
100
100
—
—
100
100
11
SH
100
100
100
100
100
100
100
100
100
100
12
ZA2
100
100
100
100
100
100
100
100
100
100
13
ZA5
100
100
100
100
100
100
—
—
100
100
14
ZA10
100
100
100
100
100
100
100
—
100
100
(“—” denotes unavailable information).


## 4.4. Summary of Results

The proposed RGA-PSO and AIA-PSO algorithms have the following benefits.(1)
Parameter manipulation of the internal PSO algorithm is based on the solved UGO problems. Owing to their ability to efficiently solve UGO problems, the external RGA and AIA approaches are substituted for trial and error to manipulate the parameters (χ, c1, and c2).(2)
Besides obtaining the optimum parameter settings of the internal PSO algorithm, the RGA-PSO and AIA-PSO algorithms can yield a global minimum for an UGO problem.(3)
Beside, outperforming some published hybrid SGO methods, the proposed RGA-PSO and AIA-PSO approaches reduce the parametrization for the internal PSO algorithm, despite being more complex than individual SGO approaches.The proposed RGA-PSO and AIA-PSO algorithms are limited in that they cannot solve high-dimensional UGO problems (such asN=50). Future work will focus on increasing the diversity of the particle swarm of the internal PSO method by applying mutation to solve high-dimensional UGO problems.


## 5. Conclusions

This work developed RGA-PSO and AIA-PSO algorithms. Performances of the proposed RGA-PSO and AIA-PSO approaches were evaluated using a set of benchmark UGO problems. Numerical results indicate that the proposed RGA-PSO and AIA-PSO methods can converge to global minimum for each test UGO problem and obtain the best parameter settings of the internal PSO algorithm. Moreover, the numerical results obtained using the RGA-PSO and AIA-PSO algorithms are superior to those obtained using many alternative hybrid SGO methods. The RGA-PSO and AIA-PSO methods can thus be considered efficient SGO approaches for solving standard-dimensional UGO problems.



---

*Source: 256180-2013-03-14.xml*