# Solving Constrained Global Optimization Problems by Using Hybrid Evolutionary Computing and Artificial Life Approaches

Authors: Jui-Yu Wu
Journal: Mathematical Problems in Engineering
 (2012)

---

## Abstract

This work presents a hybrid real-coded genetic algorithm with a particle swarm optimization (RGA-PSO) algorithm and a hybrid artificial immune algorithm with a PSO (AIA-PSO) algorithm for solving 13 constrained global optimization (CGO) problems, including six nonlinear programming and seven generalized polynomial programming optimization problems. External RGA and AIA approaches are used to optimize the constriction coefficient, cognitive parameter, social parameter, penalty parameter, and mutation probability of an internal PSO algorithm. CGO problems are then solved using the internal PSO algorithm. The performances of the proposed RGA-PSO and AIA-PSO algorithms are evaluated using 13 CGO problems. Moreover, numerical results obtained using the proposed RGA-PSO and AIA-PSO algorithms are compared with those obtained using published individual GA and AIA approaches. Experimental results indicate that the proposed RGA-PSO and AIA-PSO algorithms converge to a global optimum solution to a CGO problem. Furthermore, the optimum parameter settings of the internal PSO algorithm can be obtained using the external RGA and AIA approaches. Also, the proposed RGA-PSO and AIA-PSO algorithms outperform some published individual GA and AIA approaches. Therefore, the proposed RGA-PSO and AIA-PSO algorithms are highly promising stochastic global optimization methods for solving CGO problems.


---

## Body

## 1. Introduction

Many scientific, engineering, and management problems can be expressed as constrained global optimization (CGO) problems, as follows:(1.1)Minimizef(x),s.t.gm(x)≤0,m=1,2,…,M,hk(x)=0,k=1,2,…,K,xnl≤xn≤xnu,n=1,2,…,N,
where f(x) denotes an objective function; gm(x) represents a set of m nonlinear inequality constraints; hk(x) refers to a set of k nonlinear equality constraints; x represents a vector of decision variables which take real values, and each decision variable xn is constrained by its lower and upper boundaries [xnl,xnu]; N is the total number of decision variables xn. For instance, generalized polynomial programming (GPP) belongs to the nonlinear programming (NLP) method. The formulation of GPP is a nonconvex objective function subject to nonconvex inequality constraints and possibly disjointed feasible region. The GPP approach has been successfully used to solve problems including alkylation process design, heat exchanger design, optimal reactor design [1], inventory decision problem (economic production quantity) [2], process synthesis and the design of separations, phase equilibrium, nonisothermal complex reactor networks, and molecular conformation [3].Traditional local NLP optimization approaches based on a gradient algorithm are inefficient for solving CGO problems, while an objective function is nondifferentiable. Global optimization methods can be divided into deterministic or stochastic [4]. Often involving a sophisticated optimization process, deterministic global optimization methods typically make assumptions regarding the problem to be solved [5]. Stochastic global optimization methods that do not require gradient information and numerous assumptions have received considerable attention. For instance, Sun et al. [6] devised an improved vector particle swarm optimization (PSO) algorithm with a constraint-preserving method to solve CGO problems. Furthermore, Tsoulos [7] developed a real-coded genetic algorithm (RGA) with a penalty function approach for solving CGO problems. Additionally, Deep and Dipti [8] presented a self-organizing GA with a tournament selection method for solving CGO problems. Meanwhile, Wu and Chung [9] developed a RGA with a static penalty function approach for solving GPP optimization problems. Finally, Wu [10] introduced an artificial immune algorithm (AIA) with an adaptive penalty function method to solve CGO problems.Zadeh [11] defined “soft computing” as the synergistic power of two or more fused computational intelligence (CI) schemes, which can be divided into several branches: granular computing (e.g., fuzzy sets, rough sets, and probabilistic reasoning), neurocomputing (e.g., supervised, unsupervised, and reinforcement neural learning algorithms), evolutionary computing (e.g., GAs, genetic programming, and PSO algorithms), and artificial life (e.g., artificial immune systems) [12]. Besides, outperforming individual algorithms in terms of solving certain problems, hybrid algorithms can solve general problems more efficiently [13]. Therefore, hybrid CI approaches have recently attracted considerable attention as a promising field of research. Various hybrid evolutionary computing (GA and PSO methods) and artificial life (such as AIA methods) approaches have been developed for solving optimization problems. These hybrid algorithms focus on developing diverse candidate solutions (such as chromosomes and particles) of population/swarm to solve optimization problems more efficiently. These hybrid algorithms use two different algorithms to create diverse candidate solutions using their specific operations and then merge these diverse candidate solutions to increase the diversity of the candidate population. For instance, Abd-El-Wahed et al. [14] developed an integrated PSO algorithm and GA to solve nonlinear optimization problems. Additionally, Kuo and Han [15] presented a hybrid GA and PSO algorithm for bilevel linear programming to solve a supply chain distribution problem. Furthermore, Shelokar et al. [16] presented a hybrid PSO method and ant colony optimization method for solving continuous optimization problems. Finally, Hu et al. [17] developed an immune cooperative PSO algorithm for solving the fault-tolerant routing problem.Compared to the above hybrid CI algorithms, this work optimizes the parameter settings of an individual CI method by using another individual CI algorithm. A standard PSO algorithm has certain limitations [17, 18]. For instance, a PSO algorithm includes many parameters that must be set, such as the cognitive parameter, social parameter, and constriction coefficient. In practice, the optimal parameter settings of a PSO algorithm are tuned based on trial and error and prior knowledge is required to successfully manipulate the cognitive parameter, social parameter, and constriction coefficient. The exploration and exploitative capabilities of a PSO algorithm are limited to optimum parameter settings. Moreover, conventional PSO methods involve premature convergence that rapidly losses diversity during optimization.Fortunately, optimization of parameter settings for a conventional PSO algorithm can be considered an unconstrained global optimization (UGO) problem, and the diversity of candidate solutions of the PSO method can be increased using a multi-nonuniform mutation operation [19]. Moreover, the parameter manipulation of a GA and AIA method is easy to implement without prior knowledge. Therefore, to overcome the limitations of a standard PSO algorithm, this work develops two hybrid CI algorithms to solve CGO problems efficiently. The first algorithm is a hybrid RGA and PSO (RGA-PSO) algorithm, while the second algorithm is a hybrid AIA and PSO (AIA-PSO) algorithm. The proposed RGA-PSO and AIA-PSO algorithms are considered to optimize two optimization problems simultaneously. The UGO problem (optimization of cognitive parameter, social parameter, constriction coefficient, penalty parameter, and mutation probability of an internal PSO algorithm based on a penalty function approach) is optimized using external RGA and AIA approaches, respectively. A CGO problem is then solved using the internal PSO algorithm. The performances of the proposed RGA-PSO and AIA-PSO algorithms are evaluated using a set of CGO problems (e.g., six benchmark NLP and seven GPP optimization problems).The rest of this paper is organized as follows. Section2 describes the RGA, PSO algorithm, AIA, and penalty function approaches. Section 3 then introduces the proposed RGA-PSO and AIA-PSO algorithms. Next, Section 4 compares the experimental results of the proposed RGA-PSO and AIA-PSO algorithms with those of various published individual GAs and AIAs [9, 10, 20–22] and hybrid algorithms [23, 24]. Finally, conclusions are drawn in Section 5.


## 2. Related Works

### 2.1. Real-Coded Genetic Algorithm

GAs are stochastic global optimization methods based on the concepts of natural selection and use three genetic operators, that is, selection, crossover, and mutation, to explore and exploit the solution space. RGA outperforms binary-coded GA in solving continuous function optimization problems [19]. This work thus describes operators of a RGA [25].


#### 2.1.1. Selection

A selection operation selects strong individuals from a current population based on their fitness function values and then reproduces these individuals into a crossover pool. The several selection operations developed include the roulette wheel, ranking, and tournament methods [19, 25]. This work uses the normalized geometric ranking method, as follows:
(2.1)pj=q′(1-q)r-1,j=1,2,…,psRGA,pj= probability of selecting individual j,q= probability of choosing the best individual (here q=0.35)
(2.2)q′=q1-(1-q)psRGA,r= individual ranking based on fitness value, where 1 represents the best, r=1,2,…,psRGA, psRGA = population size of the RGA.


#### 2.1.2. Crossover

While exploring the solution space by creating new offspring, the crossover operation randomly selects two parents from the crossover pool and then uses these two parents to generate two new offspring. This operation is repeated until thepsRGA/2 is satisfied. The whole arithmetic crossover is easily implemented, as follows:
(2.3)v1′=β×v1+(1-β)×v2,v2′=(1-β)v1+β×v2,
where v1 and v2= parents, v1′ and v2′= offspring, β= uniform random number in the interval [0,1.5].


#### 2.1.3. Mutation

Mutation operation can increase the diversity of individuals (candidate solutions). Multi-nonuniform mutation is described as follows:(2.4)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gRGA)ifU1(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gRGA)ifU1(0,1)≥0.5,
where pert(gRGA)=[U2(0,1)(1-gRGA/gmax,RGA)]2, perturbed factor, U1(0,1) and U2(0,1) = uniform random variable in the interval [0,1], gmax,RGA = maximum generation of the RGA, gRGA = current generation of the RGA, xcurrent,n = current decision variable xn, xtrial,n = trial candidate solution xn.


### 2.2. Particle Swarm Optimization

Kennedy and Eberhart [26] first introduced a conventional PSO algorithm, which is inspired by the social behavior of bird flocks or fish schools. Like GAs, a PSO algorithm is a population-based algorithm. A population of candidate solutions is called a particle swarm. The particle velocities can be updated by (2.5), as follows:
(2.5)vj,n(gPSO+1)=vj,n(gPSO)+c1r1j(gPSO)[pj,nlb(gPSO)-xj,n(gPSO)]+c2r2j(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]j=1,2,…,psPSO,n=1,2,…,N,vj,n(gPSO+1) = particle velocity of decision variable xn of particle j at generation gPSO+1, vj,n(gPSO)= particle velocity of decision variable xn of particle j at generation gPSO, c1= cognitive parameter, c2= social parameter, xj,n(gPSO)= particle position of decision variable xn of particle j at generation gPSO, r1,j(gPSO),r2,j(gPSO)= independent uniform random numbers in the interval [0,1] at generation gPSO, pj,nlb(gPSO)= best local solution at generation gPSO,pj,ngb(gPSO)= best global solution at generation gPSO, psPSO= population size of the PSO algorithm.The particle positions can be computed using (2.6), as follows:
(2.6)xj,n(gPSO+1)=xj,n(gPSO)+vj,n(gPSO+1)j=1,2,…,psPSO,n=1,2,…,N.Shi and Eberhart [27] developed a modified PSO algorithm by incorporating an inertia weight (ωin) into (2.7) to control the exploration and exploitation capabilities of a PSO algorithm, as follows:
(2.7)vj,n(gPSO+1)=ωinvj,n(gPSO)+c1r1j(gPSO)[pj,nlb(gPSO)-xj,n(gPSO)]+c2r2j(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]j=1,2,…,psPSO,n=1,2,…,N.A constriction coefficient (χ) was inserted into (2.8) to balance the exploration and exploitation tradeoff [28–30], as follows:
(2.8)vj,n(gPSO+1)=χ{pj,ngbvj,n(gPSO)+ρ1(gPSO)[pj,nlb(gPSO)-xj,n(gPSO)]+ρ2(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]}j=1,2,…,psPSO,n=1,2,…,N,
where
(2.9)χ=2U3(0,1)|2-τ-τ(τ-4)|,U3(0,1)= uniform random variable in the interval [0,1],τ=τ1+τ2,τ1=c1r1j,τ1=c2r2j.This work considers parametersωin and χ to update the particle velocities, as follows:
(2.10)vj,n(gPSO+1)=χ{pj,ngbωinvj,n(gPSO)+c1r1j(gPSO)[pj,nlb(gPSO)-xj,n(gPSO)]+c2r2j(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]}j=1,2,…,psPSO,n=1,2,…,N,
where ωin=((gmax,PSO-gPSO)/gmax,PSO), increased gPSO value reduces the ωin, gmax,PSO= maximum generation of the PSO algorithm.According to (2.10), the optimal values of parameters c1, c2, and χ are difficult to obtain through a trial and error. This work thus optimizes these parameter settings by using RGA and AIA approaches.


### 2.3. Artificial Immune Algorithm

Wu [10] presented an AIA based on clonal selection and immune network theories to solve CGO problems. The AIA approach comprises selection, hypermutation, receptor editing, and bone marrow operations. The selection operation is performed to reproduce strong antibodies (Abs). Also, diverse Abs are created using hypermutation, receptor editing, and bone marrow operations, as described in the following subsections.


#### 2.3.1.Ab and Ag Representation

In the human immune system, an antigen (Ag) has multiple epitopes (antigenic determinants), which can be recognized by various Abswith paratopes (recognizers), on its surface. In the AIA approach, an Ag represents known parameters of a solved problem. The Abs are the candidate solutions (i.e., decision variables xn, n=1,2,…,N) of the solved problem. The quality of a candidate solution is evaluated using an Ab-Ag affinity that is derived from the value of an objective function of the solved problem.


#### 2.3.2. Selection Operation

The selection operation, which is based on the immune network principle [31], controls the number of antigen-specific Abs. This operation is defined according to Ab-Ag and Ab-Ab recognition information, as follows:
(2.11)prj=1N∑n=1N1ednj,dnj=|xn*-xnjxn*|,j=1,2,…,rs,n=1,2,…,N,
where prj= probability that Abj recognizes Ab* (the best solution), xn*= the best Ab* with the highest Ab-Ag affinity, xnj= decision variables xn of Abj,rs= repertoire (population) size of the AIA.TheAb* is recognized by other Ab  j in a current Ab repertoire. Large prj implies that Ab  j can effectively recognize Ab*. The Ab  j with prj that is equivalent to or larger than the threshold degree prt is reproduced to generate an intermediate Ab repertoire.


#### 2.3.3. Hypermutation Operation

Multi-nonuniform mutation [19] is used as the somatic hypermutation operation, which can be expressed as follows:
(2.12)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gAIA),ifU4(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gAIA),ifU4(0,1)≥0.5,
where pert(gAIA)={U5(0,1)(1-gAIA/gmax,AIA)}2= perturbation factor, gAIA= current generation of the AIA, gmax,AIA= maximum generation number of the AIA, U4(0,1) and U5(0,1)= uniform random number in the interval [0,1].This operation has two tasks, that is, a uniform search and local fine-tuning.


#### 2.3.4. Receptor Editing Operation

A receptor editing operation is developed using the standard Cauchy distributionC(0,1), in which the local parameter is zero and the scale parameter is one. Receptor editing is performed using Cauchy random variables that are generated from C(0,1), owing to their ability to provide a large jump in the Ab-Ag affinity landscape to increase the probability of escaping from the local Ab-Ag affinity landscape. Cauchy receptor editing can be defined by
(2.13)xtrial=xcurrent+U5(0,1)2×σ,
where σ=[σ1,σ2,…,σN]T, vector of Cauchy random variables, U5(0,1)= uniform random number in the interval [0,1].This operation is employed in local fine-tuning and large perturbation.


#### 2.3.5. Bone Marrow Operation

The paratope of anAb can be generated by recombining gene segments VHDHJH and VLJL [32]. Therefore, based on this metaphor, diverse Abs are synthesized using a bone marrow operation. This operation randomly chooses two Abs from the intermediate Ab repertoire and a recombination point from the gene segments of the paratope of the selected Abs. The selected gene segments (e.g., gene x1 of Ab 1 and gene x1 of the Ab 2) are reproduced to create a library of gene segments. The selected gene segments in the paratope are then deleted. The new Ab  1  is formed by inserting the gene segment, which is gene x1 of the Ab 2 in the library plus a random variable created from standard normal distribution N(0,1), at the recombination point. The literature details the implementation of the bone marrow [10].


### 2.4. Penalty Function Methods

Stochastic global optimization approaches, including GAs, AIAs, and PSO, are naturally unconstrained optimization methods. Penalty function methods, which are constraint handling approaches, are commonly used to create feasible solutions to a CGO problem and transform it into an unconstrained optimization problem. Two popular penalty functions exist, namely, the exterior and interior functions. Exterior penalty functions use an infeasible solution as a starting point, and convergence is from the infeasible region to the feasible one. Interior penalty functions start from a feasible solution, then move from the feasible region to the constrained boundaries. Exterior penalty functions are favored over interior penalty functions, because they do not require a feasible starting point and are easily implemented. The exterior penalty functions developed to date include static, dynamic, adaptive, and death penalty functions [33]. This work uses the form of a static penalty function, as follows:
(2.14)Minimizefpseudo(x,ρ)=f(x)+ρ{∑m=1M{max[0,gm(x)]+∑k=1K[hk(x)]}2},
where fpseudo(x,ρ)= pseudo-objective function obtained using an original objective function plus a penalty term, ρ= penalty parameter.Unfortunately, the penalty function scheme is limited by the need to fine-tune the penalty parameterρ [8]. To overcome this limitation, this work attempts to find the optimum ρ for each CGO problem using the RGA and AIA approaches. Additionally, to obtain high-quality RGA-PSO and AIA-PSO solutions accurate to at least five decimal places for the violation of each constraint to a specific CGO problem, the parameter ρ is within the search space [1×109,1×1011].


## 2.1. Real-Coded Genetic Algorithm

GAs are stochastic global optimization methods based on the concepts of natural selection and use three genetic operators, that is, selection, crossover, and mutation, to explore and exploit the solution space. RGA outperforms binary-coded GA in solving continuous function optimization problems [19]. This work thus describes operators of a RGA [25].


### 2.1.1. Selection

A selection operation selects strong individuals from a current population based on their fitness function values and then reproduces these individuals into a crossover pool. The several selection operations developed include the roulette wheel, ranking, and tournament methods [19, 25]. This work uses the normalized geometric ranking method, as follows:
(2.1)pj=q′(1-q)r-1,j=1,2,…,psRGA,pj= probability of selecting individual j,q= probability of choosing the best individual (here q=0.35)
(2.2)q′=q1-(1-q)psRGA,r= individual ranking based on fitness value, where 1 represents the best, r=1,2,…,psRGA, psRGA = population size of the RGA.


### 2.1.2. Crossover

While exploring the solution space by creating new offspring, the crossover operation randomly selects two parents from the crossover pool and then uses these two parents to generate two new offspring. This operation is repeated until thepsRGA/2 is satisfied. The whole arithmetic crossover is easily implemented, as follows:
(2.3)v1′=β×v1+(1-β)×v2,v2′=(1-β)v1+β×v2,
where v1 and v2= parents, v1′ and v2′= offspring, β= uniform random number in the interval [0,1.5].


### 2.1.3. Mutation

Mutation operation can increase the diversity of individuals (candidate solutions). Multi-nonuniform mutation is described as follows:(2.4)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gRGA)ifU1(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gRGA)ifU1(0,1)≥0.5,
where pert(gRGA)=[U2(0,1)(1-gRGA/gmax,RGA)]2, perturbed factor, U1(0,1) and U2(0,1) = uniform random variable in the interval [0,1], gmax,RGA = maximum generation of the RGA, gRGA = current generation of the RGA, xcurrent,n = current decision variable xn, xtrial,n = trial candidate solution xn.


## 2.1.1. Selection

A selection operation selects strong individuals from a current population based on their fitness function values and then reproduces these individuals into a crossover pool. The several selection operations developed include the roulette wheel, ranking, and tournament methods [19, 25]. This work uses the normalized geometric ranking method, as follows:
(2.1)pj=q′(1-q)r-1,j=1,2,…,psRGA,pj= probability of selecting individual j,q= probability of choosing the best individual (here q=0.35)
(2.2)q′=q1-(1-q)psRGA,r= individual ranking based on fitness value, where 1 represents the best, r=1,2,…,psRGA, psRGA = population size of the RGA.


## 2.1.2. Crossover

While exploring the solution space by creating new offspring, the crossover operation randomly selects two parents from the crossover pool and then uses these two parents to generate two new offspring. This operation is repeated until thepsRGA/2 is satisfied. The whole arithmetic crossover is easily implemented, as follows:
(2.3)v1′=β×v1+(1-β)×v2,v2′=(1-β)v1+β×v2,
where v1 and v2= parents, v1′ and v2′= offspring, β= uniform random number in the interval [0,1.5].


## 2.1.3. Mutation

Mutation operation can increase the diversity of individuals (candidate solutions). Multi-nonuniform mutation is described as follows:(2.4)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gRGA)ifU1(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gRGA)ifU1(0,1)≥0.5,
where pert(gRGA)=[U2(0,1)(1-gRGA/gmax,RGA)]2, perturbed factor, U1(0,1) and U2(0,1) = uniform random variable in the interval [0,1], gmax,RGA = maximum generation of the RGA, gRGA = current generation of the RGA, xcurrent,n = current decision variable xn, xtrial,n = trial candidate solution xn.


## 2.2. Particle Swarm Optimization

Kennedy and Eberhart [26] first introduced a conventional PSO algorithm, which is inspired by the social behavior of bird flocks or fish schools. Like GAs, a PSO algorithm is a population-based algorithm. A population of candidate solutions is called a particle swarm. The particle velocities can be updated by (2.5), as follows:
(2.5)vj,n(gPSO+1)=vj,n(gPSO)+c1r1j(gPSO)[pj,nlb(gPSO)-xj,n(gPSO)]+c2r2j(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]j=1,2,…,psPSO,n=1,2,…,N,vj,n(gPSO+1) = particle velocity of decision variable xn of particle j at generation gPSO+1, vj,n(gPSO)= particle velocity of decision variable xn of particle j at generation gPSO, c1= cognitive parameter, c2= social parameter, xj,n(gPSO)= particle position of decision variable xn of particle j at generation gPSO, r1,j(gPSO),r2,j(gPSO)= independent uniform random numbers in the interval [0,1] at generation gPSO, pj,nlb(gPSO)= best local solution at generation gPSO,pj,ngb(gPSO)= best global solution at generation gPSO, psPSO= population size of the PSO algorithm.The particle positions can be computed using (2.6), as follows:
(2.6)xj,n(gPSO+1)=xj,n(gPSO)+vj,n(gPSO+1)j=1,2,…,psPSO,n=1,2,…,N.Shi and Eberhart [27] developed a modified PSO algorithm by incorporating an inertia weight (ωin) into (2.7) to control the exploration and exploitation capabilities of a PSO algorithm, as follows:
(2.7)vj,n(gPSO+1)=ωinvj,n(gPSO)+c1r1j(gPSO)[pj,nlb(gPSO)-xj,n(gPSO)]+c2r2j(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]j=1,2,…,psPSO,n=1,2,…,N.A constriction coefficient (χ) was inserted into (2.8) to balance the exploration and exploitation tradeoff [28–30], as follows:
(2.8)vj,n(gPSO+1)=χ{pj,ngbvj,n(gPSO)+ρ1(gPSO)[pj,nlb(gPSO)-xj,n(gPSO)]+ρ2(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]}j=1,2,…,psPSO,n=1,2,…,N,
where
(2.9)χ=2U3(0,1)|2-τ-τ(τ-4)|,U3(0,1)= uniform random variable in the interval [0,1],τ=τ1+τ2,τ1=c1r1j,τ1=c2r2j.This work considers parametersωin and χ to update the particle velocities, as follows:
(2.10)vj,n(gPSO+1)=χ{pj,ngbωinvj,n(gPSO)+c1r1j(gPSO)[pj,nlb(gPSO)-xj,n(gPSO)]+c2r2j(gPSO)[pj,ngb(gPSO)-xj,n(gPSO)]}j=1,2,…,psPSO,n=1,2,…,N,
where ωin=((gmax,PSO-gPSO)/gmax,PSO), increased gPSO value reduces the ωin, gmax,PSO= maximum generation of the PSO algorithm.According to (2.10), the optimal values of parameters c1, c2, and χ are difficult to obtain through a trial and error. This work thus optimizes these parameter settings by using RGA and AIA approaches.


## 2.3. Artificial Immune Algorithm

Wu [10] presented an AIA based on clonal selection and immune network theories to solve CGO problems. The AIA approach comprises selection, hypermutation, receptor editing, and bone marrow operations. The selection operation is performed to reproduce strong antibodies (Abs). Also, diverse Abs are created using hypermutation, receptor editing, and bone marrow operations, as described in the following subsections.


### 2.3.1.Ab and Ag Representation

In the human immune system, an antigen (Ag) has multiple epitopes (antigenic determinants), which can be recognized by various Abswith paratopes (recognizers), on its surface. In the AIA approach, an Ag represents known parameters of a solved problem. The Abs are the candidate solutions (i.e., decision variables xn, n=1,2,…,N) of the solved problem. The quality of a candidate solution is evaluated using an Ab-Ag affinity that is derived from the value of an objective function of the solved problem.


### 2.3.2. Selection Operation

The selection operation, which is based on the immune network principle [31], controls the number of antigen-specific Abs. This operation is defined according to Ab-Ag and Ab-Ab recognition information, as follows:
(2.11)prj=1N∑n=1N1ednj,dnj=|xn*-xnjxn*|,j=1,2,…,rs,n=1,2,…,N,
where prj= probability that Abj recognizes Ab* (the best solution), xn*= the best Ab* with the highest Ab-Ag affinity, xnj= decision variables xn of Abj,rs= repertoire (population) size of the AIA.TheAb* is recognized by other Ab  j in a current Ab repertoire. Large prj implies that Ab  j can effectively recognize Ab*. The Ab  j with prj that is equivalent to or larger than the threshold degree prt is reproduced to generate an intermediate Ab repertoire.


### 2.3.3. Hypermutation Operation

Multi-nonuniform mutation [19] is used as the somatic hypermutation operation, which can be expressed as follows:
(2.12)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gAIA),ifU4(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gAIA),ifU4(0,1)≥0.5,
where pert(gAIA)={U5(0,1)(1-gAIA/gmax,AIA)}2= perturbation factor, gAIA= current generation of the AIA, gmax,AIA= maximum generation number of the AIA, U4(0,1) and U5(0,1)= uniform random number in the interval [0,1].This operation has two tasks, that is, a uniform search and local fine-tuning.


### 2.3.4. Receptor Editing Operation

A receptor editing operation is developed using the standard Cauchy distributionC(0,1), in which the local parameter is zero and the scale parameter is one. Receptor editing is performed using Cauchy random variables that are generated from C(0,1), owing to their ability to provide a large jump in the Ab-Ag affinity landscape to increase the probability of escaping from the local Ab-Ag affinity landscape. Cauchy receptor editing can be defined by
(2.13)xtrial=xcurrent+U5(0,1)2×σ,
where σ=[σ1,σ2,…,σN]T, vector of Cauchy random variables, U5(0,1)= uniform random number in the interval [0,1].This operation is employed in local fine-tuning and large perturbation.


### 2.3.5. Bone Marrow Operation

The paratope of anAb can be generated by recombining gene segments VHDHJH and VLJL [32]. Therefore, based on this metaphor, diverse Abs are synthesized using a bone marrow operation. This operation randomly chooses two Abs from the intermediate Ab repertoire and a recombination point from the gene segments of the paratope of the selected Abs. The selected gene segments (e.g., gene x1 of Ab 1 and gene x1 of the Ab 2) are reproduced to create a library of gene segments. The selected gene segments in the paratope are then deleted. The new Ab  1  is formed by inserting the gene segment, which is gene x1 of the Ab 2 in the library plus a random variable created from standard normal distribution N(0,1), at the recombination point. The literature details the implementation of the bone marrow [10].


## 2.3.1.Ab and Ag Representation

In the human immune system, an antigen (Ag) has multiple epitopes (antigenic determinants), which can be recognized by various Abswith paratopes (recognizers), on its surface. In the AIA approach, an Ag represents known parameters of a solved problem. The Abs are the candidate solutions (i.e., decision variables xn, n=1,2,…,N) of the solved problem. The quality of a candidate solution is evaluated using an Ab-Ag affinity that is derived from the value of an objective function of the solved problem.


## 2.3.2. Selection Operation

The selection operation, which is based on the immune network principle [31], controls the number of antigen-specific Abs. This operation is defined according to Ab-Ag and Ab-Ab recognition information, as follows:
(2.11)prj=1N∑n=1N1ednj,dnj=|xn*-xnjxn*|,j=1,2,…,rs,n=1,2,…,N,
where prj= probability that Abj recognizes Ab* (the best solution), xn*= the best Ab* with the highest Ab-Ag affinity, xnj= decision variables xn of Abj,rs= repertoire (population) size of the AIA.TheAb* is recognized by other Ab  j in a current Ab repertoire. Large prj implies that Ab  j can effectively recognize Ab*. The Ab  j with prj that is equivalent to or larger than the threshold degree prt is reproduced to generate an intermediate Ab repertoire.


## 2.3.3. Hypermutation Operation

Multi-nonuniform mutation [19] is used as the somatic hypermutation operation, which can be expressed as follows:
(2.12)xtrial,n={xcurrent,n+(xnu-xcurrent,n)pert(gAIA),ifU4(0,1)<0.5,xcurrent,n-(xcurrent,n-xnl)pert(gAIA),ifU4(0,1)≥0.5,
where pert(gAIA)={U5(0,1)(1-gAIA/gmax,AIA)}2= perturbation factor, gAIA= current generation of the AIA, gmax,AIA= maximum generation number of the AIA, U4(0,1) and U5(0,1)= uniform random number in the interval [0,1].This operation has two tasks, that is, a uniform search and local fine-tuning.


## 2.3.4. Receptor Editing Operation

A receptor editing operation is developed using the standard Cauchy distributionC(0,1), in which the local parameter is zero and the scale parameter is one. Receptor editing is performed using Cauchy random variables that are generated from C(0,1), owing to their ability to provide a large jump in the Ab-Ag affinity landscape to increase the probability of escaping from the local Ab-Ag affinity landscape. Cauchy receptor editing can be defined by
(2.13)xtrial=xcurrent+U5(0,1)2×σ,
where σ=[σ1,σ2,…,σN]T, vector of Cauchy random variables, U5(0,1)= uniform random number in the interval [0,1].This operation is employed in local fine-tuning and large perturbation.


## 2.3.5. Bone Marrow Operation

The paratope of anAb can be generated by recombining gene segments VHDHJH and VLJL [32]. Therefore, based on this metaphor, diverse Abs are synthesized using a bone marrow operation. This operation randomly chooses two Abs from the intermediate Ab repertoire and a recombination point from the gene segments of the paratope of the selected Abs. The selected gene segments (e.g., gene x1 of Ab 1 and gene x1 of the Ab 2) are reproduced to create a library of gene segments. The selected gene segments in the paratope are then deleted. The new Ab  1  is formed by inserting the gene segment, which is gene x1 of the Ab 2 in the library plus a random variable created from standard normal distribution N(0,1), at the recombination point. The literature details the implementation of the bone marrow [10].


## 2.4. Penalty Function Methods

Stochastic global optimization approaches, including GAs, AIAs, and PSO, are naturally unconstrained optimization methods. Penalty function methods, which are constraint handling approaches, are commonly used to create feasible solutions to a CGO problem and transform it into an unconstrained optimization problem. Two popular penalty functions exist, namely, the exterior and interior functions. Exterior penalty functions use an infeasible solution as a starting point, and convergence is from the infeasible region to the feasible one. Interior penalty functions start from a feasible solution, then move from the feasible region to the constrained boundaries. Exterior penalty functions are favored over interior penalty functions, because they do not require a feasible starting point and are easily implemented. The exterior penalty functions developed to date include static, dynamic, adaptive, and death penalty functions [33]. This work uses the form of a static penalty function, as follows:
(2.14)Minimizefpseudo(x,ρ)=f(x)+ρ{∑m=1M{max[0,gm(x)]+∑k=1K[hk(x)]}2},
where fpseudo(x,ρ)= pseudo-objective function obtained using an original objective function plus a penalty term, ρ= penalty parameter.Unfortunately, the penalty function scheme is limited by the need to fine-tune the penalty parameterρ [8]. To overcome this limitation, this work attempts to find the optimum ρ for each CGO problem using the RGA and AIA approaches. Additionally, to obtain high-quality RGA-PSO and AIA-PSO solutions accurate to at least five decimal places for the violation of each constraint to a specific CGO problem, the parameter ρ is within the search space [1×109,1×1011].


## 3. Method

### 3.1. RGA-PSO Algorithm

Figure1 shows the pseudocode of the proposed RGA-PSO algorithm. The external RGA approach is used to optimize the best parameter settings of the internal PSO algorithm, and the internal PSO algorithm is employed to solve CGO problems.Figure 1
The pseudocode of the proposed RGA-PSO algorithm.External RGAStep 1 (initialize the parameter settings).
Parameter settings are given such aspsRGA, crossover probability pc, mutation probability of the external RGA approach pm,RGA, the lower and upper boundaries of these parameters c1, c2, χ, ρ, and the mutation probability of the internal PSO algorithm pm,PSO. The candidate solutions (individuals) of the external RGA represent the optimized parameters of the internal PSO algorithm. Finally, Figure 2 illustrates the candidate solution of the external RGA approach.Figure 2
Chromosome representation of the external RGA.Step 2 (compute the fitness function value).
The fitness function valuefitnessj of the external RGA approach is the best objective function value f(xPSO*) obtained from the best solution xPSO* of each internal PSO algorithm execution, as follows:
(3.1)fitnessj=f(xPSO*),j=1,2,…,psRGA.Candidate solutionj of the external RGA approach is incorporated into the internal PSO algorithm, and a CGO problem is then solved using the internal PSO algorithm, which is executed as follows.Internal PSO Algorithm
Step (1)
(create an initial particle swarm). An initial particle swarm is created based on thepsPSO from [xnl, xnu] of a CGO problem. A particle represents a candidate solution of a CGO problem, as shown in Figure 3.
Step (2)
(calculate the objective function value). According to (2.14), the pseudo-objective function value of the internal PSO algorithm is defined by
(3.2)fpseudo,j=f(xPSO,j)+{ρ×∑m=1M{max[0,gm(xPSO,j)]}2},j=1,2,…,psPSO.
Step (3)
(update the particle velocity and position). The particle position and velocity can be updated using (2.6) and (2.10), respectively.
Step (4)
(implement a mutation operation). The standard PSO algorithm lacks evolution operations of GAs such as crossover and mutation. To maintain the diversity of particles, this work uses the multi-nonuniform mutation operator defined by (2.4).
Step (5)
(perform an elitist strategy). A new particle swarm is created from internal step (3). Notably,f(xPSO,j) of a candidate solution j (particle j) in the particle swarm is evaluated. Here, a pairwise comparison is made between the f(xPSO,j) value of candidate solutions in the new and current particle swarms. A situation in which the candidate solution j (j=1,2,…,psPSO) in the new particle swarm is superior to candidate solution j in the current particle swarm implies that the strong candidate solution j in the new particle swarm replaces the candidate solution j in the current particle swarm. The elitist strategy guarantees that the best candidate solution is always preserved in the next generation. The current particle swarm is updated to the particle swarm of the next generation.Figure 3
Candidate solution of the internal PSO algorithm.Internal steps (2) to (5) are repeated until thegmax,PSO value of the internal PSO algorithm is satisfied.EndStep 3 (implement selection operation).
The parents in a crossover pool are selected using (2.1).Step 4 (perform crossover operation).
In GAs, the crossover operation performs a global search. Thus, the crossover probabilitypc usually exceeds 0.5. Additionally, candidate solutions are created using (2.3).Step 5 (conduct mutation operation).
In GAs, the mutation operation implements a local search. Additionally, a solution space is exploited using (2.4).Step 6 (implement an elitist strategy).
This work updates the population using an elitist strategy. A situation in which thefitnessj of candidate solution j in the new population is larger than that in the current population suggests that the weak candidate solution j is replaced. Additionally, a situation in which the fitnessj of candidate solution j in the new population is equal to or worse than that in the current population implies that the candidate solution j in the current population survives. In addition to maintaining the strong candidate solutions, this strategy eliminates weak candidate solutions.External Steps2 to 6 are repeated until the gmax,RGA value of the external RGA approach is met.


### 3.2. AIA-PSO Algorithm

Figure4 shows the pseudocode of the proposed AIA-PSO algorithm, in which the external AIA approach is used to optimize the parameter settings of the internal PSO algorithm and the PSO algorithm is used to solve CGO problems.Figure 4
The pseudocode of the AIA-PSO algorithm.External AIAStep 1 (initialize the parameter settings).
Several parameters must be predetermined. These includers and the threshold for Ab-Ab recognition prt, as well as the lower and upper boundaries of these parameters c1, c2, χ, ρ, and pm,PSO. Figure 5 shows the Ab and Ag representation.Figure 5
Ag and Ab representation.Step 2 (evaluate theAb-Ag affinity).
Internal PSO Algorithm
The external AIA approach offers parameter settingsc1, c2, χ, ρ, and pm,PSO for the internal PSO algorithm, subsequently leading to the implementation of internal steps (1)–(5) of the PSO algorithm. The PSO algorithm returns the best fitness value of PSO f(xPSO*) to the external AIA approach. Step (1)
(create an initial particle swarm). An initial particle swarm is created based onpsPSO from [xnl, xnu] of a CGO problem. A particle represents a candidate solution of a CGO problem.Step (2)
(calculate the objective function value). Equation (3.2) is used as the pseudo-objective function value of the internal PSO algorithm.Step (3)
(update the particle velocity and position). Equations (2.6) and (2.10) can be used to update the particle position and velocity.Step (4)
(implement a mutation operation). The diversity of the particle swarm is increased using (2.4).Step (5)
(perform an elitist strategy). A new particle swarm (population) is generated from internal step(3). Notably, f(xPSO,j) of a candidate solution j (particle j) in the particle swarm is evaluated. Here, a pairwise comparison is made between the f(xPSO,j) value of candidate solutions in the new and current particle swarms. The elitist strategy guarantees that the best candidate solution is always preserved in the next generation. The current particle swarm is updated to the particle swarm of the next generation.Internal steps (2) to (5) are repeated until thegmax,PSO value of the internal PSO algorithm is satisfied.End
Consistent with theAb-Ag affinity metaphor, an Ab-Ag affinity is determined using (3.3), as follows:
(3.3)max(affinityj)=-1×f(xPSO*)j=1,2,…,rs.
Following the evaluation of the Ab-Ag affinities of Absin the current Ab repertoire, the Ab with the highest Ab-Ag affinity (Ab*) is chosen to undergo clonal selection operation in external Step 3.Step 3 (perform clonal selection operation).
To control the number of antigen-specificAbs, (2.11) is used.Step 4 (implementAb-Ag affinity maturation).
The intermediateAb repertoire that is created in external Step 3 is divided into two subsets. These Abs undergo somatic hypermutation operation by using (2.12) when the random number is 0.5 or less. Notably, these Abs suffer receptor editing operation using (2.13) when the random number exceeds 0.5.Step 5 (introduce diverseAbs).
Based on the bone marrow operation, diverseAbs are created to recruit the Abssuppressed in external Step 3.Step 6 (update anAb repertoire).
A newAbrepertoire is generated from external Steps 3–5. The Ab-Ag affinities of the Abs in the generated Ab repertoire are evaluated. This work presents a strategy for updating the Ab repertoire. A situation in which the Ab-Ag affinity of Ab  j in the new Ab repertoire exceeds that in the current Ab repertoire implies that a strong Ab in the new Ab repertoire replaces the weak Ab in the current Ab repertoire. Additionally, a situation in which the Ab-Ag affinity of Ab  j in the new Ab repertoire equals to or is worse than that in the current Ab repertoire implies that the Ab  j in the current Ab repertoire survives. In addition to maintaining the strong Abs, this strategy eliminates nonfunctional Abs.External Steps2–6 are repeated until the termination criterion gmax,AIA is satisfied.


## 3.1. RGA-PSO Algorithm

Figure1 shows the pseudocode of the proposed RGA-PSO algorithm. The external RGA approach is used to optimize the best parameter settings of the internal PSO algorithm, and the internal PSO algorithm is employed to solve CGO problems.Figure 1
The pseudocode of the proposed RGA-PSO algorithm.External RGAStep 1 (initialize the parameter settings).
Parameter settings are given such aspsRGA, crossover probability pc, mutation probability of the external RGA approach pm,RGA, the lower and upper boundaries of these parameters c1, c2, χ, ρ, and the mutation probability of the internal PSO algorithm pm,PSO. The candidate solutions (individuals) of the external RGA represent the optimized parameters of the internal PSO algorithm. Finally, Figure 2 illustrates the candidate solution of the external RGA approach.Figure 2
Chromosome representation of the external RGA.Step 2 (compute the fitness function value).
The fitness function valuefitnessj of the external RGA approach is the best objective function value f(xPSO*) obtained from the best solution xPSO* of each internal PSO algorithm execution, as follows:
(3.1)fitnessj=f(xPSO*),j=1,2,…,psRGA.Candidate solutionj of the external RGA approach is incorporated into the internal PSO algorithm, and a CGO problem is then solved using the internal PSO algorithm, which is executed as follows.Internal PSO Algorithm
Step (1)
(create an initial particle swarm). An initial particle swarm is created based on thepsPSO from [xnl, xnu] of a CGO problem. A particle represents a candidate solution of a CGO problem, as shown in Figure 3.
Step (2)
(calculate the objective function value). According to (2.14), the pseudo-objective function value of the internal PSO algorithm is defined by
(3.2)fpseudo,j=f(xPSO,j)+{ρ×∑m=1M{max[0,gm(xPSO,j)]}2},j=1,2,…,psPSO.
Step (3)
(update the particle velocity and position). The particle position and velocity can be updated using (2.6) and (2.10), respectively.
Step (4)
(implement a mutation operation). The standard PSO algorithm lacks evolution operations of GAs such as crossover and mutation. To maintain the diversity of particles, this work uses the multi-nonuniform mutation operator defined by (2.4).
Step (5)
(perform an elitist strategy). A new particle swarm is created from internal step (3). Notably,f(xPSO,j) of a candidate solution j (particle j) in the particle swarm is evaluated. Here, a pairwise comparison is made between the f(xPSO,j) value of candidate solutions in the new and current particle swarms. A situation in which the candidate solution j (j=1,2,…,psPSO) in the new particle swarm is superior to candidate solution j in the current particle swarm implies that the strong candidate solution j in the new particle swarm replaces the candidate solution j in the current particle swarm. The elitist strategy guarantees that the best candidate solution is always preserved in the next generation. The current particle swarm is updated to the particle swarm of the next generation.Figure 3
Candidate solution of the internal PSO algorithm.Internal steps (2) to (5) are repeated until thegmax,PSO value of the internal PSO algorithm is satisfied.EndStep 3 (implement selection operation).
The parents in a crossover pool are selected using (2.1).Step 4 (perform crossover operation).
In GAs, the crossover operation performs a global search. Thus, the crossover probabilitypc usually exceeds 0.5. Additionally, candidate solutions are created using (2.3).Step 5 (conduct mutation operation).
In GAs, the mutation operation implements a local search. Additionally, a solution space is exploited using (2.4).Step 6 (implement an elitist strategy).
This work updates the population using an elitist strategy. A situation in which thefitnessj of candidate solution j in the new population is larger than that in the current population suggests that the weak candidate solution j is replaced. Additionally, a situation in which the fitnessj of candidate solution j in the new population is equal to or worse than that in the current population implies that the candidate solution j in the current population survives. In addition to maintaining the strong candidate solutions, this strategy eliminates weak candidate solutions.External Steps2 to 6 are repeated until the gmax,RGA value of the external RGA approach is met.


## 3.2. AIA-PSO Algorithm

Figure4 shows the pseudocode of the proposed AIA-PSO algorithm, in which the external AIA approach is used to optimize the parameter settings of the internal PSO algorithm and the PSO algorithm is used to solve CGO problems.Figure 4
The pseudocode of the AIA-PSO algorithm.External AIAStep 1 (initialize the parameter settings).
Several parameters must be predetermined. These includers and the threshold for Ab-Ab recognition prt, as well as the lower and upper boundaries of these parameters c1, c2, χ, ρ, and pm,PSO. Figure 5 shows the Ab and Ag representation.Figure 5
Ag and Ab representation.Step 2 (evaluate theAb-Ag affinity).
Internal PSO Algorithm
The external AIA approach offers parameter settingsc1, c2, χ, ρ, and pm,PSO for the internal PSO algorithm, subsequently leading to the implementation of internal steps (1)–(5) of the PSO algorithm. The PSO algorithm returns the best fitness value of PSO f(xPSO*) to the external AIA approach. Step (1)
(create an initial particle swarm). An initial particle swarm is created based onpsPSO from [xnl, xnu] of a CGO problem. A particle represents a candidate solution of a CGO problem.Step (2)
(calculate the objective function value). Equation (3.2) is used as the pseudo-objective function value of the internal PSO algorithm.Step (3)
(update the particle velocity and position). Equations (2.6) and (2.10) can be used to update the particle position and velocity.Step (4)
(implement a mutation operation). The diversity of the particle swarm is increased using (2.4).Step (5)
(perform an elitist strategy). A new particle swarm (population) is generated from internal step(3). Notably, f(xPSO,j) of a candidate solution j (particle j) in the particle swarm is evaluated. Here, a pairwise comparison is made between the f(xPSO,j) value of candidate solutions in the new and current particle swarms. The elitist strategy guarantees that the best candidate solution is always preserved in the next generation. The current particle swarm is updated to the particle swarm of the next generation.Internal steps (2) to (5) are repeated until thegmax,PSO value of the internal PSO algorithm is satisfied.End
Consistent with theAb-Ag affinity metaphor, an Ab-Ag affinity is determined using (3.3), as follows:
(3.3)max(affinityj)=-1×f(xPSO*)j=1,2,…,rs.
Following the evaluation of the Ab-Ag affinities of Absin the current Ab repertoire, the Ab with the highest Ab-Ag affinity (Ab*) is chosen to undergo clonal selection operation in external Step 3.Step 3 (perform clonal selection operation).
To control the number of antigen-specificAbs, (2.11) is used.Step 4 (implementAb-Ag affinity maturation).
The intermediateAb repertoire that is created in external Step 3 is divided into two subsets. These Abs undergo somatic hypermutation operation by using (2.12) when the random number is 0.5 or less. Notably, these Abs suffer receptor editing operation using (2.13) when the random number exceeds 0.5.Step 5 (introduce diverseAbs).
Based on the bone marrow operation, diverseAbs are created to recruit the Abssuppressed in external Step 3.Step 6 (update anAb repertoire).
A newAbrepertoire is generated from external Steps 3–5. The Ab-Ag affinities of the Abs in the generated Ab repertoire are evaluated. This work presents a strategy for updating the Ab repertoire. A situation in which the Ab-Ag affinity of Ab  j in the new Ab repertoire exceeds that in the current Ab repertoire implies that a strong Ab in the new Ab repertoire replaces the weak Ab in the current Ab repertoire. Additionally, a situation in which the Ab-Ag affinity of Ab  j in the new Ab repertoire equals to or is worse than that in the current Ab repertoire implies that the Ab  j in the current Ab repertoire survives. In addition to maintaining the strong Abs, this strategy eliminates nonfunctional Abs.External Steps2–6 are repeated until the termination criterion gmax,AIA is satisfied.


## 4. Results

The 13 CGO problems were taken from other studies [1, 20, 21, 23, 34]. The set of CGO problems comprises six benchmark NLP problems (TPs 1–4 and 12–13), and seven GPP problems, in which TP 5 (alkylation process design in chemical engineering), TP 6 (optimal reactor design), TP 12 (a tension/compression string design problem), and TP 13 (a pressure vessel design problem) are constrained engineering problems, were used to evaluate the performances of the proposed RGA-PSO and AIA-PSO algorithms. In the appendix, the objective function, constraints, boundary conditions of decision variables, and known global optimum for TPs 1−11 are described and the problem characteristics of TPs 5, 12, and 13 are further detailed.The proposed RGA-PSO and AIA-PSO algorithms were coded in MATLAB software and executed on a Pentium D 3.0 (GHz) personal computer. Fifty independent runs were conducted to solve each test problem (TP). Numerical results were summarized, including the best, median, mean, and worst results, as well as the standard deviation (S.D.) of objective function values obtained using RGA-PSO and AIA-PSO solutions, mean computational CPU times (MCCTs), and mean absolute percentage error MAPE, as defined by(4.1)MAPE=∑s=150|(f(x*)-f(xstochastics))/f(x*)|50×100%,s=1,2,…,50,
where f(x*)= value of the known global solution, f(xstochastics) = values obtained from solutions of stochastic global optimization approaches (e.g., RGA-PSO and AIA-PSO algorithms).Table1 lists the parameter settings for the RGA-PSO and AIA-PSO algorithms, as shown in Table 1.Table 1
The parameter settings for the RGA-PSO and AIA-PSO algorithms.
Methods
Parameter settings
Search space
The external RGA
p
c
=
1
p
m
,
R
G
A
=
0.15
[χl,χu] = [0.1,1]
p
s
R
G
A
=
10
[c1l,c1u] = [0.1,2]
g
m
a
x
,
R
G
A
=
3
[c2l,c2u] = [0.1,2]
The external AIA
p
r
t
=
0.9
[ρl,ρu] = [1×109,1×1011]
r
s
=
10
[pm,PSOl,pm,PSOu]=[0.1,0.5]
g
m
a
x
,
A
I
A
=
3
The internal PSO algorithm
p
s
P
S
O
=
100
[xnl, xnu] for a CGO problem
g
m
a
x
,
P
S
O
=
3500 for TPs 1–4
g
m
a
x
,
P
S
O
=
3000 for TPs 5–13
[
χ
l
,
χ
u]: the lower and upper boundaries of parameter χ.
[c1l,c1u]: the lower and upper boundaries of parameter c1.
[c2l,c2u]: the lower and upper boundaries of parameter c2.
[ρl,ρu]: the lower and upper boundaries of parameter ρ.
[pm,PSOl,pm,PSOu]: the lower and upper boundaries of pm,PSO for the internal PSO algorithm pm,PSO.


### 4.1. Comparison of the Results Obtained Using the RGA-PSO and AIA-PSO Algorithms

Table2 summarizes the numerical results obtained using the proposed RGA-PSO and AIA-PSO algorithms for TPs 1–13. Numerical results indicate that the RGA-PSO and the AIA-PSO algorithms can obtain the global minimum solution to TPs 1–11, since each MAPE% is small. Moreover, the best, median, worst, and S.D. of objective function values obtained using the RGA-PSO and AIA-PSO solutions are identical for TPs 1, 2, 3, 4, 6, 7, 8, 9, and 11. Furthermore, the worst values obtained using the AIA-PSO algorithm for TPs 5 and 13 are smaller than those obtained using the RGA-PSO algorithm. Additionally, t-test is performed for each TP, indicating that the mean values obtained using the RGA-PSO and AIA-PSO algorithms are statistically significant for TPs 5, 10, 12, and 13, since P value is smaller than a significant level 0.05. Based on the results of t-test, the AIA-PSO algorithm yields better mean values than the RGA-PSO algorithm for TPs 5, 12, and 13, and the AIA-PSO algorithm yields worse mean value than the RGA-PSO algorithm for TP 10.Table 2
Numerical results of the proposed RGA-PSO and AIA-PSO algorithms for TPs 1–13.
TP number
Global optimum
Methods
Best
Mean
Median
Worst
MAPE %
S.D.
MCCT (sec)
P value
1
24.306
The proposedRGA-PSO
24.323
24.568
24.521
24.831
1.078
0.13
432.39
0.643
The proposedAIA-PSO
24.358
24.579
24.564
24.809
1.125
0.11
417.63
2
−30665.539
The proposedRGA-PSO
−30665.539
−30665.539
−30665.539
−30665.534
1.34
E
-
06
7.36
E
-
04
349.55
0.478
The proposedAIA-PSO
−30665.539
−30665.539
−30665.539
−30665.539
9.95
E
-
07
3.81
E
-
05
349.76
3
680.630
The proposedRGA-PSO
680.632
680.640
680.639
680.658
1.46
E
-
03
5.55
E
-
03
398.72
0.839
The proposedAIA-PSO
680.633
680.640
680.640
680.657
1.50
E
-
03
0.01
398.84
4
−15
The proposedRGA-PSO
−15
−15
−15
−15
1.27
E
-
06
1.27
E
-
06
466.00
0.294
The proposedAIA-PSO
−15
−15
−15
−15
1.20
E
-
10
2.64
E
-
11
450.28
5
1227.1978
The proposedRGA-PSO
1227.2321
1228.8471
1228.3935
1231.6003
1.34
E
-
01
1.27
481.86
0.002*
The proposedAIA-PSO
1227.1598
1228.1800
1227.9762
1229.6590
8.02
E
-
02
0.71
461.30
6
3.9511
The proposedRGA-PSO
3.9521
3.9577
3.9566
3.9722
1.67
E
-
01
5
E
-
03
439.16
0.682
The proposedAIA-PSO
3.9516
3.9581
3.9569
3.9735
1.77
E
-
01
4.79
E
-
03
438.55
7
−5.7398
The proposedRGA-PSO
−5.7398
−5.7398
−5.7398
−5.7398
2.54
E
-
04
1.13
E
-
05
325.73
0.358
The proposedAIA-PSO
−5.7398
−5.7398
−5.7398
−5.7398
2.42
E
-
04
7.74
E
-
06
329.12
8
−83.254
The proposedRGA-PSO
−83.2497
−83.2497
−83.2497
−83.2497
5.13
E
-
03
1.26
E
-
06
211.91
0.291
The proposedAIA-PSO
−83.2497
−83.2497
−83.2497
−83.2497
5.13
E
-
03
6.85
E
-
07
213.01
9
−6.0482
The proposedRGA-PSO
−6.0441
−5.9883
−5.9968
−5.8051
0.991
0.05
440.72
0.577
The proposedAIA-PSO
−6.0467
−5.9828
−5.9979
−5.8274
1.082
0.05
441.42
10
6300
The proposedRGA-PSO
6299.8374
6299.8412
6299.8419
6299.8428
2.52
E
-
03
1.42
E
-
03
224.13
0.001*
The proposedAIA-PSO
6299.8395
6299.8420
6299.8423
6299.8425
2.51
E
-
03
6.83
E
-
04
223.91
11
10122.6964
The proposedRGA-PSO
10122.4732
10122.4925
10122.4925
10122.6444
2.01
E
-
03
2.26
E
-
02
340.89
0.884
The proposedAIA-PSO
10122.4852
10122.4920
10122.4927
10122.4931
2.02
E
-
03
1.85
E
-
03
338.37
12
—
The proposedRGA-PSO
0.012692
0.012724
0.012721
0.012784
—
1.46
E
-
05
293.67
0.014*
The proposedAIA-PSO
0.012667
0.012715
0.012719
0.012778
—
2.00
E
-
05
296.15
13
—
The proposedRGA-PSO
5885.3018
5895.0381
5885.3326
6005.4351
—
24.33
291.17
0.019*
The proposedAIA-PSO
5885.3249
5886.5426
5885.3323
5906.7404
—
4.54
292.16
(The “—” denotes unavailable information, and *represents that the mean values obtained using the RGA-PSO and AIA-PSO algorithms are statistically different.)Tables3 and 4 list the best solutions obtained using the RGA-PSO and AIA-PSO algorithms from TPs 1–13, respectively, indicating that each constraint is satisfied (i.e., the violation of each constraint is accurate to at least five decimal places) for every TP. Tables 5 and 6 list the best parameter settings of the internal PSO algorithm obtained using the external RGA and AIA approaches, respectively.Table 3
The best solutions obtained using the RGA-PSO algorithm from TPs 1–13.
TP number
f
(
x
R
G
A-
P
S
O
*
)
x
R
G
A-
P
S
O
*
1
24.323
x
RGA-PSO
*
=
(
2.16727760, 2.37789634, 8.78162804, 5.12372885, 0.97991270, 1.39940993, 1.31127142, 9.81945011, 8.30004549, 8.45891329)
g
m
(
x
RGA-PSO
*
)
=
(
-
0.000171
≤
0
,
-
0.003109
≤
0
,
-
0.000027
≤
0
,
-
0.000123
≤
0
,
-
0.001371
≤
0
,
-
0.002101
≤
0
,
-
6.245957
≤
0
,
-
47.846302
≤
0
)
2
−30665.539
x
RGA-PSO
* = (78, 33, 29.99525450, 45, 36.77581373)
g
m
(
x
RGA-PSO
*
)
=
(
-
92.000000
≤
0
,
2.24
E
-
07
≤
0
,
-
8.840500
≤
0
,
-
11.159500
≤
0
,
4.28
E
-
07
≤
0
,
-
5.000000
≤
0
)
3
680.632
x
RGA-PSO
*
=
(2.33860239, 1.95126191, −0.45483579, 4.36300325, −0.62317747, 1.02938443, 1.59588410)
g
m
(
x
RGA-PSO
*
)
=
(
-
7.76
E
-
07
≤
0
,
-
252.7211
≤
0
,
-
144.8140
≤
0
,
-
6.15
E
-
05
≤
0
)
4
−15
x
RGA-PSO
*
=
(1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 1)
g
m
(
x
RGA-PSO
*
)
=
(
0
≤
0
,
0
≤
0
,
0
≤
0
,
-
5
≤
0
,
-
5
≤
0
,
-
5
≤
0
,
0
≤
0
,
0
≤
0
,
0
≤
0
)
5
1227.1139
x
RGA-PSO
*
=
(1697.13793410, 54.17332240, 3030.44600072, 90.18199040, 94.99999913, 10.42097385, 153.53771370)
g
m
(
x
RGA-PSO
*
)
=
(
0.999978
≤
1
,
0.980114
≤
1
,
1.000005
≤
1
,
0.980097
≤
1
,
0.990565
≤
1
,
1.000005
≤
1
,
1.00000
≤
1
,
0.976701
≤
1
,
1.000003
≤
1
,
0.459311
≤
1
,
0.387432
≤
1
,
0.981997
≤
1
,
0.980364
≤
1
,
-
8.241926
≤
1
)
6
3.9521
x
RGA-PSO
*
=
(6.444100620, 2.243029250, 0.642672939, 0.582321363, 5.940008650, 5.531235784, 1.018087316, 0.403665649)
g
m
(
x
RGA-PSO
*
)
=
(
1.000000
≤
1
,
1.000000
≤
1
,
0.999996
≤
1
,
0.99986
≤
1
)
7
−5.7398
x
RGA-PSO
*
=
(8.12997229, 0.61463971, 0.56407162, 5.63623069)
g
m
(
x
RGA-PSO
*
)
=
(
1.000000
≤
1
,
1.000000
≤
1
)
8
−83.2497
x
RGA-PSO
*
=
(88.35595404, 7.67259607, 1.31787691)
g
m
(
x
RGA-PSO
*
)
=
(
1.000000
≤
1
)
9
−6.0441
x
RGA-PSO
*
=
(6.40497368, 0.64284563, 1.02766984, 5.94729224, 2.21044814, 0.59816471, 0.42450835, 5.54339987)
g
m
(
x
RGA-PSO
*
)
=
(
0.999997
≤
1
,
0.999959
≤
1
,
0.999930
≤
1
,
0.999975
≤
1
)
10
6299.8374
x
RGA-PSO
*
=
(108.66882633, 85.10837983, 204.35894362)
g
m
(
x
RGA-PSO
*
)
=
(
1.000002
≤
1
)
11
10122.4732
x
RGA-PSO
*
=
(78, 33, 29.99564864, 45, 36.77547104)
g
m
(
x
RGA-PSO
*
)
=
(
-
0.309991
≤
1
,
1.000004
≤
1
,
-
0.021379
≤
1
,
0.621403
≤
1
,
1.000002
≤
1
,
0.681516
≤
1
)
12
0.012692
x
RGA-PSO
*
=
(0.050849843, 0.336663305, 12.579603478)
g
m
(
x
RGA-PSO
*
)
=
(
-
0
.
000143
≤
0
,
-
0.000469
≤
0
,
-
4.009021
≤
0
,
-
0.741658
≤
0
)
13
5885.3018
x
RGA-PSO
*
=
(0.77816852, 0.38464913, 40.31961883, 199.99999988)
g
m
(
x
RGA-PSO
*
)
=
(
1.23
E
-
07
≤
0
,
3.36
E
-
08
≤
0
,
-
0.006916
≤
0
,
-
40.00000012
≤
0
)Table 4
The best solutions obtained using the AIA-PSO algorithm from TPs 1–13.
TP number
f
(
x
A
I
A-
P
S
O
*
)
x
A
I
A-
P
S
O
*
1
24.358
x
AIA-PSO
*
=
(2.18024296, 2.35746157, 8.75670935, 5.11326109, 1.03976363 1.54784227, 1.32994030, 9.83127443, 8.27618717, 8.32717779)
g
m
(
x
AIA-PSO
*
)
=
(
-
0.000071
≤
0
,
-
0.003699
≤
0
,
-
0.000440
≤
0
,
-
0.684025
≤
0
,
-
0.000086
≤
0
,
-
0.001024
≤
0
,
-
5.973866
≤
0
,
-
47.371958
≤
0
)
2
−30665.539
x
AIA-PSO
*
=
(78, 33, 29.99525527, 45, 36.77581286)
g
m
(
x
AIA-PSO
*
)
=
(
-
92.000000
≤
0
,
5.57
E
-
08
≤
0
,
-
8.840500
≤
0
,
-
11.159500
≤
0
,
2.76
E
-
07
≤
0
,
-
5.000000
≤
0
)
3
680.633
x
AIA-PSO
*
=
(2.32925164, 1.95481519, −0.47307614, 4.35691576, −0.62313420, 1.05236194, 1.59750978)
g
m
(
x
AIA-PSO
*
)
=
(
-
4.26
E
-
06
≤
0
,
-
252.612733
≤
0
,
-
144.741194
≤
0
,
-
1.00
E
-
05
≤
0
)
4
−15
x
AIA-PSO
*
=
(1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 1)
g
m
(
x
AIA-PSO
*
)
=
(
0
≤
0
,
0
≤
0
,
0
≤
0
,
-
5
≤
0
,
-
5
≤
0
,
-
5
≤
0
,
0
≤
0
,
0
≤
0
,
0
≤
0
)
5
1227.1598
x
AIA-PSO
*
=
(
1697.91645044
,
53.78132825
,
3031.07989059
,
90.12651301
,
94.99999995
,
10.48103708
,
153.53594861
)
g
m
(
x
AIA-PSO
*
)
=
(
1.000000
≤
1
,
0.980100
≤
1
,
1.000002
≤
1
,
0.980099
≤
1
,
0.990562
≤
1
,
1.000001
≤
1
,
1.000002
≤
1
,
0.976715
≤
1
,
1.000001
≤
1
,
0.459425
≤
1
,
0.387515
≤
1
,
0.981856
≤
1
,
0.987245
≤
1
,
-
8.302533
≤
1
)
6
3.9516
x
AIA-PSO
*
=
(
6.44373647
,
2.23335111
,
0.68233303
,
0.60026617
,
5.93745119
,
5.53146186
,
1.01862958
,
0.40673661
)
g
m
(
x
AIA-PSO
*
)
=
(
0.999999
≤
1
,
0.999999
≤
1
,
0.999994
≤
1
,
0.999985
≤
1
)
7
−5.7398
x
AIA-PSO
*
=
(
8.13042985
,
0.61758136
,
0.56393603
,
5.63618191
)
g
m
(
x
AIA-PSO
*
)
=
(
0.999999
≤
1
,
0.999998
≤
1
)
8
−83.2497
x
AIA-PSO
*
=
(88.35635930, 7.67202113, 1.31765768)
g
m
(
x
AIA-PSO
*
)
=
(
3.00
E
-
18
≤
1
)
9
−6.0467
x
AIA-PSO
*
=
(
6.46393173, 0.67575021, 1.01188804, 5.94072071, 2.24639462, 0.60683404, 0.39677469, 5.52596342)
g
m
(
x
AIA-PSO
*
)
=
(
0.999980
≤
1
,
0.999999
≤
1
,
0.998739
≤
1
,
0.999928
≤
1
)
10
6299.8395
x
AIA-PSO
*
=
(108.97708780, 85.02248757, 204.45439729)
g
m
(
x
AIA-PSO
*
)
=
(
1.000003
≤
1
)
11
10122.4852
x
AIA-PSO
*
=
(78, 33, 29.99571251, 45, 36.77534593)
g
m
(
x
AIA-PSO
*
)
=
(
-
0.309991
≤
1
,
1.000001
≤
1
,
-
0.021380
≤
1
,
0.621403
≤
1
,
1.000001
≤
1
,
0.681517
≤
1
)
12
0.012667
x
AIA-PSO
*
=
(0.05164232,   0.35558085,  11.35742676)
g
m
(
x
AIA-PSO
*
)
=
(
-
8.83
E
-
05
≤
0
,
-
3.04
E
-
05
≤
0
,
-
4.050924
≤
0
,
-
0.728518
≤
0
)
13
5885.3310
x
AIA-PSO
*
=
(0.77816843, 0.38464909, 40.31961929, 199.99999330)
g
m
(
x
AIA-PSO
*
)
=
(
2.22
E
-
07
≤
0
,
7.80
E
-
08
≤
0
,
-
0.006015
≤
0
,
-
40.000007
≤
0
)Table 5
The best parameter settings of the best solution obtained using the RGA-PSO algorithm from TPs 1–13.
TP number
χ
c
1
c
2
ρ
p
m
,
P
S
O
1
0.73759998
1.52939998
1.72753568
28709280994
0.3750
2
0.67057537
0.45010388
2
1000000000
0.2709
3
0.75493696
0.36925226
1.91198475
54900420223
0.3856
4
0.45438391
1.45998477
1.25508289
1000000000
0.1
5
1
0.77483233
2
1000000000
0.5
6
0.70600559
0.82360083
0.91946627
26622414511
0.1619
7
0.74584341
0.95474855
1.17537957
86557786169
0.1958
8
1
0.69725160
1.53028620
1000000000
0.1675
9
0.41638718
0.46594542
1.97807798
12976330236
0.3686
10
0.25378610
0.59619170
0.83891277
1000000000
0.1
11
0.48183123
2
2
1000000000
0.1
12
0.76190087
0.1
1.16855713
100000000000
0.5
13
0.71783704
1.39420750
1.33590124
29071187026
0.2098Table 6
The best parameter settings of the best solution obtained using the AIA-PSO algorithm from TPs 1–13.
TP number
χ
c
1
c
2
ρ
p
m
,
P
S
O
1
0.44896780
0.71709211
1.93302154
72140307110
0.4058
2
1
1.36225259
1.97905466
182831007
0.5
3
0.46599492
0.90346435
1.89697456
69125199709
0.2613
4
0.98124982
0.27882671
0.87437226
85199047430
0.1
5
0.99082484
0.1
1.11371788
4231387044
0.5
6
0.82869043
0.88773247
2
1387448505
0.5
7
0.87571243
1.89936723
0.74306310
94752095153
0.2194
8
0.93583844
1.53906226
1.30374874
22520728225
0.1798
9
1
0.22556712
1.52263349
67578847151
0.4507
10
1
1.93003999
0.1
1351461763
0.5
11
1
1.51209364
1.63826995
1811017789
0.5
12
0.52068067
0.1
2
81914376144
0.1
13
0.82395890
1.60107152
0.93611204
17767111886
0.1813


### 4.2. Comparison of the Results for the Proposed RGA-PSO and AIA-PSO Algorithms with Those Obtained Using the Published Individual GA and AIA Approaches and Hybrid Algorithms

Table7 compares the numerical results of the proposed RGA-PSO and AIA-PSO algorithms with those obtained using published individual GA and AIA approaches for TPs 1–4. In this table, GA-1 is a GA with a penalty function methods, as used by Michalewicz [20]. Notably, GA-2 represents a GA with a penalty function, but without any penalty parameter, as used by Deb [21]. Also, GA-3 is an RGA with a static penalty function, as developed by Wu and Chung [9]. Notably, AIA-1 is an AIA method called CLONALG, as proposed by Cruz-Cortés et al. [22]. Finally, AIA-2 is an AIA approach based on an adaptive penalty function, as developed by Wu [10]. The numerical results of GA-1, GA-2, and AIA-1 methods for solving TPs 1–4 were collected from the published literature [20–22]. Furthermore, the GA-1, GA-2, and AIA-1 approaches were executed under 350,000 objective function evaluations. To fairly compare the performances of the proposed hybrid CI algorithms and the individual GA and AIA approaches, the GA-3, AIA-2, the internal PSO algorithm of RGA-PSO method, and the internal PSO algorithm of AIA-PSO method were independently executed 50 times under 350,000 objective function evaluations for solving TPs 1–4.Table 7
Comparison of the results of the proposed RGA-PSO and AIA-PSO algorithms and those of the published individual GA and AIA approaches for TPs 1–4.
TP number
Global optimum
Methods
Best
Mean
Median
Worst
MAPE%
GA-1 [20]
24.690
—
29.258
36.060
—
GA-2 [21]
24.372
—
24.409
25.075
—
GA-3 [9]
24.935
27.314
27.194
33.160
12.377
1
24.306
AIA-1 [22]
24.506
25.417
—
26.422
—
AIA-2 [10]
24.377
24.669
24.663
24.988
1.495
The proposed RGA-PSO
24.323
24.568
24.521
24.831
1.078
The proposed AIA-PSO
24.358
24.579
24.564
24.809
1.125
GA-1 [20]
—
—
—
—
—
GA-2 [21]
—
—
—
—
—
GA-3 [9]
−30665.526
−30662.922
−30664.709
−30632.445
8.53
E
-
03
2
−30665.539
AIA-1 [22]
−30665.539
−30665.539
—
−30665.539
—
AIA-2 [10]
−30665.539
−30665.526
−30665.527
−30665.506
4.20
E
-
05
The proposed RGA-PSO
−30665.539
−30665.539
−30665.539
−30665.534
1.34
E
-
06
The proposed AIA-PSO
−30665.539
−30665.539
−30665.539
−30665.539
9.95
E
-
07
GA-1 [20]
680.642
—
680.718
680.955
—
GA-2 [21]
680.634
—
680.642
680.651
—
GA-3 [9]
680.641
680.815
680.768
681.395
2.72
E
-
02
3
680.630
AIA-1 [22]
680.631
680.652
—
680.697
—
AIA-2 [10]
680.634
680.653
680.650
680.681
3.45
E
-
03
The proposed RGA-PSO
680.632
680.640
680.639
680.658
1.46
E
-
03
The proposed AIA-PSO
680.633
680.640
680.640
680.657
1.50
E
-
03
GA-1 [20]
−15
−15
−15
−15
—
GA-2 [21]
—
—
—
—
—
GA-3 [9]
−13.885
−12.331
−12.267
−10.467
17.795
4
−15
AIA-1 [22]
−14.987
−14.726
—
−12.917
—
AIA-2 [10]
−14.998
−14.992
−14.992
−14.988
5.08
E
-
02
The proposed RGA-PSO
−15
−15
−15
−15
1.27
E
-
06
The proposed AIA-PSO
−15
−15
−15
−15
1.20
E
-
10
(The “—” denotes unavailable information.)For solving TP 1, the median values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the GA-1, GA-3, and AIA-2 approaches, and the worst values obtained using RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the GA-1, GA-2, GA-3, AIA-1, and AIA-2 approaches. For solving TP 2, the median and worst values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the GA-3 method. For solving TP 3, the median and worst values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the GA-1 and GA-3 approaches. For solving TP 4, the median and worst values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the GA-3 method, and the worst values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the AIA-1 approach. Moreover, the GA-3 method obtained the worst MAPE% for TP 1 and TP 4. Table8 lists the results of the t-test for the GA-3, AIA-2, RGA-PSO, and AIA-PSO methods. This table indicates that the mean values of the RGA-PSO, and AIA-PSO algorithms are not statistically significant, since P values are larger than a significant level 0.05, and the mean values between GA-3 versus AIA-2, GA-3 versus RGA-PSO, GA-3 versus AIA-PSO, AIA-2 versus RGA-PSO, and AIA-2 versus AIA-PSO are statistically significant. According to Tables 7 and 8, the mean values obtained using the RGA-PSO and AIA-PSO algorithms are better than those of obtained using the GA-3 and AIA-1 methods for TPs 1–4.Table 8
Results of thet-test for TPs 1–4.
TP number
GA-3 versus AIA-2
GA-3 versus RGA-PSO
GA-3 versus AIA-PSO
AIA-2 versus RGA-PSO
AIA-2 versus AIA-PSO
RGA-PSO versus AIA-PSO
1
0.000*
0.000*
0.000*
0.000*
0.001*
0.643
2
0.003*
0.003*
0.003*
0.000*
0.000*
0.478
3
0.000*
0.000*
0.000*
0.000*
0.000*
0.839
4
0.000*
0.000*
0.000*
0.000*
0.000*
0.294
*Represents that the mean values obtained using two algorithms are statistically different.)Table9 compares the numerical results obtained using the proposed RGA-PSO and AIA-PSO algorithms and those obtained using AIA-2 and GA-3 for solving TPs 5–13. The AIA-2, GA-3, the internal PSO algorithm of the RGA-PSO approach, and the internal PSO algorithm of AIA-PSO approach were independently executed 50 times under 300,000 objective function evaluations. Table 9 shows that MAPE% obtained using the proposed RGA-PSO and AIA-PSO algorithms is close to 1%, or smaller than 1% for TPs 5–11, indicating that the proposed RGA-PSO and AIA-PSO algorithms can converge to global optimum for TPs 5–11. Moreover, the worst values obtained using the RGA-PSO and AIA-PSO algorithms are significantly smaller than those obtained using the GA-3 method for TPs 5, 6, 11, and 13. Additionally, the worst values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the AIA-2 method for TPs 5, 6, and 13.Table 9
Comparison of the numerical results of the proposed RGA-PSO and AIA-PSO algorithms and those of the published individual AIA and RGA for TPs 5–13.
TP number
Global optimum
Methods
Best
Mean
Median
Worst
MAPE%
5
1227.1978
AIA-2 [10]
1227.3191
1228.6097
1227.8216
1239.5205
1.15
E
-
01
GA-3 [9]
1228.5118
1279.3825
1263.2589
1481.3710
4.266
The proposed RGA-PSO
1227.2321
1228.8471
1228.3935
1231.6003
1.34
E
-
01
The proposed AIA-PSO
1227.1598
1228.1800
1227.9762
1229.6590
8.02
E
-
02
6
3.9511
AIA-2 [10]
3.9518
4.1005
4.0460
4.2997
3.781
GA-3 [9]
3.9697
4.1440
4.1358
4.3743
4.881
The proposed RGA-PSO
3.9521
3.9577
3.9566
3.9722
1.67
E
-
01
The proposed AIA-PSO
3.9516
3.9581
3.9569
3.9735
1.77
E
-
01
7
−5.7398
AIA-2 [10]
−5.7398
−5.7398
−5.7398
−5.7396
6.03
E
-
04
GA-3 [9]
−5.7398
−5.7375
−5.7383
−5.7316
3.98
E
-
02
The proposed RGA-PSO
−5.7398
−5.7398
−5.7398
−5.7398
2.54
E
-
04
The proposed AIA-PSO
−5.7398
−5.7398
−5.7398
−5.7398
2.42
E
-
04
8
−83.254
AIA-2 [10]
−83.2504
−83.2499
−83.2499
−83.2496
4.93
E
-
03
GA-3 [9]
−83.2497
−83.2490
−83.2494
−83.2461
6.06
E
-
03
The proposed RGA-PSO
−83.2497
−83.2497
−83.2497
−83.2497
5.13
E
-
03
The proposed AIA-PSO
−83.2497
−83.2497
−83.2497
−83.2497
5.13
E
-
03
9
−6.0482
AIA-2 [10]
−6.0471
−5.9795
−6.0138
−5.7159
1.136
GA-3 [9]
−6.0417
−5.8602
−5.8772
−5.6815
3.108
The proposed RGA-PSO
−6.0441
−5.9883
−5.9968
−5.8051
0.991
The proposed AIA-PSO
−6.0466
−5.9808
−5.9969
−5.8274
1.082
10
6300
AIA-2 [10]
6299.8232
6299.8388
6299.8403
6299.8494
2.56
E
-
03
GA-3 [9]
6299.8399
6299.8546
6299.8453
6299.9407
2.31
E
-
03
The proposed RGA-PSO
6299.8374
6299.8412
6299.8419
6299.8428
2.52
E
-
03
The proposed AIA-PSO
6299.8395
6299.8420
6299.8423
6299.8425
2.51
E
-
03
11
10122.6964
AIA-2 [10]
10122.4343
10122.5014
10122.5023
10122.6271
1.93
E
-
03
GA-3 [9]
10122.4809
10125.2441
10123.9791
10155.1349
2.58
E
-
04
The proposed RGA-PSO
10122.4732
10122.4925
10122.4925
10122.6444
2.01
E
-
03
The proposed AIA-PSO
10122.4852
10122.4920
10122.4927
10122.4931
2.02
E
-
03
12
—
AIA-2 [10]
0.012665
0.012703
0.012719
0.012725
—
GA-3 [9]
0.012665
0.012739
0.012717
0.013471
—
The proposed RGA-PSO
0.012692
0.012724
0.012721
0.012784
—
The proposed AIA-PSO
0.012667
0.012715
0.012719
0.012778
—
13
—
AIA-2 [10]
5885.5312
5900.4860
5894.7929
6014.2198
—
GA-3 [9]
5885.3076
5943.4714
5897.2904
6289.7314
—
The proposed RGA-PSO
5885.3018
5895.0381
5885.3326
6005.4351
—
The proposed AIA-PSO
5885.3310
5886.5426
5885.3323
5906.7404
—
(The “—” denotes unavailable information.)Table10 summarizes the results of the t-test for TPs 5–13. According to Tables 9 and 10, the mean values of the RGA-PSO and AIA-PSO algorithms are smaller than those of the GA-3 approach for TPs 5, 6, 7, 8, 9, 10, 11, and 13. Moreover, the mean values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those of the AIA-2 approach for TPs 6, 7, 8, 10, and 12. Totally, according to Tables 7−10, the performances of the hybrid CI methods are superior to those of individual GA and AIA methods.Table 10
Results oft-test for TPs 5–13.
TP number
GA-3 versus AIA-2
GA-3 versus RGA-PSO
GA-3 versus AIA-PSO
AIA-2 versus RGA-PSO
AIA-2 versus AIA-PSO
RGA-PSO versus AIA-PSO
5
0.000*
0.000*
0.000*
0.485
0.163
0.002*
6
0.112
0.000*
0.000*
0.000*
0.000*
0.682
7
0.000*
0.000*
0.000*
0.000*
0.000*
0.358
8
0.000*
0.000*
0.000*
0.000*
0.000*
0.291
9
0.000*
0.000*
0.000*
0.516
0.814
0.577
10
0.000*
0.000*
0.000*
0.009*
0.001*
0.001*
11
0.000*
0.000*
0.000*
0.069
0.013*
0.884
12
0.049*
0.389
0.178
0.000*
0.007*
0.014*
13
0.003*
0.001*
0.000*
0.240
0.000*
0.019*
*Represents that the mean values obtained using two algorithms are statistically different.)The TPs 12 and 13 have been solved by many hybrid algorithms. For instance, Huang et al. [23] presented a coevolutionary differential evolution (CDE) that integrates a coevolution mechanism and a DE approach. Zahara and Kao [24] developed a hybrid Nelder-Mead simplex search method and a PSO algorithm (NM-PSO). Table 11 compares the numerical results of the CDE, NM-PSO, RGA-PSO, and AIA-PSO methods for solving TPs 12−13. The table indicates that the best, mean, and worst values obtained using the NM-PSO method are superior to those obtained using the CDE, RGA-PSO, and AIA-PSO approaches for TP 12. Moreover, the best, mean, and worst values obtained using the AIA-PSO algorithm are better than those of the CDE, NM-PSO, and RGA-PSO algorithms.Table 11
Comparison of the numerical results of the proposed RGA-PSO and AIA-PSO algorithms and those of the published hybrid algorithms for TPs 12-13.
TP number
Methods
Best
Mean
Median
Worst
S.D.
12
CDE [23]
0.0126702
0.012703
—
0.012790
2.7
E
-
05
NM-PSO [24]
0.0126302
0.0126314
—
0.012633
8.73
E
-
07
The proposed RGA-PSO
0.012692
0.012724
0.012721
0.012784
1.46
E
-
05
The proposed AIA-PSO
0.012667
0.012715
0.012719
0.012778
2.00
E
-
05
13
CDE [23]
6059.7340
6085.2303
—
6371.0455
43.01
NM-PSO [24]
5930.3137
5946.7901
—
5960.0557
9.16
The proposed RGA-PSO
5885.3018
5895.0381
5885.3326
6005.4351
24.33
The proposed AIA-PSO
5885.3310
5886.5426
5885.3323
5906.7404
4.54According to the No Free Lunch theorem [35], if algorithm A outperforms algorithm B on average for one class of problems, then the average performance of the former must be worse than that of the latter over the remaining problems. Therefore, it is unlikely that any unique stochastic global optimization approach exists that performs best for all CGO problems.


### 4.3. Summary of Results

The proposed RGA-PSO and AIA-PSO algorithms with a penalty function method have the following benefits.(1)
Parameter manipulation of the internal PSO algorithm is based on the solved CGO problems. Owing to their ability to efficiently solve an UGO problem, the external RGA and AIA approaches are substituted for trial and error to manipulate the parameters (χ, c1, c2, ρ, and pm,PSO).(2)
Besides obtaining the optimum parameter settings of the internal PSO algorithm, the RGA-PSO and AIA-PSO algorithms can yield a global optimum for a CGO problem.(3)
In addition to performing better than approaches of some published individual GA and AIA approaches, the proposed RGA-PSO and AIA-PSO algorithms reduce the parametrization for the internal PSO algorithm, despite the RGA-PSO and AIA-PSO algorithms being more complex than individual GA and AIA approaches.The proposed RGA-PSO and AIA-PSO algorithms have the following limitations.(1)
The proposed RGA-PSO and AIA-PSO algorithms increase the computational CPU time, as shown in Table2.(2)
The proposed RGA-PSO and AIA-PSO algorithms are designed to solve CGO problems with continuous decision variablesxn. Therefore, the proposed algorithms cannot be applied to manufacturing problems such as job shop scheduling and quadratic assignment problems (combinatorial optimization problems).


## 4.1. Comparison of the Results Obtained Using the RGA-PSO and AIA-PSO Algorithms

Table2 summarizes the numerical results obtained using the proposed RGA-PSO and AIA-PSO algorithms for TPs 1–13. Numerical results indicate that the RGA-PSO and the AIA-PSO algorithms can obtain the global minimum solution to TPs 1–11, since each MAPE% is small. Moreover, the best, median, worst, and S.D. of objective function values obtained using the RGA-PSO and AIA-PSO solutions are identical for TPs 1, 2, 3, 4, 6, 7, 8, 9, and 11. Furthermore, the worst values obtained using the AIA-PSO algorithm for TPs 5 and 13 are smaller than those obtained using the RGA-PSO algorithm. Additionally, t-test is performed for each TP, indicating that the mean values obtained using the RGA-PSO and AIA-PSO algorithms are statistically significant for TPs 5, 10, 12, and 13, since P value is smaller than a significant level 0.05. Based on the results of t-test, the AIA-PSO algorithm yields better mean values than the RGA-PSO algorithm for TPs 5, 12, and 13, and the AIA-PSO algorithm yields worse mean value than the RGA-PSO algorithm for TP 10.Table 2
Numerical results of the proposed RGA-PSO and AIA-PSO algorithms for TPs 1–13.
TP number
Global optimum
Methods
Best
Mean
Median
Worst
MAPE %
S.D.
MCCT (sec)
P value
1
24.306
The proposedRGA-PSO
24.323
24.568
24.521
24.831
1.078
0.13
432.39
0.643
The proposedAIA-PSO
24.358
24.579
24.564
24.809
1.125
0.11
417.63
2
−30665.539
The proposedRGA-PSO
−30665.539
−30665.539
−30665.539
−30665.534
1.34
E
-
06
7.36
E
-
04
349.55
0.478
The proposedAIA-PSO
−30665.539
−30665.539
−30665.539
−30665.539
9.95
E
-
07
3.81
E
-
05
349.76
3
680.630
The proposedRGA-PSO
680.632
680.640
680.639
680.658
1.46
E
-
03
5.55
E
-
03
398.72
0.839
The proposedAIA-PSO
680.633
680.640
680.640
680.657
1.50
E
-
03
0.01
398.84
4
−15
The proposedRGA-PSO
−15
−15
−15
−15
1.27
E
-
06
1.27
E
-
06
466.00
0.294
The proposedAIA-PSO
−15
−15
−15
−15
1.20
E
-
10
2.64
E
-
11
450.28
5
1227.1978
The proposedRGA-PSO
1227.2321
1228.8471
1228.3935
1231.6003
1.34
E
-
01
1.27
481.86
0.002*
The proposedAIA-PSO
1227.1598
1228.1800
1227.9762
1229.6590
8.02
E
-
02
0.71
461.30
6
3.9511
The proposedRGA-PSO
3.9521
3.9577
3.9566
3.9722
1.67
E
-
01
5
E
-
03
439.16
0.682
The proposedAIA-PSO
3.9516
3.9581
3.9569
3.9735
1.77
E
-
01
4.79
E
-
03
438.55
7
−5.7398
The proposedRGA-PSO
−5.7398
−5.7398
−5.7398
−5.7398
2.54
E
-
04
1.13
E
-
05
325.73
0.358
The proposedAIA-PSO
−5.7398
−5.7398
−5.7398
−5.7398
2.42
E
-
04
7.74
E
-
06
329.12
8
−83.254
The proposedRGA-PSO
−83.2497
−83.2497
−83.2497
−83.2497
5.13
E
-
03
1.26
E
-
06
211.91
0.291
The proposedAIA-PSO
−83.2497
−83.2497
−83.2497
−83.2497
5.13
E
-
03
6.85
E
-
07
213.01
9
−6.0482
The proposedRGA-PSO
−6.0441
−5.9883
−5.9968
−5.8051
0.991
0.05
440.72
0.577
The proposedAIA-PSO
−6.0467
−5.9828
−5.9979
−5.8274
1.082
0.05
441.42
10
6300
The proposedRGA-PSO
6299.8374
6299.8412
6299.8419
6299.8428
2.52
E
-
03
1.42
E
-
03
224.13
0.001*
The proposedAIA-PSO
6299.8395
6299.8420
6299.8423
6299.8425
2.51
E
-
03
6.83
E
-
04
223.91
11
10122.6964
The proposedRGA-PSO
10122.4732
10122.4925
10122.4925
10122.6444
2.01
E
-
03
2.26
E
-
02
340.89
0.884
The proposedAIA-PSO
10122.4852
10122.4920
10122.4927
10122.4931
2.02
E
-
03
1.85
E
-
03
338.37
12
—
The proposedRGA-PSO
0.012692
0.012724
0.012721
0.012784
—
1.46
E
-
05
293.67
0.014*
The proposedAIA-PSO
0.012667
0.012715
0.012719
0.012778
—
2.00
E
-
05
296.15
13
—
The proposedRGA-PSO
5885.3018
5895.0381
5885.3326
6005.4351
—
24.33
291.17
0.019*
The proposedAIA-PSO
5885.3249
5886.5426
5885.3323
5906.7404
—
4.54
292.16
(The “—” denotes unavailable information, and *represents that the mean values obtained using the RGA-PSO and AIA-PSO algorithms are statistically different.)Tables3 and 4 list the best solutions obtained using the RGA-PSO and AIA-PSO algorithms from TPs 1–13, respectively, indicating that each constraint is satisfied (i.e., the violation of each constraint is accurate to at least five decimal places) for every TP. Tables 5 and 6 list the best parameter settings of the internal PSO algorithm obtained using the external RGA and AIA approaches, respectively.Table 3
The best solutions obtained using the RGA-PSO algorithm from TPs 1–13.
TP number
f
(
x
R
G
A-
P
S
O
*
)
x
R
G
A-
P
S
O
*
1
24.323
x
RGA-PSO
*
=
(
2.16727760, 2.37789634, 8.78162804, 5.12372885, 0.97991270, 1.39940993, 1.31127142, 9.81945011, 8.30004549, 8.45891329)
g
m
(
x
RGA-PSO
*
)
=
(
-
0.000171
≤
0
,
-
0.003109
≤
0
,
-
0.000027
≤
0
,
-
0.000123
≤
0
,
-
0.001371
≤
0
,
-
0.002101
≤
0
,
-
6.245957
≤
0
,
-
47.846302
≤
0
)
2
−30665.539
x
RGA-PSO
* = (78, 33, 29.99525450, 45, 36.77581373)
g
m
(
x
RGA-PSO
*
)
=
(
-
92.000000
≤
0
,
2.24
E
-
07
≤
0
,
-
8.840500
≤
0
,
-
11.159500
≤
0
,
4.28
E
-
07
≤
0
,
-
5.000000
≤
0
)
3
680.632
x
RGA-PSO
*
=
(2.33860239, 1.95126191, −0.45483579, 4.36300325, −0.62317747, 1.02938443, 1.59588410)
g
m
(
x
RGA-PSO
*
)
=
(
-
7.76
E
-
07
≤
0
,
-
252.7211
≤
0
,
-
144.8140
≤
0
,
-
6.15
E
-
05
≤
0
)
4
−15
x
RGA-PSO
*
=
(1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 1)
g
m
(
x
RGA-PSO
*
)
=
(
0
≤
0
,
0
≤
0
,
0
≤
0
,
-
5
≤
0
,
-
5
≤
0
,
-
5
≤
0
,
0
≤
0
,
0
≤
0
,
0
≤
0
)
5
1227.1139
x
RGA-PSO
*
=
(1697.13793410, 54.17332240, 3030.44600072, 90.18199040, 94.99999913, 10.42097385, 153.53771370)
g
m
(
x
RGA-PSO
*
)
=
(
0.999978
≤
1
,
0.980114
≤
1
,
1.000005
≤
1
,
0.980097
≤
1
,
0.990565
≤
1
,
1.000005
≤
1
,
1.00000
≤
1
,
0.976701
≤
1
,
1.000003
≤
1
,
0.459311
≤
1
,
0.387432
≤
1
,
0.981997
≤
1
,
0.980364
≤
1
,
-
8.241926
≤
1
)
6
3.9521
x
RGA-PSO
*
=
(6.444100620, 2.243029250, 0.642672939, 0.582321363, 5.940008650, 5.531235784, 1.018087316, 0.403665649)
g
m
(
x
RGA-PSO
*
)
=
(
1.000000
≤
1
,
1.000000
≤
1
,
0.999996
≤
1
,
0.99986
≤
1
)
7
−5.7398
x
RGA-PSO
*
=
(8.12997229, 0.61463971, 0.56407162, 5.63623069)
g
m
(
x
RGA-PSO
*
)
=
(
1.000000
≤
1
,
1.000000
≤
1
)
8
−83.2497
x
RGA-PSO
*
=
(88.35595404, 7.67259607, 1.31787691)
g
m
(
x
RGA-PSO
*
)
=
(
1.000000
≤
1
)
9
−6.0441
x
RGA-PSO
*
=
(6.40497368, 0.64284563, 1.02766984, 5.94729224, 2.21044814, 0.59816471, 0.42450835, 5.54339987)
g
m
(
x
RGA-PSO
*
)
=
(
0.999997
≤
1
,
0.999959
≤
1
,
0.999930
≤
1
,
0.999975
≤
1
)
10
6299.8374
x
RGA-PSO
*
=
(108.66882633, 85.10837983, 204.35894362)
g
m
(
x
RGA-PSO
*
)
=
(
1.000002
≤
1
)
11
10122.4732
x
RGA-PSO
*
=
(78, 33, 29.99564864, 45, 36.77547104)
g
m
(
x
RGA-PSO
*
)
=
(
-
0.309991
≤
1
,
1.000004
≤
1
,
-
0.021379
≤
1
,
0.621403
≤
1
,
1.000002
≤
1
,
0.681516
≤
1
)
12
0.012692
x
RGA-PSO
*
=
(0.050849843, 0.336663305, 12.579603478)
g
m
(
x
RGA-PSO
*
)
=
(
-
0
.
000143
≤
0
,
-
0.000469
≤
0
,
-
4.009021
≤
0
,
-
0.741658
≤
0
)
13
5885.3018
x
RGA-PSO
*
=
(0.77816852, 0.38464913, 40.31961883, 199.99999988)
g
m
(
x
RGA-PSO
*
)
=
(
1.23
E
-
07
≤
0
,
3.36
E
-
08
≤
0
,
-
0.006916
≤
0
,
-
40.00000012
≤
0
)Table 4
The best solutions obtained using the AIA-PSO algorithm from TPs 1–13.
TP number
f
(
x
A
I
A-
P
S
O
*
)
x
A
I
A-
P
S
O
*
1
24.358
x
AIA-PSO
*
=
(2.18024296, 2.35746157, 8.75670935, 5.11326109, 1.03976363 1.54784227, 1.32994030, 9.83127443, 8.27618717, 8.32717779)
g
m
(
x
AIA-PSO
*
)
=
(
-
0.000071
≤
0
,
-
0.003699
≤
0
,
-
0.000440
≤
0
,
-
0.684025
≤
0
,
-
0.000086
≤
0
,
-
0.001024
≤
0
,
-
5.973866
≤
0
,
-
47.371958
≤
0
)
2
−30665.539
x
AIA-PSO
*
=
(78, 33, 29.99525527, 45, 36.77581286)
g
m
(
x
AIA-PSO
*
)
=
(
-
92.000000
≤
0
,
5.57
E
-
08
≤
0
,
-
8.840500
≤
0
,
-
11.159500
≤
0
,
2.76
E
-
07
≤
0
,
-
5.000000
≤
0
)
3
680.633
x
AIA-PSO
*
=
(2.32925164, 1.95481519, −0.47307614, 4.35691576, −0.62313420, 1.05236194, 1.59750978)
g
m
(
x
AIA-PSO
*
)
=
(
-
4.26
E
-
06
≤
0
,
-
252.612733
≤
0
,
-
144.741194
≤
0
,
-
1.00
E
-
05
≤
0
)
4
−15
x
AIA-PSO
*
=
(1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 1)
g
m
(
x
AIA-PSO
*
)
=
(
0
≤
0
,
0
≤
0
,
0
≤
0
,
-
5
≤
0
,
-
5
≤
0
,
-
5
≤
0
,
0
≤
0
,
0
≤
0
,
0
≤
0
)
5
1227.1598
x
AIA-PSO
*
=
(
1697.91645044
,
53.78132825
,
3031.07989059
,
90.12651301
,
94.99999995
,
10.48103708
,
153.53594861
)
g
m
(
x
AIA-PSO
*
)
=
(
1.000000
≤
1
,
0.980100
≤
1
,
1.000002
≤
1
,
0.980099
≤
1
,
0.990562
≤
1
,
1.000001
≤
1
,
1.000002
≤
1
,
0.976715
≤
1
,
1.000001
≤
1
,
0.459425
≤
1
,
0.387515
≤
1
,
0.981856
≤
1
,
0.987245
≤
1
,
-
8.302533
≤
1
)
6
3.9516
x
AIA-PSO
*
=
(
6.44373647
,
2.23335111
,
0.68233303
,
0.60026617
,
5.93745119
,
5.53146186
,
1.01862958
,
0.40673661
)
g
m
(
x
AIA-PSO
*
)
=
(
0.999999
≤
1
,
0.999999
≤
1
,
0.999994
≤
1
,
0.999985
≤
1
)
7
−5.7398
x
AIA-PSO
*
=
(
8.13042985
,
0.61758136
,
0.56393603
,
5.63618191
)
g
m
(
x
AIA-PSO
*
)
=
(
0.999999
≤
1
,
0.999998
≤
1
)
8
−83.2497
x
AIA-PSO
*
=
(88.35635930, 7.67202113, 1.31765768)
g
m
(
x
AIA-PSO
*
)
=
(
3.00
E
-
18
≤
1
)
9
−6.0467
x
AIA-PSO
*
=
(
6.46393173, 0.67575021, 1.01188804, 5.94072071, 2.24639462, 0.60683404, 0.39677469, 5.52596342)
g
m
(
x
AIA-PSO
*
)
=
(
0.999980
≤
1
,
0.999999
≤
1
,
0.998739
≤
1
,
0.999928
≤
1
)
10
6299.8395
x
AIA-PSO
*
=
(108.97708780, 85.02248757, 204.45439729)
g
m
(
x
AIA-PSO
*
)
=
(
1.000003
≤
1
)
11
10122.4852
x
AIA-PSO
*
=
(78, 33, 29.99571251, 45, 36.77534593)
g
m
(
x
AIA-PSO
*
)
=
(
-
0.309991
≤
1
,
1.000001
≤
1
,
-
0.021380
≤
1
,
0.621403
≤
1
,
1.000001
≤
1
,
0.681517
≤
1
)
12
0.012667
x
AIA-PSO
*
=
(0.05164232,   0.35558085,  11.35742676)
g
m
(
x
AIA-PSO
*
)
=
(
-
8.83
E
-
05
≤
0
,
-
3.04
E
-
05
≤
0
,
-
4.050924
≤
0
,
-
0.728518
≤
0
)
13
5885.3310
x
AIA-PSO
*
=
(0.77816843, 0.38464909, 40.31961929, 199.99999330)
g
m
(
x
AIA-PSO
*
)
=
(
2.22
E
-
07
≤
0
,
7.80
E
-
08
≤
0
,
-
0.006015
≤
0
,
-
40.000007
≤
0
)Table 5
The best parameter settings of the best solution obtained using the RGA-PSO algorithm from TPs 1–13.
TP number
χ
c
1
c
2
ρ
p
m
,
P
S
O
1
0.73759998
1.52939998
1.72753568
28709280994
0.3750
2
0.67057537
0.45010388
2
1000000000
0.2709
3
0.75493696
0.36925226
1.91198475
54900420223
0.3856
4
0.45438391
1.45998477
1.25508289
1000000000
0.1
5
1
0.77483233
2
1000000000
0.5
6
0.70600559
0.82360083
0.91946627
26622414511
0.1619
7
0.74584341
0.95474855
1.17537957
86557786169
0.1958
8
1
0.69725160
1.53028620
1000000000
0.1675
9
0.41638718
0.46594542
1.97807798
12976330236
0.3686
10
0.25378610
0.59619170
0.83891277
1000000000
0.1
11
0.48183123
2
2
1000000000
0.1
12
0.76190087
0.1
1.16855713
100000000000
0.5
13
0.71783704
1.39420750
1.33590124
29071187026
0.2098Table 6
The best parameter settings of the best solution obtained using the AIA-PSO algorithm from TPs 1–13.
TP number
χ
c
1
c
2
ρ
p
m
,
P
S
O
1
0.44896780
0.71709211
1.93302154
72140307110
0.4058
2
1
1.36225259
1.97905466
182831007
0.5
3
0.46599492
0.90346435
1.89697456
69125199709
0.2613
4
0.98124982
0.27882671
0.87437226
85199047430
0.1
5
0.99082484
0.1
1.11371788
4231387044
0.5
6
0.82869043
0.88773247
2
1387448505
0.5
7
0.87571243
1.89936723
0.74306310
94752095153
0.2194
8
0.93583844
1.53906226
1.30374874
22520728225
0.1798
9
1
0.22556712
1.52263349
67578847151
0.4507
10
1
1.93003999
0.1
1351461763
0.5
11
1
1.51209364
1.63826995
1811017789
0.5
12
0.52068067
0.1
2
81914376144
0.1
13
0.82395890
1.60107152
0.93611204
17767111886
0.1813


## 4.2. Comparison of the Results for the Proposed RGA-PSO and AIA-PSO Algorithms with Those Obtained Using the Published Individual GA and AIA Approaches and Hybrid Algorithms

Table7 compares the numerical results of the proposed RGA-PSO and AIA-PSO algorithms with those obtained using published individual GA and AIA approaches for TPs 1–4. In this table, GA-1 is a GA with a penalty function methods, as used by Michalewicz [20]. Notably, GA-2 represents a GA with a penalty function, but without any penalty parameter, as used by Deb [21]. Also, GA-3 is an RGA with a static penalty function, as developed by Wu and Chung [9]. Notably, AIA-1 is an AIA method called CLONALG, as proposed by Cruz-Cortés et al. [22]. Finally, AIA-2 is an AIA approach based on an adaptive penalty function, as developed by Wu [10]. The numerical results of GA-1, GA-2, and AIA-1 methods for solving TPs 1–4 were collected from the published literature [20–22]. Furthermore, the GA-1, GA-2, and AIA-1 approaches were executed under 350,000 objective function evaluations. To fairly compare the performances of the proposed hybrid CI algorithms and the individual GA and AIA approaches, the GA-3, AIA-2, the internal PSO algorithm of RGA-PSO method, and the internal PSO algorithm of AIA-PSO method were independently executed 50 times under 350,000 objective function evaluations for solving TPs 1–4.Table 7
Comparison of the results of the proposed RGA-PSO and AIA-PSO algorithms and those of the published individual GA and AIA approaches for TPs 1–4.
TP number
Global optimum
Methods
Best
Mean
Median
Worst
MAPE%
GA-1 [20]
24.690
—
29.258
36.060
—
GA-2 [21]
24.372
—
24.409
25.075
—
GA-3 [9]
24.935
27.314
27.194
33.160
12.377
1
24.306
AIA-1 [22]
24.506
25.417
—
26.422
—
AIA-2 [10]
24.377
24.669
24.663
24.988
1.495
The proposed RGA-PSO
24.323
24.568
24.521
24.831
1.078
The proposed AIA-PSO
24.358
24.579
24.564
24.809
1.125
GA-1 [20]
—
—
—
—
—
GA-2 [21]
—
—
—
—
—
GA-3 [9]
−30665.526
−30662.922
−30664.709
−30632.445
8.53
E
-
03
2
−30665.539
AIA-1 [22]
−30665.539
−30665.539
—
−30665.539
—
AIA-2 [10]
−30665.539
−30665.526
−30665.527
−30665.506
4.20
E
-
05
The proposed RGA-PSO
−30665.539
−30665.539
−30665.539
−30665.534
1.34
E
-
06
The proposed AIA-PSO
−30665.539
−30665.539
−30665.539
−30665.539
9.95
E
-
07
GA-1 [20]
680.642
—
680.718
680.955
—
GA-2 [21]
680.634
—
680.642
680.651
—
GA-3 [9]
680.641
680.815
680.768
681.395
2.72
E
-
02
3
680.630
AIA-1 [22]
680.631
680.652
—
680.697
—
AIA-2 [10]
680.634
680.653
680.650
680.681
3.45
E
-
03
The proposed RGA-PSO
680.632
680.640
680.639
680.658
1.46
E
-
03
The proposed AIA-PSO
680.633
680.640
680.640
680.657
1.50
E
-
03
GA-1 [20]
−15
−15
−15
−15
—
GA-2 [21]
—
—
—
—
—
GA-3 [9]
−13.885
−12.331
−12.267
−10.467
17.795
4
−15
AIA-1 [22]
−14.987
−14.726
—
−12.917
—
AIA-2 [10]
−14.998
−14.992
−14.992
−14.988
5.08
E
-
02
The proposed RGA-PSO
−15
−15
−15
−15
1.27
E
-
06
The proposed AIA-PSO
−15
−15
−15
−15
1.20
E
-
10
(The “—” denotes unavailable information.)For solving TP 1, the median values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the GA-1, GA-3, and AIA-2 approaches, and the worst values obtained using RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the GA-1, GA-2, GA-3, AIA-1, and AIA-2 approaches. For solving TP 2, the median and worst values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the GA-3 method. For solving TP 3, the median and worst values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the GA-1 and GA-3 approaches. For solving TP 4, the median and worst values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the GA-3 method, and the worst values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the AIA-1 approach. Moreover, the GA-3 method obtained the worst MAPE% for TP 1 and TP 4. Table8 lists the results of the t-test for the GA-3, AIA-2, RGA-PSO, and AIA-PSO methods. This table indicates that the mean values of the RGA-PSO, and AIA-PSO algorithms are not statistically significant, since P values are larger than a significant level 0.05, and the mean values between GA-3 versus AIA-2, GA-3 versus RGA-PSO, GA-3 versus AIA-PSO, AIA-2 versus RGA-PSO, and AIA-2 versus AIA-PSO are statistically significant. According to Tables 7 and 8, the mean values obtained using the RGA-PSO and AIA-PSO algorithms are better than those of obtained using the GA-3 and AIA-1 methods for TPs 1–4.Table 8
Results of thet-test for TPs 1–4.
TP number
GA-3 versus AIA-2
GA-3 versus RGA-PSO
GA-3 versus AIA-PSO
AIA-2 versus RGA-PSO
AIA-2 versus AIA-PSO
RGA-PSO versus AIA-PSO
1
0.000*
0.000*
0.000*
0.000*
0.001*
0.643
2
0.003*
0.003*
0.003*
0.000*
0.000*
0.478
3
0.000*
0.000*
0.000*
0.000*
0.000*
0.839
4
0.000*
0.000*
0.000*
0.000*
0.000*
0.294
*Represents that the mean values obtained using two algorithms are statistically different.)Table9 compares the numerical results obtained using the proposed RGA-PSO and AIA-PSO algorithms and those obtained using AIA-2 and GA-3 for solving TPs 5–13. The AIA-2, GA-3, the internal PSO algorithm of the RGA-PSO approach, and the internal PSO algorithm of AIA-PSO approach were independently executed 50 times under 300,000 objective function evaluations. Table 9 shows that MAPE% obtained using the proposed RGA-PSO and AIA-PSO algorithms is close to 1%, or smaller than 1% for TPs 5–11, indicating that the proposed RGA-PSO and AIA-PSO algorithms can converge to global optimum for TPs 5–11. Moreover, the worst values obtained using the RGA-PSO and AIA-PSO algorithms are significantly smaller than those obtained using the GA-3 method for TPs 5, 6, 11, and 13. Additionally, the worst values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those obtained using the AIA-2 method for TPs 5, 6, and 13.Table 9
Comparison of the numerical results of the proposed RGA-PSO and AIA-PSO algorithms and those of the published individual AIA and RGA for TPs 5–13.
TP number
Global optimum
Methods
Best
Mean
Median
Worst
MAPE%
5
1227.1978
AIA-2 [10]
1227.3191
1228.6097
1227.8216
1239.5205
1.15
E
-
01
GA-3 [9]
1228.5118
1279.3825
1263.2589
1481.3710
4.266
The proposed RGA-PSO
1227.2321
1228.8471
1228.3935
1231.6003
1.34
E
-
01
The proposed AIA-PSO
1227.1598
1228.1800
1227.9762
1229.6590
8.02
E
-
02
6
3.9511
AIA-2 [10]
3.9518
4.1005
4.0460
4.2997
3.781
GA-3 [9]
3.9697
4.1440
4.1358
4.3743
4.881
The proposed RGA-PSO
3.9521
3.9577
3.9566
3.9722
1.67
E
-
01
The proposed AIA-PSO
3.9516
3.9581
3.9569
3.9735
1.77
E
-
01
7
−5.7398
AIA-2 [10]
−5.7398
−5.7398
−5.7398
−5.7396
6.03
E
-
04
GA-3 [9]
−5.7398
−5.7375
−5.7383
−5.7316
3.98
E
-
02
The proposed RGA-PSO
−5.7398
−5.7398
−5.7398
−5.7398
2.54
E
-
04
The proposed AIA-PSO
−5.7398
−5.7398
−5.7398
−5.7398
2.42
E
-
04
8
−83.254
AIA-2 [10]
−83.2504
−83.2499
−83.2499
−83.2496
4.93
E
-
03
GA-3 [9]
−83.2497
−83.2490
−83.2494
−83.2461
6.06
E
-
03
The proposed RGA-PSO
−83.2497
−83.2497
−83.2497
−83.2497
5.13
E
-
03
The proposed AIA-PSO
−83.2497
−83.2497
−83.2497
−83.2497
5.13
E
-
03
9
−6.0482
AIA-2 [10]
−6.0471
−5.9795
−6.0138
−5.7159
1.136
GA-3 [9]
−6.0417
−5.8602
−5.8772
−5.6815
3.108
The proposed RGA-PSO
−6.0441
−5.9883
−5.9968
−5.8051
0.991
The proposed AIA-PSO
−6.0466
−5.9808
−5.9969
−5.8274
1.082
10
6300
AIA-2 [10]
6299.8232
6299.8388
6299.8403
6299.8494
2.56
E
-
03
GA-3 [9]
6299.8399
6299.8546
6299.8453
6299.9407
2.31
E
-
03
The proposed RGA-PSO
6299.8374
6299.8412
6299.8419
6299.8428
2.52
E
-
03
The proposed AIA-PSO
6299.8395
6299.8420
6299.8423
6299.8425
2.51
E
-
03
11
10122.6964
AIA-2 [10]
10122.4343
10122.5014
10122.5023
10122.6271
1.93
E
-
03
GA-3 [9]
10122.4809
10125.2441
10123.9791
10155.1349
2.58
E
-
04
The proposed RGA-PSO
10122.4732
10122.4925
10122.4925
10122.6444
2.01
E
-
03
The proposed AIA-PSO
10122.4852
10122.4920
10122.4927
10122.4931
2.02
E
-
03
12
—
AIA-2 [10]
0.012665
0.012703
0.012719
0.012725
—
GA-3 [9]
0.012665
0.012739
0.012717
0.013471
—
The proposed RGA-PSO
0.012692
0.012724
0.012721
0.012784
—
The proposed AIA-PSO
0.012667
0.012715
0.012719
0.012778
—
13
—
AIA-2 [10]
5885.5312
5900.4860
5894.7929
6014.2198
—
GA-3 [9]
5885.3076
5943.4714
5897.2904
6289.7314
—
The proposed RGA-PSO
5885.3018
5895.0381
5885.3326
6005.4351
—
The proposed AIA-PSO
5885.3310
5886.5426
5885.3323
5906.7404
—
(The “—” denotes unavailable information.)Table10 summarizes the results of the t-test for TPs 5–13. According to Tables 9 and 10, the mean values of the RGA-PSO and AIA-PSO algorithms are smaller than those of the GA-3 approach for TPs 5, 6, 7, 8, 9, 10, 11, and 13. Moreover, the mean values obtained using the RGA-PSO and AIA-PSO algorithms are smaller than those of the AIA-2 approach for TPs 6, 7, 8, 10, and 12. Totally, according to Tables 7−10, the performances of the hybrid CI methods are superior to those of individual GA and AIA methods.Table 10
Results oft-test for TPs 5–13.
TP number
GA-3 versus AIA-2
GA-3 versus RGA-PSO
GA-3 versus AIA-PSO
AIA-2 versus RGA-PSO
AIA-2 versus AIA-PSO
RGA-PSO versus AIA-PSO
5
0.000*
0.000*
0.000*
0.485
0.163
0.002*
6
0.112
0.000*
0.000*
0.000*
0.000*
0.682
7
0.000*
0.000*
0.000*
0.000*
0.000*
0.358
8
0.000*
0.000*
0.000*
0.000*
0.000*
0.291
9
0.000*
0.000*
0.000*
0.516
0.814
0.577
10
0.000*
0.000*
0.000*
0.009*
0.001*
0.001*
11
0.000*
0.000*
0.000*
0.069
0.013*
0.884
12
0.049*
0.389
0.178
0.000*
0.007*
0.014*
13
0.003*
0.001*
0.000*
0.240
0.000*
0.019*
*Represents that the mean values obtained using two algorithms are statistically different.)The TPs 12 and 13 have been solved by many hybrid algorithms. For instance, Huang et al. [23] presented a coevolutionary differential evolution (CDE) that integrates a coevolution mechanism and a DE approach. Zahara and Kao [24] developed a hybrid Nelder-Mead simplex search method and a PSO algorithm (NM-PSO). Table 11 compares the numerical results of the CDE, NM-PSO, RGA-PSO, and AIA-PSO methods for solving TPs 12−13. The table indicates that the best, mean, and worst values obtained using the NM-PSO method are superior to those obtained using the CDE, RGA-PSO, and AIA-PSO approaches for TP 12. Moreover, the best, mean, and worst values obtained using the AIA-PSO algorithm are better than those of the CDE, NM-PSO, and RGA-PSO algorithms.Table 11
Comparison of the numerical results of the proposed RGA-PSO and AIA-PSO algorithms and those of the published hybrid algorithms for TPs 12-13.
TP number
Methods
Best
Mean
Median
Worst
S.D.
12
CDE [23]
0.0126702
0.012703
—
0.012790
2.7
E
-
05
NM-PSO [24]
0.0126302
0.0126314
—
0.012633
8.73
E
-
07
The proposed RGA-PSO
0.012692
0.012724
0.012721
0.012784
1.46
E
-
05
The proposed AIA-PSO
0.012667
0.012715
0.012719
0.012778
2.00
E
-
05
13
CDE [23]
6059.7340
6085.2303
—
6371.0455
43.01
NM-PSO [24]
5930.3137
5946.7901
—
5960.0557
9.16
The proposed RGA-PSO
5885.3018
5895.0381
5885.3326
6005.4351
24.33
The proposed AIA-PSO
5885.3310
5886.5426
5885.3323
5906.7404
4.54According to the No Free Lunch theorem [35], if algorithm A outperforms algorithm B on average for one class of problems, then the average performance of the former must be worse than that of the latter over the remaining problems. Therefore, it is unlikely that any unique stochastic global optimization approach exists that performs best for all CGO problems.


## 4.3. Summary of Results

The proposed RGA-PSO and AIA-PSO algorithms with a penalty function method have the following benefits.(1)
Parameter manipulation of the internal PSO algorithm is based on the solved CGO problems. Owing to their ability to efficiently solve an UGO problem, the external RGA and AIA approaches are substituted for trial and error to manipulate the parameters (χ, c1, c2, ρ, and pm,PSO).(2)
Besides obtaining the optimum parameter settings of the internal PSO algorithm, the RGA-PSO and AIA-PSO algorithms can yield a global optimum for a CGO problem.(3)
In addition to performing better than approaches of some published individual GA and AIA approaches, the proposed RGA-PSO and AIA-PSO algorithms reduce the parametrization for the internal PSO algorithm, despite the RGA-PSO and AIA-PSO algorithms being more complex than individual GA and AIA approaches.The proposed RGA-PSO and AIA-PSO algorithms have the following limitations.(1)
The proposed RGA-PSO and AIA-PSO algorithms increase the computational CPU time, as shown in Table2.(2)
The proposed RGA-PSO and AIA-PSO algorithms are designed to solve CGO problems with continuous decision variablesxn. Therefore, the proposed algorithms cannot be applied to manufacturing problems such as job shop scheduling and quadratic assignment problems (combinatorial optimization problems).


## 5. Conclusions

This work presents novel RGA-PSO and AIA-PSO algorithms. The synergistic power of the RGA with PSO algorithm and the AIA with PSO algorithm is also demonstrated by using 13 CGO problems. Numerical results indicate that, in addition to converging to a global minimum for each test CGO problem, the proposed RGA-PSO and AIA-PSO algorithms obtain the optimum parameter settings of the internal PSO algorithm. Moreover, the numerical results obtained using the RGA-PSO and AIA-PSO algorithms are superior to those obtained using alternative stochastic global optimization methods such as individual GA and AIA approaches. The RGA-PSO and AIA-PSO algorithms are highly promising stochastic global optimization approaches for solving CGO problems.



---

*Source: 841410-2012-08-29.xml*