# Cultural Emperor Penguin Optimizer and Its Application for Face Recognition

Authors: Jie Yang; Hongyuan Gao
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

Face recognition is an important technology with practical application prospect. One of the most popular classifiers for face recognition is support vector machine (SVM). However, selection of penalty parameter and kernel parameter determines the performance of SVM, which is the major challenge for SVM to solve classification problems. In this paper, with a view to obtaining the optimal SVM model for face recognition, a new hybrid intelligent algorithm is proposed for multiparameter optimization problem of SVM, which is a fusion of cultural algorithm (CA) and emperor penguin optimizer (EPO), namely, cultural emperor penguin optimizer (CEPO). The key aim of CEPO is to enhance the exploitation capability of EPO with the help of cultural algorithm basic framework. The performance of CEPO is evaluated by six well-known benchmark test functions compared with eight state-of-the-art algorithms. To verify the performance of CEPO-SVM, particle swarm optimization-based SVM (PSO-SVM), genetic algorithm-based SVM (GA-SVM), CA-SVM, and EPO-SVM, moth-flame optimization-based SVM (MFO-SVM), grey wolf optimizer-based SVM (GWO-SVM), cultural firework algorithm-based SVM (CFA-SVM), and emperor penguin and social engineering optimizer-based SVM (EPSEO-SVM) are used for the comparison experiments. The experimental results confirm that the parameters optimized by CEPO are more instructive to make the classification performance of SVM better in terms of accuracy, convergence rate, stability, robustness, and run time.


---

## Body

## 1. Introduction

As an important branch of pattern recognition, face recognition has the widespread application demands in the field, such as intelligent monitoring, virtual reality, medicine examination, and human-computer interaction. The key step in face recognition is classifier design, namely, how to use the extracted features to classify the new face images. Recently, many classifiers have been presented, such as decision tree [1], neural network [2], k-nearest (KNN) [3], and support vector machine (SVM) [4], among which SVM is the most popular one.Although SVM has the advantages of good generalization ability, small training set size limitation, and high noise stability, it sometimes leads to overfitting problems. To address this problem, some researchers have applied genetic algorithm (GA) [5], particle swarm optimizer (PSO) [6], and simulated annealing (SA) [7] to parameter optimization of SVM. However, the implement of GA is complex although the search process has the characteristic of exclusivity, which makes the convergence speed slow. PSO and SA are successfully applied to solve some engineering optimization problems, but they have poor global search ability for parameters optimization. Therefore, it is worthy to design a new intelligent algorithm with superior performance for parameters optimization of SVM.In the face of complex optimization problems which are difficult to deal with effectively by traditional methods, many new inspired algorithms have been proposed for solving problems by simulations of natural rules and social behaviours of biological population. Recently, some algorithms such as grey wolf optimizer (GWO) [8], moth-flame optimization (MFO) [9], tunicate swarm algorithm (TSA) [10], and dice game optimizer (DGO) [11] are widely used. In 2018, Dhiman and Kumar proposed a novel bio-inspired algorithm named emperor penguin optimizer [12], which is inspired by the budding behavior of emperor penguin. However, the experimental results showed the convergence rate of EPO will decrease continuously in the later stage, which will affect the efficiency of optimization. Afterwards, Baliarsingh et al. proposed a hybrid algorithm using EPO with social engineering optimizer (SEO), namely, emperor penguin and social engineering optimizer (EPSEO) [13] to improve the performance of EPO.Cultural algorithm (CA) is a bio-inspired intelligent algorithm proposed by Reynolds in 1994 [14], which is by simulating the cultural evolution of human society. CA is a double-layer evolutionary mechanism, which uses all kinds of knowledge accumulated in belief space to update continuously, so as to guide the evolution of population space and accelerate the optimization efficiency of the algorithm. Therefore, it can provide a space that allows other evolutionary algorithms to be embedded to cooperate and promote each other. Recently, some researchers have proposed PSO, bacterial foraging algorithm (BFA), and firework algorithm (FA) as a population space to ingrate with CA [15–17] and verified that the performance of the hybrid algorithms are significantly improved.Based on the above observations, due to the fact that CA with single population does not make full use of the knowledge of belief space and the convergence speed of EPO will decrease continuously in the later stage, we propose a hybrid metaheuristic algorithm named cultural emperor penguin optimizer (CEPO) with the help of the embedded framework provided by CA. In CEPO, hybrid of two different evolution processes for locations is an effective mechanism to enhance the exploitation capability of EPO.Specifically, CEPO are implemented on six well-known benchmark test functions compared with eight state-of-the-art optimization algorithms. Furthermore, CEPO is applied to multiparameter optimization problem of SVM for face recognition. In the process of feature, extraction feature representation vectors of face images are obtained by Gabor filter and Principal Component Analysis (PCA), which are as input to CEPO-SVM. PCA is used to make up Gabor wavelet transform disadvantages caused by increased dimensionality. Along with feature classification, penalty parameter and kernel parameter of SVM are optimized by CEPO to get the training model with high classification accuracy. To prove the performance of CEPO-SVM, eight state-of-the-art optimization algorithm-based SVM are used for the comparison experiments in terms of accuracy, convergence rate, stability, robustness, and run time.The main contributions of this paper are as follows:(1)
A new hybrid intelligent algorithm named CEPO inspired by EPO and the framework of CA is proposed(2)
CEPO is applied to automatic learning of the network parameters of SVM to solve the classification problem for face recognition(3)
We demonstrate that the performance of CEPO-SVM is superior on the basis of accuracy, convergence rate, stability, robustness, and run time, compared with eight state-of-the-art algorithm-based SVM(4)
CEPO as a multidimensional search algorithm can not only obtain the optimal penalty parameter and kernel parameter of SVM but also can be employed in other classifiers and other real number optimization problemsThe rest of the work is organized as follows. Section2 describes feature extraction of face images including the Gabor face image representation and dimensionality reduction by Principal Component Analysis. Section 3 presents formulations of parameter optimization of SVM for classification as an optimization problem. The design of cultural emperor penguin optimizer for parameter optimization of SVM is depicted in Section 4. Section 5 describes the experimental setup. In section 6, the experiment and result analysis are presented. Section 7 provides the conclusion and future works.


## 2. The Feature Extraction for Face Images

At the preprocessing stage, each image is first converted into grayscale and then adjusted to the same size. Due to the superior robustness of the Gabor filter to changes in brightness and attitude of face images, the Gabor filter is used to capture more useful features from face images. Furthermore, to make up Gabor wavelet transform disadvantages caused by increased dimensionality, PCA is applied to dimensionality reduction. After the preprocessing stage, face images are divided into training sets and testing sets to obtain feature representation vectors, which are as the input to SVM. Figure1 shows the block diagram of face recognition based on Gabor-PCA.Figure 1
Block diagram of face recognition based on Gabor-PCA.


### 2.1. Gabor Face Image Representation

The two-dimensional Gabor filter can be defined as follows:(1)φo,rz=Wo,r2σ2exp−Wo,r2z22σ2expjWo,rz−exp−σ22,where z=x¯,y¯ defines the pixel position in the spatial domain, o defines the orientation of the Gabor filter, r defines the scale of the Gabor filter. σ represents the radius of Gaussian function, which is used to limit the size of the Gabor filter, and Wo,r is the wave vector of the filter at orientation o and scale r [18, 19]. In this paper, with five different scales, r∈0,…,4, and eight different orientations, o∈0,…,7, 40 different Gabor filters can be obtained. The real part and the magnitude of the 40 Gabor filters are shown in Figures 2 and 3, which exhibit desirable characteristics of spatial locality, spatial frequency, and orientation selectivity.Figure 2
The real part of the 40 Gabor filters.Figure 3
The magnitude of the 40 Gabor filters.LetYz represent the grey level distribution of an image, and the Gabor wavelet representation of an image can be given by(2)Go,rz=Yz∗φo,rz,where Go,rz defines the convolution result between image Yz and the Gabor filters φo,rz at different orientations and scales. In order to improve the computation speed, the convolution computation can be transformed by FFT [20]. To contain the different spatial localities, spatial frequencies, and orientation selectivities, all these representation results will be concatenated to derive an augmented feature vector Q. In [19], before the concatenation, each Go,rz will be downsampled by a factor δ at different scales and orientations. Therefore, the augmented Gabor feature vector Qδ can be given by(3)Qδ=G0,0δTG0,1δT⋯G4,7δTT,where T denotes transpose operator, δ is the downsample factor, and Go,rδ is the concatenated column vector.


### 2.2. Principle Component Analysis for Feature Optimization

The augmented Gabor feature vector which is introduced in equation (3) has very high dimensionality: Qδ∈RO, where O is the dimensionality of vector space. Obviously, the high dimensionality seriously affects the computing speed and recognition rate in the process of face recognition. In the method with PCA [21], we can find orthogonal basis for feature vector, sort dimensions in order of importance, and discard low significance dimensions. Let ∑Qδ∈RO×O be the covariance matrix of the augmented feature vector Qδ:(4)∑Qδ=EQδ−EQδQδ−EQδT,where E⋅ is the expectation operator. The covariance matrix ∑Qδ can be transformed into the following form:(5)∑Qδ=ΩΛΩT,(6)Ω=U1,U2,…,UO,(7)Λ=diagp1,p2,…,pO,where Ω∈RO×O is an orthogonal eigenvector matrix and Λ∈RO×O is a diagonal eigenvalue matrix, and the diagonal elements are arranged in descending order p1≥p2≥⋯≥pO.An important property of PCA is that it is the optimal signal reconstruction in the sense of minimum mean-square error by using a subset of principal components to represent the original signal [19]. Following this property, PCA can be applied to dimensionality reduction:(8)Xδ=STQδ,where S=U1,U2,…,UJ, J<O, and S∈RO×J.The lower dimensional vector Xδ∈RJ captures the most expressive features of the original data Qδ.


## 2.1. Gabor Face Image Representation

The two-dimensional Gabor filter can be defined as follows:(1)φo,rz=Wo,r2σ2exp−Wo,r2z22σ2expjWo,rz−exp−σ22,where z=x¯,y¯ defines the pixel position in the spatial domain, o defines the orientation of the Gabor filter, r defines the scale of the Gabor filter. σ represents the radius of Gaussian function, which is used to limit the size of the Gabor filter, and Wo,r is the wave vector of the filter at orientation o and scale r [18, 19]. In this paper, with five different scales, r∈0,…,4, and eight different orientations, o∈0,…,7, 40 different Gabor filters can be obtained. The real part and the magnitude of the 40 Gabor filters are shown in Figures 2 and 3, which exhibit desirable characteristics of spatial locality, spatial frequency, and orientation selectivity.Figure 2
The real part of the 40 Gabor filters.Figure 3
The magnitude of the 40 Gabor filters.LetYz represent the grey level distribution of an image, and the Gabor wavelet representation of an image can be given by(2)Go,rz=Yz∗φo,rz,where Go,rz defines the convolution result between image Yz and the Gabor filters φo,rz at different orientations and scales. In order to improve the computation speed, the convolution computation can be transformed by FFT [20]. To contain the different spatial localities, spatial frequencies, and orientation selectivities, all these representation results will be concatenated to derive an augmented feature vector Q. In [19], before the concatenation, each Go,rz will be downsampled by a factor δ at different scales and orientations. Therefore, the augmented Gabor feature vector Qδ can be given by(3)Qδ=G0,0δTG0,1δT⋯G4,7δTT,where T denotes transpose operator, δ is the downsample factor, and Go,rδ is the concatenated column vector.


## 2.2. Principle Component Analysis for Feature Optimization

The augmented Gabor feature vector which is introduced in equation (3) has very high dimensionality: Qδ∈RO, where O is the dimensionality of vector space. Obviously, the high dimensionality seriously affects the computing speed and recognition rate in the process of face recognition. In the method with PCA [21], we can find orthogonal basis for feature vector, sort dimensions in order of importance, and discard low significance dimensions. Let ∑Qδ∈RO×O be the covariance matrix of the augmented feature vector Qδ:(4)∑Qδ=EQδ−EQδQδ−EQδT,where E⋅ is the expectation operator. The covariance matrix ∑Qδ can be transformed into the following form:(5)∑Qδ=ΩΛΩT,(6)Ω=U1,U2,…,UO,(7)Λ=diagp1,p2,…,pO,where Ω∈RO×O is an orthogonal eigenvector matrix and Λ∈RO×O is a diagonal eigenvalue matrix, and the diagonal elements are arranged in descending order p1≥p2≥⋯≥pO.An important property of PCA is that it is the optimal signal reconstruction in the sense of minimum mean-square error by using a subset of principal components to represent the original signal [19]. Following this property, PCA can be applied to dimensionality reduction:(8)Xδ=STQδ,where S=U1,U2,…,UJ, J<O, and S∈RO×J.The lower dimensional vector Xδ∈RJ captures the most expressive features of the original data Qδ.


## 3. Formulations of Parameter Optimization of SVM for Classification

### 3.1. Model of Support Vector Machine

Support vector machine (SVM) [22] is a kind of binary classifier with strong learning ability and generalization ability. The key point of SVM is to find the optimal hyperplane for accurate classification of two classes and ensure that the classification gap is large enough. Figure 4 shows the optimal hyperplane of SVM, where H indicates the hyperplane and margin represents the gap between class H1 and class H2.Figure 4
The schematic model of a linear SVM.Suppose the training dataV can be given by(9)V=ah,bh|ah∈Rw,bh∈−1,+1,h=1,2,…,n,where ah are the training samples, bh are the labels in w-dimensional vector, n is the number of training data, and each variable must subject to the criteria ah∈Rw and bh∈−1,+1. For linear data, the hyperplane ga=0 that separates the given data can be determined:(10)ga=ωTa+c,where ω is n-dimensional vector and c is a scalar. ω and c determine all the boundaries. SVM could be classified according to the problem types, when the hyperplane with minimum margin width ω2/2 subject to(11)bhωTa+c≥1.For the data that cannot be separated linearly, the relaxing variablesξh are introduced to allow for optimal generalization and classification by controlling the size of them. Then, the optimal hyperplane can be obtained by the following optimization problem:(12)Ψω,ξ=12ω2+C∑h=1nξh,(13)subject tobhωTa+c≥1−ξh,ξh≥1,where C denotes the penalty parameter. To satisfy the Karush–Kuhn–Tucker (KKT) conditions, the Lagrange multipliers αh is introduced. The abovementioned optimization problem is converted into the dual quadratic optimization problem:(14)minα12∑h=1n∑k=1nbhbkαhαkah⋅ak−∑k=1nαk,(15)subject to∑h=1nbhαh=0,0≤αh≤C.Thus, the ultimate classification function can be obtained by solving the dual optimization problem:(16)ga=sgn∑h=1nαhbhah⋅ak+c,where sgn represents symbolic function.Kernel functions can be used to map the data in the low-dimensional input space to the high-dimensional space by nonlinearity. Therefore, the linear indivisibility of the input can be transformed into the linear separable problem by this mapping. Kernel function can be defined as follows:(17)Kah,bk=ψah,ψbk.Then, the optimal function can be given by(18)ga=sgn∑h=1nαhbhKah⋅ak+c.Several types of kernel functions such as Linear Kernel, Polynomial Kernel, and Radial Basis Function (RBF) are widely used. However, RBF has advantages of realizing nonlinear mapping, less parameters, and less numerical difficulties. In this paper, RBF is selected for face recognition, which can be represented as follows:(19)Ka,b=exp−γa−b2,where γ denotes the kernel parameter.


### 3.2. Objective Function of SVM

With a view to obtaining training model with high classification accuracy, the penalty parameterC which is used to adjust confidence scale and the kernel parameter γ which determines the width of kernel and the range of data points could be optimized by an effective intelligent algorithm. Furthermore, the mean square error of parameters of SVM can be selected as the objective function of the intelligent algorithm:(20)objective=1n∑h=1nbh−bh¯,where bh is the output value of the corresponding parameter and bh¯ is the actual value of the corresponding parameter.


## 3.1. Model of Support Vector Machine

Support vector machine (SVM) [22] is a kind of binary classifier with strong learning ability and generalization ability. The key point of SVM is to find the optimal hyperplane for accurate classification of two classes and ensure that the classification gap is large enough. Figure 4 shows the optimal hyperplane of SVM, where H indicates the hyperplane and margin represents the gap between class H1 and class H2.Figure 4
The schematic model of a linear SVM.Suppose the training dataV can be given by(9)V=ah,bh|ah∈Rw,bh∈−1,+1,h=1,2,…,n,where ah are the training samples, bh are the labels in w-dimensional vector, n is the number of training data, and each variable must subject to the criteria ah∈Rw and bh∈−1,+1. For linear data, the hyperplane ga=0 that separates the given data can be determined:(10)ga=ωTa+c,where ω is n-dimensional vector and c is a scalar. ω and c determine all the boundaries. SVM could be classified according to the problem types, when the hyperplane with minimum margin width ω2/2 subject to(11)bhωTa+c≥1.For the data that cannot be separated linearly, the relaxing variablesξh are introduced to allow for optimal generalization and classification by controlling the size of them. Then, the optimal hyperplane can be obtained by the following optimization problem:(12)Ψω,ξ=12ω2+C∑h=1nξh,(13)subject tobhωTa+c≥1−ξh,ξh≥1,where C denotes the penalty parameter. To satisfy the Karush–Kuhn–Tucker (KKT) conditions, the Lagrange multipliers αh is introduced. The abovementioned optimization problem is converted into the dual quadratic optimization problem:(14)minα12∑h=1n∑k=1nbhbkαhαkah⋅ak−∑k=1nαk,(15)subject to∑h=1nbhαh=0,0≤αh≤C.Thus, the ultimate classification function can be obtained by solving the dual optimization problem:(16)ga=sgn∑h=1nαhbhah⋅ak+c,where sgn represents symbolic function.Kernel functions can be used to map the data in the low-dimensional input space to the high-dimensional space by nonlinearity. Therefore, the linear indivisibility of the input can be transformed into the linear separable problem by this mapping. Kernel function can be defined as follows:(17)Kah,bk=ψah,ψbk.Then, the optimal function can be given by(18)ga=sgn∑h=1nαhbhKah⋅ak+c.Several types of kernel functions such as Linear Kernel, Polynomial Kernel, and Radial Basis Function (RBF) are widely used. However, RBF has advantages of realizing nonlinear mapping, less parameters, and less numerical difficulties. In this paper, RBF is selected for face recognition, which can be represented as follows:(19)Ka,b=exp−γa−b2,where γ denotes the kernel parameter.


## 3.2. Objective Function of SVM

With a view to obtaining training model with high classification accuracy, the penalty parameterC which is used to adjust confidence scale and the kernel parameter γ which determines the width of kernel and the range of data points could be optimized by an effective intelligent algorithm. Furthermore, the mean square error of parameters of SVM can be selected as the objective function of the intelligent algorithm:(20)objective=1n∑h=1nbh−bh¯,where bh is the output value of the corresponding parameter and bh¯ is the actual value of the corresponding parameter.


## 4. Design of Cultural Emperor Penguin Optimizer for Parameter Optimization of SVM

### 4.1. The Design of Cultural Emperor Penguin Optimizer

EPO is a novel optimization algorithm presented by Dhiman and Kumar in 2018 [12], which is inspired by the budding behavior of emperor penguin. In this paper, a hybrid algorithm named CEPO is proposed to solve the real numbers optimization problems, which is with the help of cultural algorithm basic framework in Figure 5. The key idea of CEPO is to obtain problem-solving knowledge from the budding behavior of emperor penguin and make use of that knowledge to guide the evolution of emperor penguin population in return.Figure 5
The cultural algorithm basic framework.Suppose CEPO is designed for general minimum optimization problem:(21)minfxi,where xi=xi1,xi2,…,xiD denotes the position of the ith emperor penguin in the D-dimensional search space, xjmin<xij<xjmax,j=1,2,…,D. f is the objective function, and fxi denotes the objective value of the position xi. xjmin and xjmax represent the lower and upper boundary of the position of emperor penguins in the jth dimension.Belief space of emperor penguin population in thetth generation defined is given by st and Njt, where st is situational knowledge component. Njt is normative knowledge which represents the value space information for each parameter in the jth dimension and in the tth generation. Njt denotes I,L,U. Ijt=ljt,ujt, where Ijt represents the interval of normative knowledge in the jth dimension. The lower boundary ljt and the upper boundary ujt are initialized according to the value range of variables given by the problem. Ljt represents the objective value of the lower boundary ljt of the jth parameter, and Ujt represents the objective value of the upper boundary ujt of the jth parameter.The acceptance function is used to select the emperor penguins who can directly influence the current belief space. In CEPO, the acceptance function selects the cultural individual in proportion top 20% from the current population space to update the belief space.Situational knowledgest can be updated by update function:(22)st+1=xbestt+1iffxbestt+1<fst,st,else,where xbestt+1 is the optimal position of emperor penguin population space in the t+1th generation.Assume that, for theqth cultural individual, a random variable θq lies in the range of 0,1 is produced. The qth cultural individual affects the lower boundary of normative knowledge in the jth dimension when θq<0.5 is satisfied. Normative knowledge Njt can be updated by update function:(23)ljt+1=xqjt+1,ifxqjt+1≤ljtorfxqt+1<Ljt,ljt,else,Ljt+1=fxqt+1,ifxqjt+1≤ljtorfxqt+1<Ljt,Ljt,else.Theqth cultural individual affects the upper boundary of normative knowledge in the jth dimension when θq≥0.5 is satisfied:(24)ujt+1=xqjt+1,ifxqjt+1≥ujtorfxqt+1<Ujt,ujt,else,Ujt+1=fxqt+1,ifxqjt+1≥ujtorfxqt+1<Ujt,Ujt,else.Situational knowledge and normative knowledge can be used to guide emperor penguin population evolution by the influence function. In CEPO, a selection operatorβ is produced to select one of two ways to influence the of evolution emperor penguin population:(25)β=Maxiteration−tMaxiteration,where Maxiteration denotes the maximum number of iterations.Assume that, for theith emperor penguin, a random variable λi which lies in the range of 0,1 is produced. The first way is to update the position of emperor penguin by changing the search size and direction of the variation with belief space, which will be implemented when satisfied λi≤β. The position of emperor penguin in the jth dimension can be updated by(26)xijt+1=xijt+sizeIjt⋅N0,1,ifxijt<sjt,xijt−sizeIjt⋅N0,1,ifxijt>sjt,xijt+η⋅sizeIjt⋅N0,1,else,where N0,1 is a random number subjecting to the standard normal distribution. sizeIjt is the length of adjustable interval of the jth parameter in belief space in the tth generation. η is set to be in the range of 0.01,0.6 by [14].The other way is by a series of steps in EPO which are the huddle boundary generation, temperature profile around the huddle computing, the distance calculation between emperor penguins, and the position update of emperor penguins, which will be carried when satisfiedλi>β. The specific steps can be represented as follows:(27)T′=T−t−MaxiterationMaxiteration,T=0,ifR≥0.5,1,ifR<0.5,where T′ represents the temperature profile around the huddle, T is the time for finding best optimal solution, and R is a random variable which lies in the range of 0,1.(28)Dept=SepAt⋅xbestt−Bt⋅xit,where Dept denotes the distance between the emperor penguin and the optimal solution, xbestt represents the current optimal solution found in emperor penguin population space in the tth generation, Sep represents the social forces of the emperor penguin that is responsible for convergence towards the optimal solution, At and Bt are used to avoid the collision between neighboring emperor penguins, and Bt is a random variable which lies in the range of 0,1. At can be computed as follows:(29)At=M×T′+PgridtAccuracy×rand−T′,PgridtAccuracy=xbestt−xit,where M is the movement parameter which holds a gap between emperor penguins for collision avoidance and PgridtAccuracy defines the absolute difference by comparing the difference between emperor penguins. SepAt in equation (28) is computed as follows:(30)SepAt=ε⋅e−t/ρ−e−t,where e represents the base of natural logarithm. ε and ρ are two control parameters for better exploration and exploitation, which is in the range of 1.5,2 and 2,3. Ultimately, the position of emperor penguin is updated as follows:(31)xit+1=xbestt−At⋅Dept.The algorithm procedure of CEPO is sketched in Algorithm1.Algorithm 1: The algorithm procedure of CEPO.
Algorithm: CEPOInput: the emperor penguin population xii=1,2,…,mOutput: the optimal position of emperor penguin(1)
Procedure CEPO(2)
Initialize the size of population,Maxiteration, the parameters of EPO and CA(3)
Initialize the situational knowledge and normative knowledge of the belief space(4)
Initialize the population space of Eps(5)
Compute the fitness of each EP(6)
Arrange the fitness value of each EP(7)
Initialize the belief space by acceptance proportion 20% andAdjustCulture(8)
Initialize the belief space(9)
While (t<Maxiteration) do(10)
CalculateT and T′ using equations. (9) and (10)(11)
Fori⟵1 to mdo(12)
λi⟵Rand(13)
Forj⟵1 to Ddo(14)
Ifλi≤β using equation (7)(15)
Update the position of EPs using equation (8)(16)
Else(17)
ComputeA and B using equations (12) and (13)(18)
ComputeSepA using equation (14)(19)
Compute the new position of EPs using equation (15)(20)
Compute the fitness of current Eps(21)
Update the better position of EP compared with the previous position(22)
End if(23)
End for(24)
End for(25)
Amend EPs which cross the boundary(26)
Arrange the fitness value of each EP(27)
Update the belief space by acceptance proportion top 20% andAdjustCulture(28)
t⟵t+1(29)
End while(30)
Return the optimal position(31)
End procedure(32)
Procedure AdjustCulture(33)
Update situational knowledge using equation (2)(34)
Forq⟵1 to m/5do(35)
θq⟵Rand(36)
Ifθq<0.5(37)
Forj⟵1 to Ddo(38)
Update the lower boundary of normative knowledge using equation (3) and (4)(39)
Else(40)
Forj⟵1 to Ddo(41)
Update the upper boundary of normative knowledge using equation (5) and (6)(42)
End if(43)
End for(44)
End for(45)
End procedure


### 4.2. CEPO for Parameter Optimization of SVM

During the iterative searching process, each emperor penguin adjusts the optimal position by comparing the value of the objective function, and the optimal positionxi is considered to be the parameter pair (C and γ) for repeated training of SVM and update of each emperor penguin. Figure 6 shows a schematic diagram of SVM based on CEPO.Figure 6
Block diagram of SVM based on CEPO.


## 4.1. The Design of Cultural Emperor Penguin Optimizer

EPO is a novel optimization algorithm presented by Dhiman and Kumar in 2018 [12], which is inspired by the budding behavior of emperor penguin. In this paper, a hybrid algorithm named CEPO is proposed to solve the real numbers optimization problems, which is with the help of cultural algorithm basic framework in Figure 5. The key idea of CEPO is to obtain problem-solving knowledge from the budding behavior of emperor penguin and make use of that knowledge to guide the evolution of emperor penguin population in return.Figure 5
The cultural algorithm basic framework.Suppose CEPO is designed for general minimum optimization problem:(21)minfxi,where xi=xi1,xi2,…,xiD denotes the position of the ith emperor penguin in the D-dimensional search space, xjmin<xij<xjmax,j=1,2,…,D. f is the objective function, and fxi denotes the objective value of the position xi. xjmin and xjmax represent the lower and upper boundary of the position of emperor penguins in the jth dimension.Belief space of emperor penguin population in thetth generation defined is given by st and Njt, where st is situational knowledge component. Njt is normative knowledge which represents the value space information for each parameter in the jth dimension and in the tth generation. Njt denotes I,L,U. Ijt=ljt,ujt, where Ijt represents the interval of normative knowledge in the jth dimension. The lower boundary ljt and the upper boundary ujt are initialized according to the value range of variables given by the problem. Ljt represents the objective value of the lower boundary ljt of the jth parameter, and Ujt represents the objective value of the upper boundary ujt of the jth parameter.The acceptance function is used to select the emperor penguins who can directly influence the current belief space. In CEPO, the acceptance function selects the cultural individual in proportion top 20% from the current population space to update the belief space.Situational knowledgest can be updated by update function:(22)st+1=xbestt+1iffxbestt+1<fst,st,else,where xbestt+1 is the optimal position of emperor penguin population space in the t+1th generation.Assume that, for theqth cultural individual, a random variable θq lies in the range of 0,1 is produced. The qth cultural individual affects the lower boundary of normative knowledge in the jth dimension when θq<0.5 is satisfied. Normative knowledge Njt can be updated by update function:(23)ljt+1=xqjt+1,ifxqjt+1≤ljtorfxqt+1<Ljt,ljt,else,Ljt+1=fxqt+1,ifxqjt+1≤ljtorfxqt+1<Ljt,Ljt,else.Theqth cultural individual affects the upper boundary of normative knowledge in the jth dimension when θq≥0.5 is satisfied:(24)ujt+1=xqjt+1,ifxqjt+1≥ujtorfxqt+1<Ujt,ujt,else,Ujt+1=fxqt+1,ifxqjt+1≥ujtorfxqt+1<Ujt,Ujt,else.Situational knowledge and normative knowledge can be used to guide emperor penguin population evolution by the influence function. In CEPO, a selection operatorβ is produced to select one of two ways to influence the of evolution emperor penguin population:(25)β=Maxiteration−tMaxiteration,where Maxiteration denotes the maximum number of iterations.Assume that, for theith emperor penguin, a random variable λi which lies in the range of 0,1 is produced. The first way is to update the position of emperor penguin by changing the search size and direction of the variation with belief space, which will be implemented when satisfied λi≤β. The position of emperor penguin in the jth dimension can be updated by(26)xijt+1=xijt+sizeIjt⋅N0,1,ifxijt<sjt,xijt−sizeIjt⋅N0,1,ifxijt>sjt,xijt+η⋅sizeIjt⋅N0,1,else,where N0,1 is a random number subjecting to the standard normal distribution. sizeIjt is the length of adjustable interval of the jth parameter in belief space in the tth generation. η is set to be in the range of 0.01,0.6 by [14].The other way is by a series of steps in EPO which are the huddle boundary generation, temperature profile around the huddle computing, the distance calculation between emperor penguins, and the position update of emperor penguins, which will be carried when satisfiedλi>β. The specific steps can be represented as follows:(27)T′=T−t−MaxiterationMaxiteration,T=0,ifR≥0.5,1,ifR<0.5,where T′ represents the temperature profile around the huddle, T is the time for finding best optimal solution, and R is a random variable which lies in the range of 0,1.(28)Dept=SepAt⋅xbestt−Bt⋅xit,where Dept denotes the distance between the emperor penguin and the optimal solution, xbestt represents the current optimal solution found in emperor penguin population space in the tth generation, Sep represents the social forces of the emperor penguin that is responsible for convergence towards the optimal solution, At and Bt are used to avoid the collision between neighboring emperor penguins, and Bt is a random variable which lies in the range of 0,1. At can be computed as follows:(29)At=M×T′+PgridtAccuracy×rand−T′,PgridtAccuracy=xbestt−xit,where M is the movement parameter which holds a gap between emperor penguins for collision avoidance and PgridtAccuracy defines the absolute difference by comparing the difference between emperor penguins. SepAt in equation (28) is computed as follows:(30)SepAt=ε⋅e−t/ρ−e−t,where e represents the base of natural logarithm. ε and ρ are two control parameters for better exploration and exploitation, which is in the range of 1.5,2 and 2,3. Ultimately, the position of emperor penguin is updated as follows:(31)xit+1=xbestt−At⋅Dept.The algorithm procedure of CEPO is sketched in Algorithm1.Algorithm 1: The algorithm procedure of CEPO.
Algorithm: CEPOInput: the emperor penguin population xii=1,2,…,mOutput: the optimal position of emperor penguin(1)
Procedure CEPO(2)
Initialize the size of population,Maxiteration, the parameters of EPO and CA(3)
Initialize the situational knowledge and normative knowledge of the belief space(4)
Initialize the population space of Eps(5)
Compute the fitness of each EP(6)
Arrange the fitness value of each EP(7)
Initialize the belief space by acceptance proportion 20% andAdjustCulture(8)
Initialize the belief space(9)
While (t<Maxiteration) do(10)
CalculateT and T′ using equations. (9) and (10)(11)
Fori⟵1 to mdo(12)
λi⟵Rand(13)
Forj⟵1 to Ddo(14)
Ifλi≤β using equation (7)(15)
Update the position of EPs using equation (8)(16)
Else(17)
ComputeA and B using equations (12) and (13)(18)
ComputeSepA using equation (14)(19)
Compute the new position of EPs using equation (15)(20)
Compute the fitness of current Eps(21)
Update the better position of EP compared with the previous position(22)
End if(23)
End for(24)
End for(25)
Amend EPs which cross the boundary(26)
Arrange the fitness value of each EP(27)
Update the belief space by acceptance proportion top 20% andAdjustCulture(28)
t⟵t+1(29)
End while(30)
Return the optimal position(31)
End procedure(32)
Procedure AdjustCulture(33)
Update situational knowledge using equation (2)(34)
Forq⟵1 to m/5do(35)
θq⟵Rand(36)
Ifθq<0.5(37)
Forj⟵1 to Ddo(38)
Update the lower boundary of normative knowledge using equation (3) and (4)(39)
Else(40)
Forj⟵1 to Ddo(41)
Update the upper boundary of normative knowledge using equation (5) and (6)(42)
End if(43)
End for(44)
End for(45)
End procedure


## 4.2. CEPO for Parameter Optimization of SVM

During the iterative searching process, each emperor penguin adjusts the optimal position by comparing the value of the objective function, and the optimal positionxi is considered to be the parameter pair (C and γ) for repeated training of SVM and update of each emperor penguin. Figure 6 shows a schematic diagram of SVM based on CEPO.Figure 6
Block diagram of SVM based on CEPO.


## 5. The Experiments

### 5.1. Face Databases

In the following experiments, to test our proposed model, we select four face databases that are used commonly and internationally, i.e., YALE (http://cvc.yale.edu/projects/yalefaces/yalefaces.html), ORL (http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html), UMIST (http://images.ee.umist.ac.uk/danny/database.html), and FERET (http://www.frvt.org/). Specifically, the YALE database contains 165 images of 15 individuals, including changes in illumination, expression, and posture. The ORL database consists of 400 photographs of 40 human subjects with different expressions and variations at different scales and orientations. The UMIST database includes 564 images of 20 human subjects with different angles and poses. The FERET database consists of a total of 1400 images for 200 human subjects, corresponding to different poses, expressions, and illumination conditions. To fully verify the validity and rationality of the experiments, configuration of training and testing samples on four face databases is shown in Table 1.Table 1
Configuration of training and test samples on four face databases.
DatabasesTotal numbersNumbers of training samplesNumbers of testing samplesYALE16570 percentthe restORL40070 percentthe restUMIST56470 percentthe restFERET20070 percentthe rest


### 5.2. State-of-the-Art Algorithms for Comparison

To validate the performance of CEPO, the eight state-of-the-art metaheuristics are used for comparison. These are Moth-Flame Optimization (MFO) [9], Grey Wolf Optimizer (GWO) [8], Particle Swarm Optimization (PSO) [6], Genetic Algorithm (GA) [5], Cultural Algorithm (CA) [14], Emperor Penguin Optimizer (EPO) [12], Cultural Firework Algorithm (CFA) [17],and Emperor Penguin and Social Engineering Optimizer (EPSEO) [13].


### 5.3. Experimental Setup

The experimental environment is MATLAB 2018a with LIBSVM on the computer with Intel Core i5 processor and 16 GB of RAM. Table2 Shows the parameter settings of the proposed CEPO and eight competitor intelligent algorithms, i.e., MFO, GWO, PSO, GA, CA, EPO, CFA, and EPSEO. And the parameter values of these algorithms are set to be recommended in their original paper.Table 2
Parameter setting of nine algorithms.
AlgorithmsParametersValuesCultural emperor penguin optimizer (CEPO)Size of population80Control parameterε[1.5, 2]Control parameterρ[2, 3]Movement parameterM2The constantη0.06Maximum iteration100Moth-flame optimization (MFO) [9]Size of population80Convergence constant[−1, −2]Logarithmic spiral0.75Maximum iteration100Grey wolf optimizer (GWO) [8]Size of population80Control parameter[0, 2]Maximum iteration100Particle swarm optimization (PSO) [6]Size of population80Inertia weight0.75Cognitive and social coeff1.8, 2Maximum iteration100Genetic algorithm (GA) [5]Size of population80Probability of crossover0.9Probability of mutation0.05Maximum iteration100Cultural algorithm (CA) [14]Size of population80The constant0.06Maximum iteration100Emperor penguin optimizer (EPO) [12]Size of population80Control parameter[1.5, 2]Control parameter[2, 3]Movement parameter2Maximum iteration100Cultural firework algorithm (CFA) [17]Size of population80Cost parameter0.025, 0.2The constant0.3Maximum iteration100Emperor penguin and social engineering optimizer (EPSEO) [13]Size of population80Rate of training0.2Rate of spotting an attack0.05Number of attacks50Control parameter[1.5, 2]Control parameter[2, 3]Movement parameter2Maximum iteration100


## 5.1. Face Databases

In the following experiments, to test our proposed model, we select four face databases that are used commonly and internationally, i.e., YALE (http://cvc.yale.edu/projects/yalefaces/yalefaces.html), ORL (http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html), UMIST (http://images.ee.umist.ac.uk/danny/database.html), and FERET (http://www.frvt.org/). Specifically, the YALE database contains 165 images of 15 individuals, including changes in illumination, expression, and posture. The ORL database consists of 400 photographs of 40 human subjects with different expressions and variations at different scales and orientations. The UMIST database includes 564 images of 20 human subjects with different angles and poses. The FERET database consists of a total of 1400 images for 200 human subjects, corresponding to different poses, expressions, and illumination conditions. To fully verify the validity and rationality of the experiments, configuration of training and testing samples on four face databases is shown in Table 1.Table 1
Configuration of training and test samples on four face databases.
DatabasesTotal numbersNumbers of training samplesNumbers of testing samplesYALE16570 percentthe restORL40070 percentthe restUMIST56470 percentthe restFERET20070 percentthe rest


## 5.2. State-of-the-Art Algorithms for Comparison

To validate the performance of CEPO, the eight state-of-the-art metaheuristics are used for comparison. These are Moth-Flame Optimization (MFO) [9], Grey Wolf Optimizer (GWO) [8], Particle Swarm Optimization (PSO) [6], Genetic Algorithm (GA) [5], Cultural Algorithm (CA) [14], Emperor Penguin Optimizer (EPO) [12], Cultural Firework Algorithm (CFA) [17],and Emperor Penguin and Social Engineering Optimizer (EPSEO) [13].


## 5.3. Experimental Setup

The experimental environment is MATLAB 2018a with LIBSVM on the computer with Intel Core i5 processor and 16 GB of RAM. Table2 Shows the parameter settings of the proposed CEPO and eight competitor intelligent algorithms, i.e., MFO, GWO, PSO, GA, CA, EPO, CFA, and EPSEO. And the parameter values of these algorithms are set to be recommended in their original paper.Table 2
Parameter setting of nine algorithms.
AlgorithmsParametersValuesCultural emperor penguin optimizer (CEPO)Size of population80Control parameterε[1.5, 2]Control parameterρ[2, 3]Movement parameterM2The constantη0.06Maximum iteration100Moth-flame optimization (MFO) [9]Size of population80Convergence constant[−1, −2]Logarithmic spiral0.75Maximum iteration100Grey wolf optimizer (GWO) [8]Size of population80Control parameter[0, 2]Maximum iteration100Particle swarm optimization (PSO) [6]Size of population80Inertia weight0.75Cognitive and social coeff1.8, 2Maximum iteration100Genetic algorithm (GA) [5]Size of population80Probability of crossover0.9Probability of mutation0.05Maximum iteration100Cultural algorithm (CA) [14]Size of population80The constant0.06Maximum iteration100Emperor penguin optimizer (EPO) [12]Size of population80Control parameter[1.5, 2]Control parameter[2, 3]Movement parameter2Maximum iteration100Cultural firework algorithm (CFA) [17]Size of population80Cost parameter0.025, 0.2The constant0.3Maximum iteration100Emperor penguin and social engineering optimizer (EPSEO) [13]Size of population80Rate of training0.2Rate of spotting an attack0.05Number of attacks50Control parameter[1.5, 2]Control parameter[2, 3]Movement parameter2Maximum iteration100


## 6. Simulation Results and Discussion

### 6.1. Performance of CEPO Comparison Experiment

Six benchmark test functions are applied to verify the superiority and applicability of CEPO. Table3 shows the details of six benchmark test functions including single-peak benchmark functions which are Sphere (F1) and Schwefel 2.22 (F2) and multipeak benchmark functions which are Rastrigin (F3), Griewank (F4), Ackley (F5), and Schaffer (F6). For each benchmark test function, each algorithm carries out thirty independent experiments. The mean and the standard deviation of the thirty independent results are shown in Table 4. As we can see, the statistical results of CEPO for six benchmark test functions are obviously superior to the other eight comparison algorithms. Under the same optimization conditions, the two evaluation indexes of CEPO are both superior to several or even ten orders of magnitude of other algorithms, which proves that the CEPO has strong optimization performance. The optimal mean index shows that the CEPO maintains a high overall optimization level in thirty independent experiments and implies that CEPO has better single optimization accuracy. And the optimal standard deviation index proves that CEPO has strong optimization stability and robustness. Furthermore, under the same test constraints, CEPO attains the high optimization accuracy in the single-peak functions (F1-F2) and multipeak functions (F3-F6) optimization process, which indicates that the CEPO has strong local mining performance and good convergence accuracy, especially for the four multipeak functions (F3-F6) still maintain a high optimization objective value, which verifies its better global exploration performance and stronger local extreme value avoidance ability. If CEPO takes the target value optimization precision as the evaluation index, the preset precision threshold can be as high as 1.0E-12. Especially, for function F1、F3、F6, the threshold value can even be relaxed to 1.0E-25 under the premise of ensuring the success rate of optimization, and the precision can effectively meet the error tolerance range of engineering problems, which further proves that CEPO has better application potential in engineering optimization problems.Table 3
The details of six test benchmark functions.
No.FunctionFormulationDSearch rangeF1Sphere∑i=1Dxi210−100,100F2Schwefel 2.22∑i=1Dxi+∏i=1Dxi()10−10,10F3Rastrigin∑i=1Dxi2−10cos2πxi+10()10−5.12,5.12F4Griewank∑i=1Dxi2/4000−∏i=1Dcosxi/i+1()10−600,600F5Ackley−20exp−0.21/n∑i=1Dxi2−exp1/n∑i=1Dcos2πxi+20+e()10−32,32F6Schaffer0.5+sinx12−x222−0.5/1+0.001x12+x222()2−100,100Table 4
The results of benchmark test functions by nine algorithms.
AlgorithmsFunctionsF1F2F3F4F5F6MFOMean5.28E − 111.69E − 067.21E − 108.38E − 112.67E − 125.09E − 13Std4.19E − 123.77E − 064.93E − 113.41E − 117.19E − 124.81E − 12GWOMean7.03E − 134.43E − 058.48E − 129.76E − 093.83E − 132.92E − 14Std6.92E − 137.82E − 053.12E − 124.28E − 099.06E − 126.37E − 14PSOMean6.36E − 066.14E − 045.56E − 042.07E − 052.98E − 078.14E − 06Std4.23E − 058.25E − 031.79E − 046.23E − 055.24E − 069.93E − 07GAMean2.55E − 072.47E − 045.32E − 054.95E − 051.56E − 083.38E − 07Std3.14E − 078.05E − 043.87E − 051.69E − 056.06E − 082.72E − 07EPOMean8.57E − 069.14E − 053.52E − 076.89E − 075.83E − 094.02E − 08Std4.87E − 062.43E − 054.99E − 074.27E − 077.18E − 095.39E − 08CAMean4.13E − 121.49E − 054.41E − 096.65E − 122.33E − 077.53E − 11Std3.78E − 131.36E − 052.23E − 104.89E − 132.68E − 086.21E − 09CEPOMean1.28E − 274.15E − 146.54E − 262.13E − 212.39E − 247.74E − 29Std4.63E − 272.39E − 147.16E − 261.49E − 214.76E − 243.22E − 29CFAMean4.23E − 233.49E − 102.37E − 198.01E − 175.41E − 201.19E − 24Std1.89E − 228.83E − 109.44E − 192.41E − 172.93E − 194.32E − 23EPSEOMean8.67E − 247.25E − 113.91E − 206.61E − 184.25E − 218.81E − 22Std5.42E − 249.41E − 118.23E − 203.93E − 187.89E − 216.67E − 21


### 6.2. Performance of CEPO-SVM Comparison Experiment

To verify the superiority of the proposed CEPO-SVM classification model for face recognition, CEPO and eight competitor intelligent algorithms are used to iteratively optimize the penalty parameterC and kernel parameter γ of SVM to assess the performance of CEPO. The average recognition rates of thirty independent runs are shown in Table 5. As can be seen clearly from Table 5, CEPO-SVM achieves the highest accuracy among all nine models. In case of YALE face database, an accuracy of 98.1% is obtained by CEPO-SVM model. For ORL, UMIST, and FERET face database, CEPO-SVM gets an accuracy of 97.8%, 98.4%, and 98.5%, respectively. Furthermore, the convergence rate of CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM are examined and depicted in Figures 7(a)–7(d). From this figure, it can be observed that the recognition rate successively increases from iteration 1 to iteration 100 on all four face databases. For YALE database, the accuracy rate has not improved after 14th, 22nd, 31st, 19th, 18th, 21st, 20th, 23rd, and 24th iterations using CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM, respectively. Similarly, for ORL database, no improvement in accuracy is noticed after 11th, 20th, 22nd, 19th, 15th, 20th, 18th, 21st, and 24th iterations using CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM, respectively. In case of UMIST database, CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM converge after 15th, 24th, 25th, 19th, 20th, 26th, 18th, 24th, and 21st iterations, respectively. For FERET database, no rise in accuracy is seen after 16th, 22nd, 24th, 19th, 18th, 20th, 18th, 21st, and 25th iterations using CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM, respectively. Obviously, CEPO-SVM converges firstly among nine proposed models, which verifies that EPO with the help of the embedded framework provided by CA could remarkably enhance convergence rate. Therefore, the parameters of optimization by CEPO are superior to make the classification performance of SVM better.Table 5
Average recognition rates (%) obtained by nine models on four databases.
Average recognition rate (%)DatabaseYALEORLUMISTFERETMFO-SVM96.696.898.097.7GWO-SVM97.197.096.598.2PSO-SVM96.996.597.197.2GA-SVM97.497.597.697.4EPO-SVM97.697.297.997.6CA-SVM97.897.497.797.4CEPO-SVM98.197.898.498.5CFA-SVM97.997.798.398.3EPSEO-SVM97.797.698.198.0Figure 7
Number of iterations versus accuracy on four databases. (a) YALE database. (b) ORL database. (c) UMIST database. (d) FERET database.
(a)(b)(c)(d)


### 6.3. Stability of CEPO-SVM Comparison Experiment

For all nine models, the stability of the models is measured by the mean and standard deviation of penalty parameterC and kernel parameter γ. In this experiment, each model runs thirty experiments. Tables 6–9 show the comparison of mean and standard deviation of parameters among nine models on four face databases. As we can see from Tables 6–9, the mean of penalty parameter C in CEPO-SVM is less than that of other models on four face databases, which could reduce the possibility of overlearning of SVM. And CEPO-SVM provides the lowest standard deviation of penalty parameter C among all nine models on all four databases, which verifies the stability of CEPO-SVM is better. For all four databases, the mean and standard deviation of kernel parameter γ are close among the nine models. However, it can still be seen that CEPO-SVM has the minimum standard deviation of kernel parameter γ. Obviously, CEPO is more stable to optimize the parameters of SVM for face recognition.Table 6
Comparison of mean and standard deviation of parameter on YALE face database among nine optimization algorithms.
AlgorithmsPenalty parameterCKernel parameterγMeanStdMeanStdMFO18.37214.49210.06330.1109GWO16.89834.07480.05180.0978PSO25.23197.42950.12920.1753GA28.36848.70160.14490.2387CA19.16704.86030.06720.1283EPO22.67925.70650.08120.1456CEPO13.24412.49820.01000.0112CFA15.40393.04090.03490.0778EPSEO15.63822.76360.04260.0692Table 7
Comparison of mean and standard deviation of parameter on ORL face database among nine optimization algorithms.
AlgorithmsPenalty parameterCKernel parameterγMeanStdMeanStdMFO18.39214.78170.09650.1143GWO19.04185.15260.08830.1277PSO26.11296.49900.16200.2563GA27.58238.06520.19530.2497CA19.93013.97210.08270.1683EPO22.28016.20760.11980.1762CEPO14.38732.01870.01210.0098CFA16.22654.15520.05730.0829EPSEO15.71923.52280.05250.0701Table 8
Comparison of mean and standard deviation of parameter on UMIST face database among nine optimization algorithms.
AlgorithmsPenalty parameterCKernel parameterγMeanStdMeanStdMFO17.27824.20480.08040.1449GWO16.95563.76490.07730.1283PSO24.90317.19910.14390.1958GA25.34158.23150.15570.2261CA18.87374.01030.07620.1813EPO22.02175.95250.09730.1502CEPO12.91092.20810.01140.0101CFA15.22713.45940.06380.0822EPSEO15.99643.01820.05140.0769Table 9
Comparison of mean and standard deviation of parameter on FERET face database among nine optimization algorithms.
AlgorithmsPenalty parameterCKernel parameterγMeanStdMeanStdMFO18.19284.18920.06190.1023GWO17.32093.97130.05570.0921PSO25.58276.91430.14010.1882GA26.69298.49390.13790.2273CA15.34625.05410.09240.1419EPO21.26795.63140.08460.1364CEPO11.18112.11920.01260.0087CFA15.06473.76720.04890.0748EPSEO14.46893.42950.03770.0661


### 6.4. Robustness of CEPO-SVM Comparison Experiment

To prove the robustness of CEPO-SVM, salt and pepper noise is added to each image of the original four databases. The noise databases generated are used for face recognition by CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM. For each noise databases, we selected 70% of noise samples as the training samples and the rest 30% of noise samples as the testing samples. Salt and pepper noise is a random black or white pixels in an image. Four different levels of noise image on YALE database which are 0%, 5%, 10%, and 15% are shown in Figure8. From Figure 8, we can see that, as the noise level improves, the face image becomes more blurred. Figure 9 shows the detailed recognition rates among different models on four databases. As we can see from Figure 9, CEPO-SVM obtains the highest recognition rate at all four noise levels on among four databases in comparison with other eight models. Although the face image is seriously polluted by salt and pepper noise which is up to 15%, the recognition rate of CEPO-SVM can still attain 82.5% on YALE database, 83.5% on ORL database, 81.1% on UMIST database, and 82.1% on FERET database, which far exceed that of other eight models. Besides, with the increase of noise level, the recognition rate of CEPO-SVM on four databases decreases more slowly than that of other eight models. Obviously, the results show that CEPO-SVM can not only attain good noise-free face recognition accuracy but also achieve good accuracy under noise conditions, which verifies that robustness of CEPO-SVM is superior to the other eight models.Figure 8
Four different levels of noise image on YALE database: (a) 0%. (b) 5%. (c) 10%. (d) 15%.
(a)(b)(c)(d)Figure 9
Noise percentages versus accuracy on four databases: (a) YALE database. (b) ORL database. (c) UMIST database. (d) FERET database.
(a)(b)(c)(d)


### 6.5. Run Time Analysis

The average training time of thirty independent runs by the proposed CEPO-SVM and eight competitor intelligent algorithm-based SVMs are shown in Table10. As we can see from Table 10, in case of all four databases, the average training time by the proposed CEPO-SVM is more than that of six models with single algorithms, i.e., MFO-SVM, GWO-SVM, PSO-SVM, GA-SVM, EPO-SVM, and CA-SVM. This may be because hybrid of two algorithms increases the training time. Although CEPO-SVM takes more time than these six models with single algorithms, it can be considered better than these six models because of its high classification performance. However, for all four databases, the average training time by the proposed CEPO-SVM is less than that of two models with hybrid algorithms, i.e., CFA-SVM and EPSEO-SVM. Obviously, CEPO-SVM has better performance than these two models with hybrid algorithms.Table 10
Average training time (s) obtained by nine models on four databases.
Training time (s)DatabasesYALEORLUMISTFERETMFO-SVM47.9361.5470.9546.03GWO-SVM45.0658.8172.3150.54PSO-SVM43.9554.1265.6948.91GA-SVM69.7182.7398.0379.42EPO-SVM40.6847.4968.5542.87CA-SVM46.3356.8777.4153.24CEPO-SVM59.7679.2395.7667.34CFA-SVM78.2581.96112.8784.59EPSEO-SVM82.4383.15107.4386.77


### 6.6. Limitation Analysis

Like other hybrid algorithms, although CEPO provides superior performance in face recognition, due to large amount of computation, the proposed CEPO will cost more time for iterations compared with single algorithms. Besides, when CEPO is applied to solve super-dimensional optimization problems, the performance may be reduced.


## 6.1. Performance of CEPO Comparison Experiment

Six benchmark test functions are applied to verify the superiority and applicability of CEPO. Table3 shows the details of six benchmark test functions including single-peak benchmark functions which are Sphere (F1) and Schwefel 2.22 (F2) and multipeak benchmark functions which are Rastrigin (F3), Griewank (F4), Ackley (F5), and Schaffer (F6). For each benchmark test function, each algorithm carries out thirty independent experiments. The mean and the standard deviation of the thirty independent results are shown in Table 4. As we can see, the statistical results of CEPO for six benchmark test functions are obviously superior to the other eight comparison algorithms. Under the same optimization conditions, the two evaluation indexes of CEPO are both superior to several or even ten orders of magnitude of other algorithms, which proves that the CEPO has strong optimization performance. The optimal mean index shows that the CEPO maintains a high overall optimization level in thirty independent experiments and implies that CEPO has better single optimization accuracy. And the optimal standard deviation index proves that CEPO has strong optimization stability and robustness. Furthermore, under the same test constraints, CEPO attains the high optimization accuracy in the single-peak functions (F1-F2) and multipeak functions (F3-F6) optimization process, which indicates that the CEPO has strong local mining performance and good convergence accuracy, especially for the four multipeak functions (F3-F6) still maintain a high optimization objective value, which verifies its better global exploration performance and stronger local extreme value avoidance ability. If CEPO takes the target value optimization precision as the evaluation index, the preset precision threshold can be as high as 1.0E-12. Especially, for function F1、F3、F6, the threshold value can even be relaxed to 1.0E-25 under the premise of ensuring the success rate of optimization, and the precision can effectively meet the error tolerance range of engineering problems, which further proves that CEPO has better application potential in engineering optimization problems.Table 3
The details of six test benchmark functions.
No.FunctionFormulationDSearch rangeF1Sphere∑i=1Dxi210−100,100F2Schwefel 2.22∑i=1Dxi+∏i=1Dxi()10−10,10F3Rastrigin∑i=1Dxi2−10cos2πxi+10()10−5.12,5.12F4Griewank∑i=1Dxi2/4000−∏i=1Dcosxi/i+1()10−600,600F5Ackley−20exp−0.21/n∑i=1Dxi2−exp1/n∑i=1Dcos2πxi+20+e()10−32,32F6Schaffer0.5+sinx12−x222−0.5/1+0.001x12+x222()2−100,100Table 4
The results of benchmark test functions by nine algorithms.
AlgorithmsFunctionsF1F2F3F4F5F6MFOMean5.28E − 111.69E − 067.21E − 108.38E − 112.67E − 125.09E − 13Std4.19E − 123.77E − 064.93E − 113.41E − 117.19E − 124.81E − 12GWOMean7.03E − 134.43E − 058.48E − 129.76E − 093.83E − 132.92E − 14Std6.92E − 137.82E − 053.12E − 124.28E − 099.06E − 126.37E − 14PSOMean6.36E − 066.14E − 045.56E − 042.07E − 052.98E − 078.14E − 06Std4.23E − 058.25E − 031.79E − 046.23E − 055.24E − 069.93E − 07GAMean2.55E − 072.47E − 045.32E − 054.95E − 051.56E − 083.38E − 07Std3.14E − 078.05E − 043.87E − 051.69E − 056.06E − 082.72E − 07EPOMean8.57E − 069.14E − 053.52E − 076.89E − 075.83E − 094.02E − 08Std4.87E − 062.43E − 054.99E − 074.27E − 077.18E − 095.39E − 08CAMean4.13E − 121.49E − 054.41E − 096.65E − 122.33E − 077.53E − 11Std3.78E − 131.36E − 052.23E − 104.89E − 132.68E − 086.21E − 09CEPOMean1.28E − 274.15E − 146.54E − 262.13E − 212.39E − 247.74E − 29Std4.63E − 272.39E − 147.16E − 261.49E − 214.76E − 243.22E − 29CFAMean4.23E − 233.49E − 102.37E − 198.01E − 175.41E − 201.19E − 24Std1.89E − 228.83E − 109.44E − 192.41E − 172.93E − 194.32E − 23EPSEOMean8.67E − 247.25E − 113.91E − 206.61E − 184.25E − 218.81E − 22Std5.42E − 249.41E − 118.23E − 203.93E − 187.89E − 216.67E − 21


## 6.2. Performance of CEPO-SVM Comparison Experiment

To verify the superiority of the proposed CEPO-SVM classification model for face recognition, CEPO and eight competitor intelligent algorithms are used to iteratively optimize the penalty parameterC and kernel parameter γ of SVM to assess the performance of CEPO. The average recognition rates of thirty independent runs are shown in Table 5. As can be seen clearly from Table 5, CEPO-SVM achieves the highest accuracy among all nine models. In case of YALE face database, an accuracy of 98.1% is obtained by CEPO-SVM model. For ORL, UMIST, and FERET face database, CEPO-SVM gets an accuracy of 97.8%, 98.4%, and 98.5%, respectively. Furthermore, the convergence rate of CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM are examined and depicted in Figures 7(a)–7(d). From this figure, it can be observed that the recognition rate successively increases from iteration 1 to iteration 100 on all four face databases. For YALE database, the accuracy rate has not improved after 14th, 22nd, 31st, 19th, 18th, 21st, 20th, 23rd, and 24th iterations using CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM, respectively. Similarly, for ORL database, no improvement in accuracy is noticed after 11th, 20th, 22nd, 19th, 15th, 20th, 18th, 21st, and 24th iterations using CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM, respectively. In case of UMIST database, CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM converge after 15th, 24th, 25th, 19th, 20th, 26th, 18th, 24th, and 21st iterations, respectively. For FERET database, no rise in accuracy is seen after 16th, 22nd, 24th, 19th, 18th, 20th, 18th, 21st, and 25th iterations using CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM, respectively. Obviously, CEPO-SVM converges firstly among nine proposed models, which verifies that EPO with the help of the embedded framework provided by CA could remarkably enhance convergence rate. Therefore, the parameters of optimization by CEPO are superior to make the classification performance of SVM better.Table 5
Average recognition rates (%) obtained by nine models on four databases.
Average recognition rate (%)DatabaseYALEORLUMISTFERETMFO-SVM96.696.898.097.7GWO-SVM97.197.096.598.2PSO-SVM96.996.597.197.2GA-SVM97.497.597.697.4EPO-SVM97.697.297.997.6CA-SVM97.897.497.797.4CEPO-SVM98.197.898.498.5CFA-SVM97.997.798.398.3EPSEO-SVM97.797.698.198.0Figure 7
Number of iterations versus accuracy on four databases. (a) YALE database. (b) ORL database. (c) UMIST database. (d) FERET database.
(a)(b)(c)(d)


## 6.3. Stability of CEPO-SVM Comparison Experiment

For all nine models, the stability of the models is measured by the mean and standard deviation of penalty parameterC and kernel parameter γ. In this experiment, each model runs thirty experiments. Tables 6–9 show the comparison of mean and standard deviation of parameters among nine models on four face databases. As we can see from Tables 6–9, the mean of penalty parameter C in CEPO-SVM is less than that of other models on four face databases, which could reduce the possibility of overlearning of SVM. And CEPO-SVM provides the lowest standard deviation of penalty parameter C among all nine models on all four databases, which verifies the stability of CEPO-SVM is better. For all four databases, the mean and standard deviation of kernel parameter γ are close among the nine models. However, it can still be seen that CEPO-SVM has the minimum standard deviation of kernel parameter γ. Obviously, CEPO is more stable to optimize the parameters of SVM for face recognition.Table 6
Comparison of mean and standard deviation of parameter on YALE face database among nine optimization algorithms.
AlgorithmsPenalty parameterCKernel parameterγMeanStdMeanStdMFO18.37214.49210.06330.1109GWO16.89834.07480.05180.0978PSO25.23197.42950.12920.1753GA28.36848.70160.14490.2387CA19.16704.86030.06720.1283EPO22.67925.70650.08120.1456CEPO13.24412.49820.01000.0112CFA15.40393.04090.03490.0778EPSEO15.63822.76360.04260.0692Table 7
Comparison of mean and standard deviation of parameter on ORL face database among nine optimization algorithms.
AlgorithmsPenalty parameterCKernel parameterγMeanStdMeanStdMFO18.39214.78170.09650.1143GWO19.04185.15260.08830.1277PSO26.11296.49900.16200.2563GA27.58238.06520.19530.2497CA19.93013.97210.08270.1683EPO22.28016.20760.11980.1762CEPO14.38732.01870.01210.0098CFA16.22654.15520.05730.0829EPSEO15.71923.52280.05250.0701Table 8
Comparison of mean and standard deviation of parameter on UMIST face database among nine optimization algorithms.
AlgorithmsPenalty parameterCKernel parameterγMeanStdMeanStdMFO17.27824.20480.08040.1449GWO16.95563.76490.07730.1283PSO24.90317.19910.14390.1958GA25.34158.23150.15570.2261CA18.87374.01030.07620.1813EPO22.02175.95250.09730.1502CEPO12.91092.20810.01140.0101CFA15.22713.45940.06380.0822EPSEO15.99643.01820.05140.0769Table 9
Comparison of mean and standard deviation of parameter on FERET face database among nine optimization algorithms.
AlgorithmsPenalty parameterCKernel parameterγMeanStdMeanStdMFO18.19284.18920.06190.1023GWO17.32093.97130.05570.0921PSO25.58276.91430.14010.1882GA26.69298.49390.13790.2273CA15.34625.05410.09240.1419EPO21.26795.63140.08460.1364CEPO11.18112.11920.01260.0087CFA15.06473.76720.04890.0748EPSEO14.46893.42950.03770.0661


## 6.4. Robustness of CEPO-SVM Comparison Experiment

To prove the robustness of CEPO-SVM, salt and pepper noise is added to each image of the original four databases. The noise databases generated are used for face recognition by CEPO-SVM, PSO-SVM, GA-SVM, EPO-SVM, CA-SVM, MFO-SVM, GWO-SVM, CFA-SVM, and EPSEO-SVM. For each noise databases, we selected 70% of noise samples as the training samples and the rest 30% of noise samples as the testing samples. Salt and pepper noise is a random black or white pixels in an image. Four different levels of noise image on YALE database which are 0%, 5%, 10%, and 15% are shown in Figure8. From Figure 8, we can see that, as the noise level improves, the face image becomes more blurred. Figure 9 shows the detailed recognition rates among different models on four databases. As we can see from Figure 9, CEPO-SVM obtains the highest recognition rate at all four noise levels on among four databases in comparison with other eight models. Although the face image is seriously polluted by salt and pepper noise which is up to 15%, the recognition rate of CEPO-SVM can still attain 82.5% on YALE database, 83.5% on ORL database, 81.1% on UMIST database, and 82.1% on FERET database, which far exceed that of other eight models. Besides, with the increase of noise level, the recognition rate of CEPO-SVM on four databases decreases more slowly than that of other eight models. Obviously, the results show that CEPO-SVM can not only attain good noise-free face recognition accuracy but also achieve good accuracy under noise conditions, which verifies that robustness of CEPO-SVM is superior to the other eight models.Figure 8
Four different levels of noise image on YALE database: (a) 0%. (b) 5%. (c) 10%. (d) 15%.
(a)(b)(c)(d)Figure 9
Noise percentages versus accuracy on four databases: (a) YALE database. (b) ORL database. (c) UMIST database. (d) FERET database.
(a)(b)(c)(d)


## 6.5. Run Time Analysis

The average training time of thirty independent runs by the proposed CEPO-SVM and eight competitor intelligent algorithm-based SVMs are shown in Table10. As we can see from Table 10, in case of all four databases, the average training time by the proposed CEPO-SVM is more than that of six models with single algorithms, i.e., MFO-SVM, GWO-SVM, PSO-SVM, GA-SVM, EPO-SVM, and CA-SVM. This may be because hybrid of two algorithms increases the training time. Although CEPO-SVM takes more time than these six models with single algorithms, it can be considered better than these six models because of its high classification performance. However, for all four databases, the average training time by the proposed CEPO-SVM is less than that of two models with hybrid algorithms, i.e., CFA-SVM and EPSEO-SVM. Obviously, CEPO-SVM has better performance than these two models with hybrid algorithms.Table 10
Average training time (s) obtained by nine models on four databases.
Training time (s)DatabasesYALEORLUMISTFERETMFO-SVM47.9361.5470.9546.03GWO-SVM45.0658.8172.3150.54PSO-SVM43.9554.1265.6948.91GA-SVM69.7182.7398.0379.42EPO-SVM40.6847.4968.5542.87CA-SVM46.3356.8777.4153.24CEPO-SVM59.7679.2395.7667.34CFA-SVM78.2581.96112.8784.59EPSEO-SVM82.4383.15107.4386.77


## 6.6. Limitation Analysis

Like other hybrid algorithms, although CEPO provides superior performance in face recognition, due to large amount of computation, the proposed CEPO will cost more time for iterations compared with single algorithms. Besides, when CEPO is applied to solve super-dimensional optimization problems, the performance may be reduced.


## 7. Conclusions

In this paper, with respective advantages of EPO and CA, we propose a hybrid metaheuristic algorithm named CEPO with the help of the embedded framework provided by CA to promote each other and overcome each other’s self-defects. The proposed algorithm has been experimented on six benchmark test functions compared with other eight state-of-the-art algorithms. Furthermore, we apply this hybrid metaheuristic algorithm to automatic learning of the network parameters of SVM to solve the classification problem for face recognition. The proposed classifier model has been tested on four face databases that are used commonly and internationally by comparing with models with other eight state-of-the-art algorithms. The experimental results show that the proposed model has better performance in terms of accuracy, convergence rate, stability, robustness, and run time.We provide future works in different directions. First, the proposed model has only been experimented on four face databases; however, other new face databases should be tested to extend the current work. Second, the proposed hybrid algorithm should be applied to other classifiers for face recognition. Lastly, the proposed hybrid algorithm should be applied to solve other optimization problems, such as digital filters design, cognitive radio spectrum allocation, and image segmentation.



---

*Source: 9579538-2020-11-05.xml*