# Gait Recognition Based on the Feature Extraction of Gabor Filter and Linear Discriminant Analysis and Improved Local Coupled Extreme Learning Machine

Authors: Hongli Guo; Bin Li; Youmei Zhang; Yu Zhang; Wei Li; Fengjuan Qiao; Xuewen Rong; Shuwang Zhou
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

A gait energy image contains much gait information, which is one of the most effective means to recognize gait characteristics. The accuracy of gait recognition is greatly affected by covariates, such as the viewing angle, occlusion of clothing, and walking speed. Gait features differ somewhat by angles. Therefore, how to improve the recognition accuracy of a cross-view gait is a challenging task. This study proposes a new gait recognition algorithm structure. A Gabor filter is used to extract gait features from gait energy images, since it can extract features of different directions and scales. We use linear discriminant analysis (LDA) to tackle the problem that the feature dimension restricts the process. Finally, the improved local coupled extreme learning machine based on particle swarm optimization is used for the classification process of the extracted features of the gait. The proposed method and other current mainstream algorithms are compared in terms of the recognition accuracy based on the CASIA-A and CASIA-B datasets, and the simulation results show that the proposed algorithm has good performance and performs well at cross-view gait recognition.


---

## Body

## 1. Introduction

Identity recognition is a foundation of human social life, and increasing interest in biometrics of recognition algorithms has led to rapid improvements in biometric technologies with better performance. Biometric recognition is identification based on personal physiological or behavioral characteristics [1]. Common biometric recognition technologies include facial, fingerprint, palm print, and iris recognition. Biological patterns and features are usually unique and are impossible to forge or copy [2], so biometric authentication has great advantages over the traditional authentication.A person’s gait, or way of walking, is a complex spatiotemporal biological feature that can be used to distinguish an individual [3], and hence, to realize personal identification [4]. Research [5] shows that the human gait is unique, and it is difficult to fake. We all have the experience of identifying friends and family through gait information. Unlike biometrics, such as the face, fingerprint, and iris, gait sequences can be collected undisturbed at long distances with minimal cooperation [6]. Therefore, gait recognition has the advantages that it requires no contact and is noninvasive. Furthermore, gait is difficult to hide or camouflage [7].Gait recognition methods can be categorized as model-based [8–10] or appearance-based [11, 12]. Model-based methods construct the gait image to analyze the changes of parameters in the movement process and obtain static or dynamic parameter information of the body, such as height, leg length, swing angle, swing frequency of arms or legs, and stride length. Lee and Grimson [13] proposed to divide the silhouette of a foreground walking person into seven regions, fit each with an ellipse, and obtain the parameter information as the gait characteristics for classification and recognition. Cunado et al. [14] modeled the leg as two linked pendulums, one being the thigh between the knee and hip and the other the calf between the knee and ankle. They extracted the step length and distance from the hip to the ankle as gait characteristics for identification and classification. Training and test sequences are often selected from gait sequences captured from different angles [15]. The model-based method is difficult as regards model construction, which requires the simulation of gait information. Another problem is the complexity of parameter calculation.Appearance-based methods focus on the shape of the silhouette or the overall movement of an individual. For example, an appearance-based method analyzes the changes of the contour of a walker over time [16] to obtain spatiotemporal characteristics for gait recognition. Wu et al. [17] proposed a nonnegative matrix factorization (NMF) method to obtain the local structure features of a human body to compensate for the loss of accuracy. A two-dimensional linear discriminant analysis (2DLDA) was proposed to project features into the discriminant space to improve classification. Aggarwal and Vishwakarma [18] proposed a 2D spatiotemporal template to describe gait motion, calculate Zernike moment invariants, and extract features from the spatial distribution of the directional gradient and the new mean method of directional pixels. More and Deore [19] proposed to fuse dynamic and static features for gait recognition, using a cross-wavelet transform to extract dynamic features, and a bipartite graph model to extract static features, followed by normalized feature fusion. In the Bayesian framework, k-means clustering is used for classification and recognition. Appearance-based methods are increasingly popular because they do not require modeling of all or part of the human body, they have low computational complexity, and they are insensitive to the quality of the profile image.Gait recognition has three steps: target detection, feature extraction, and recognition. Feature extraction plays an important role in the recognition process and directly affects its accuracy. Hence, a suitable feature extraction algorithm is necessary. The Gabor filter is increasingly used in image processing [20]. Biological studies have shown that its expression of frequency and direction is similar to that of the human visual system [21], it can well simulate the sensory field function of single cells in the cerebral cortex, and it is a biomimetic mathematical model. In addition, it can carry out multidirection and multiscale feature extraction. Therefore, we use a Gabor filter with eight directions and five scales to extract features from the gait energy image, and the output of the Gabor filter is the extracted feature of the gait. However, the gait feature dimension increases the computing cost, and high-dimension features contain some redundant information, so it is necessary to reduce the dimension of the data extracted by means of the Gabor filter. For this, we use linear discriminant analysis (LDA), which is a supervised algorithm that can choose the best projection direction of classification performance and enhance the linear separability of data. It is a powerful tool for data dimensionality reduction. An improved local coupled extreme learning machine based on the particle swarm optimization (LC-PSO-ELM) algorithm is used to classify the gait feature and achieve better recognition accuracy. To demonstrate the algorithm’s effectiveness, we compare the dimension-reduction method to principal component analysis (PCA), and the proposed method of gait recognition in this paper is compared with the other current mainstream algorithms based on the CASIA-A and CASIA-B benchmark gait databases of the Institute of Automation, Chinese Academy of Sciences. The simulation results show that the proposed method can improve the recognition accuracy of gait and performs well at cross-view gait recognition.The contributions of this study can be summarized as follows:(1)
An automatic gait recognition structure is proposed(2)
A Gabor filter is used to extract features of the gait energy map, and LDA is used to reduce the feature dimension, thus retaining as much feature information as possible(3)
An improved local coupled extreme learning machine based on the particle swarm optimization (LC-PSO-ELM) algorithm is used to classify the extracted feature of the gait, which can improve the recognition accuracy at cross-viewThe rest of this study is arranged as follows: the gait feature extraction method is described in Section2. Dimension reduction and the recognition method are described in Section 3. Experimental results and discussion are presented in Section 4. Section 5 presents our conclusions.


## 2. Gait Feature Extraction

Feature extraction is an important step in gait recognition. In this study, a Gabor filter is used to extract gait features from a gait energy image (GEI).


### 2.1. Gait Energy Image

The GEI is one of the most effective ways to represent gait information [22]. It is the average silhouette of a gait cycle, which is a normalized cumulative energy image in a complete cycle [23]. The random noise of the image sequence in the periodic process is suppressed in the process of averaging the image. The obtained average image is robust and contains rich static and dynamic information. More and more researchers prefer to extract gait features from the GEI [24]. It is defined as(1)GEIx,y=1N∑t=1NBtx,y,where N is the number of frames of the silhouette image in a period and Btx,y is the gait silhouette at the time t. The GEI generation process is shown in Figure 1.Figure 1
The generation process of gait energy image.


### 2.2. Gabor Filter

A Gabor function can be used for edge detection in image processing. The two-dimensional Gabor filter can obtain the optimal localization in the spatial and frequency domains, so it can well describe the local structure information of an image corresponding to the spatial scale, spatial location, and direction selectivity. The frequency and direction representation of a Gabor filter are close to those of the human vision system and are often used to represent and describe texture features [25, 26]. We choose the following two-dimensional Gabor kernel form [27]:(2)ψu,vz=||ku,v||2σ2e−||ku,v||2||x||2/2σ2eiku,v2−e−σ2/2,where u and v define the direction and scale, respectively, of the Gabor kernel, z=x,y, |⋅| represents modulus calculation, ku,v=kveiφu, kv=kmax/fv, and φu=πu/8, kmax is the maximum frequency, and f is the distribution coefficient of the kernel function in the frequency domain. In this paper, the Gabor filter is used to extract the features of five scales and eight directions. We set the parameters of formula (2) as v=0,...,4, u=0,...,7, σ=2π, kmax=0.5π, and f=2. We extract features of gait energy images by a Gabor kernel with 40 directions and scales. The extracted features are represented by a high-dimensional column vector, and high-dimensional gait feature data can be obtained from the output of the Gabor filter.


## 2.1. Gait Energy Image

The GEI is one of the most effective ways to represent gait information [22]. It is the average silhouette of a gait cycle, which is a normalized cumulative energy image in a complete cycle [23]. The random noise of the image sequence in the periodic process is suppressed in the process of averaging the image. The obtained average image is robust and contains rich static and dynamic information. More and more researchers prefer to extract gait features from the GEI [24]. It is defined as(1)GEIx,y=1N∑t=1NBtx,y,where N is the number of frames of the silhouette image in a period and Btx,y is the gait silhouette at the time t. The GEI generation process is shown in Figure 1.Figure 1
The generation process of gait energy image.


## 2.2. Gabor Filter

A Gabor function can be used for edge detection in image processing. The two-dimensional Gabor filter can obtain the optimal localization in the spatial and frequency domains, so it can well describe the local structure information of an image corresponding to the spatial scale, spatial location, and direction selectivity. The frequency and direction representation of a Gabor filter are close to those of the human vision system and are often used to represent and describe texture features [25, 26]. We choose the following two-dimensional Gabor kernel form [27]:(2)ψu,vz=||ku,v||2σ2e−||ku,v||2||x||2/2σ2eiku,v2−e−σ2/2,where u and v define the direction and scale, respectively, of the Gabor kernel, z=x,y, |⋅| represents modulus calculation, ku,v=kveiφu, kv=kmax/fv, and φu=πu/8, kmax is the maximum frequency, and f is the distribution coefficient of the kernel function in the frequency domain. In this paper, the Gabor filter is used to extract the features of five scales and eight directions. We set the parameters of formula (2) as v=0,...,4, u=0,...,7, σ=2π, kmax=0.5π, and f=2. We extract features of gait energy images by a Gabor kernel with 40 directions and scales. The extracted features are represented by a high-dimensional column vector, and high-dimensional gait feature data can be obtained from the output of the Gabor filter.


## 3. Dimension Reduction and Recognition Methods

The extracted feature of the gait based on the Gabor filter is high-dimensional data, and the character dimension for the classification algorithm is 4400. Therefore, some dimensionality reduction methods should be used to decrease the input complexity of the classification algorithm. In this study, we use LDA for dimension reduction of gait features.


### 3.1. Linear Discriminant Analysis

LDA is widely used for dimension reduction [28]. This algorithm can produce a reduced sample whose data have the maximum distance between classes and the minimum variance within classes in the new dimension space and the best linear separability. For the n-dimensional feature data, LDA adopts the orthogonal transformation method to obtain m-dimensional new feature data that are unrelated and are beneficial to classification, while minimizing information loss and m<n. Considering the classification information of data, the algorithm can make the data easier to identify after dimension reduction. The algorithm selects the axis of the orthogonal transformation in the direction of large data variance. It reduces the correlation of different types of data and causes the transformation result to highlight the differences of feature data [29].Given the input and output datasetsxi,ti,i=1,…,L, where xi∈Rn, ti∈Rq,and L is the number of datasets. Assume that the data matrix is grouped as C=C1,…,Ck, where k is the number of classes. Suppose uki and ∑ki are the mean vector and covariance matrix of the class ki, respectively. Let ki and kj be two arbitrary classes in the group of C, the projection of the center of these two classes on a straight line is, respectively, wtuki and wtukj, and the covariance of the two types of classes is wT∑kiw and wT∑kjw, respectively. The LDA method projects high-dimensional data onto a lower dimensional space by maximizing the interclass variance from different classes and minimizing intraclass variance from the same class simultaneously, thus achieving maximum class discrimination in the dimensionality-reduced space. That is, a matrix wis calculated based on the objective function J as follows [30]:(3)J=||wTuki−wTukj||22wT∑kiw+wT∑kjw=wTuki−ukjuki−ukjwwT∑ki+wT∑kjw.The intraclass divergence matrix is defined as follows:(4)Sw=∑ki+∑kj=∑x∈kix−ukix−ukiT+∑x∈kjx−ukjx−ukjT.The class dispersion matrix is defined as follows:(5)Sb=uki−ukjuki−ukjT.Therefore, the general form of matrix representation of above objective function (3) can be obtained as follows:(6)J=wTSbwwTSww.


### 3.2. Recognition Method

#### 3.2.1. Extreme Learning Machine

Extreme learning machine (ELM) is a fast learning algorithm that was proposed by Huang in 2006 [31]. Unlike the traditional single-hidden layer neural network, ELM can randomly initialize the input weight and bias and obtain the corresponding output weight. The calculation is relatively simple, the computational complexity is low, and it has been studied and used by many researchers. In this paper, an improved ELM algorithm based on the local coupled extreme learning machine (LC-ELM) [32] and particle swarm optimization (PSO) is used for gait recognition. We first introduce the basic ELM learning algorithm.For a single-hidden-layer neural network, suppose there are arbitrary samplesXi,ti, Xixi1,xi2,...,xipT∈Rp, ti=ti1,ti2,...,tiq∈Rq. A neural network with N neurons in a hidden layer can be expressed as follows:(7)Oj=∑i=1NβigWi⋅Xj+bi,i=1,...,N,j=1,2,...,M,where gx is the activation function, M is the number of the input samples, Wi is the input weight, βi is the output weight, bi is the bias of hidden layer neurons, and Wi⋅Xj is the inner product of Wi and Xj.The goal of ELM neural network learning is to minimize the output error, which can be expressed as follows:(8)∑j=1M||Oj−tj||=0.The aboveM equations can be written compactly as(9)Hβ=T,where β=β1,…,βNN×qT and T=t1,…,tNTM×q. H is called the hidden-layer output matrix of the ELM learning algorithm [33].The training process of ELM learning algorithm is equivalent to finding a least squares solutionβ^ of the linear system Hβ=T, and the above equation can be translated to(10)β^=H+T,where H+ is the Moore–Penrose generalized inverse of H.ELM algorithms are implemented by a fully coupled framework between the input layer and hidden layer. This fully coupled structure may incur higher computing costs when ELM has many hidden nodes and higher dimensions of input data.


#### 3.2.2. Improved Extreme Learning Machine

To reduce the amount of computation, Qu [32] proposed LC-ELM, which was to decouple the input layer and the hidden layer based on ELM. Each hidden node is assigned a parameter in an input space [34]. Given a learning sample, the fuzzy membership function is used to measure the distance between the hidden node and the input sample as the activation degree of the hidden node. A fuzzy membership function and similarity relation are used to realize LC-ELM, which can reduce the complexity of the weight search space and improve the generalization performance. LC-ELM can be expressed as follows:(11)fXj=∑i=1NβigWi⋅Xj+biFSXj,di,i=1,...,N,j=1,2,...,M.In the LC-ELM learning algorithm, the similarity relationSX,di denotes the distance between the input X and the i-th hidden layer neurons, which is expressed as the address di. F⋅ is a fuzzy membership function, and there are many choices, such as a Gaussian, sigmoid, or reverse sigmoid function. The underlying radius parameter r is kept in F⋅ to adjust the width of the activation area, which is also an optimized parameter, to match the address parameter di. Combining the structure of LCFNN with the learning mechanism of ELM, LC-ELM also is a three-step learning algorithm whose network parameters (input weights W, biases b between the input layer and hidden layer, and address d of hidden neurons) are assigned randomly, which is the same as ELM [32].Compared with ELM, LC-ELM reduces the complexity of the algorithm structure, but its parameters are input randomly, and it may not be optimal, which will affect its performance. We propose a local coupled extreme learning machine based on particle swarm optimization [34] to improve the performance. This algorithm uses a particle swarm optimization strategy to optimize the four parameters W,b,d,r in LC-ELM, so as to improve its accuracy and generalization performance.Therefore, the particles in the searching space of the LC-PSO-ELM are composed of a set by the parameter values of input weights, hidden biases, address, and radius, which can be defined as(12)θ∈w11,w12,...,w1L,w21,w22,...,w2L,...,wp1,...,wpL,...,b1,b2,...,bN,d11,d12,...,d1L,d21,d22,...,d2L,...,dp1,dp2,...,dpL,...,r1,r2,...,rL,,where W=Wi|Wi∈Rp,i=1,...,L,b∈RL,d=di|di∈Rp,i=1,2,..,L,andr∈RL. When the optimal parameters of the local coupled extreme learning machine based on the particle swarm optimization algorithm are established based on formula (12), the weights between the hidden layer and the output layer of the algorithm are determined analytically based on formula (10). The flowchart of the LC-PSO-ELM algorithm is shown in Figure 2.Figure 2
Flow chart of local coupled extreme learning machine based on the particle swarm optimization.LC-PSO-ELM improves accuracy and generalization performance, and hence, we use it to recognize and classify gait features after dimension reduction.


## 3.1. Linear Discriminant Analysis

LDA is widely used for dimension reduction [28]. This algorithm can produce a reduced sample whose data have the maximum distance between classes and the minimum variance within classes in the new dimension space and the best linear separability. For the n-dimensional feature data, LDA adopts the orthogonal transformation method to obtain m-dimensional new feature data that are unrelated and are beneficial to classification, while minimizing information loss and m<n. Considering the classification information of data, the algorithm can make the data easier to identify after dimension reduction. The algorithm selects the axis of the orthogonal transformation in the direction of large data variance. It reduces the correlation of different types of data and causes the transformation result to highlight the differences of feature data [29].Given the input and output datasetsxi,ti,i=1,…,L, where xi∈Rn, ti∈Rq,and L is the number of datasets. Assume that the data matrix is grouped as C=C1,…,Ck, where k is the number of classes. Suppose uki and ∑ki are the mean vector and covariance matrix of the class ki, respectively. Let ki and kj be two arbitrary classes in the group of C, the projection of the center of these two classes on a straight line is, respectively, wtuki and wtukj, and the covariance of the two types of classes is wT∑kiw and wT∑kjw, respectively. The LDA method projects high-dimensional data onto a lower dimensional space by maximizing the interclass variance from different classes and minimizing intraclass variance from the same class simultaneously, thus achieving maximum class discrimination in the dimensionality-reduced space. That is, a matrix wis calculated based on the objective function J as follows [30]:(3)J=||wTuki−wTukj||22wT∑kiw+wT∑kjw=wTuki−ukjuki−ukjwwT∑ki+wT∑kjw.The intraclass divergence matrix is defined as follows:(4)Sw=∑ki+∑kj=∑x∈kix−ukix−ukiT+∑x∈kjx−ukjx−ukjT.The class dispersion matrix is defined as follows:(5)Sb=uki−ukjuki−ukjT.Therefore, the general form of matrix representation of above objective function (3) can be obtained as follows:(6)J=wTSbwwTSww.


## 3.2. Recognition Method

### 3.2.1. Extreme Learning Machine

Extreme learning machine (ELM) is a fast learning algorithm that was proposed by Huang in 2006 [31]. Unlike the traditional single-hidden layer neural network, ELM can randomly initialize the input weight and bias and obtain the corresponding output weight. The calculation is relatively simple, the computational complexity is low, and it has been studied and used by many researchers. In this paper, an improved ELM algorithm based on the local coupled extreme learning machine (LC-ELM) [32] and particle swarm optimization (PSO) is used for gait recognition. We first introduce the basic ELM learning algorithm.For a single-hidden-layer neural network, suppose there are arbitrary samplesXi,ti, Xixi1,xi2,...,xipT∈Rp, ti=ti1,ti2,...,tiq∈Rq. A neural network with N neurons in a hidden layer can be expressed as follows:(7)Oj=∑i=1NβigWi⋅Xj+bi,i=1,...,N,j=1,2,...,M,where gx is the activation function, M is the number of the input samples, Wi is the input weight, βi is the output weight, bi is the bias of hidden layer neurons, and Wi⋅Xj is the inner product of Wi and Xj.The goal of ELM neural network learning is to minimize the output error, which can be expressed as follows:(8)∑j=1M||Oj−tj||=0.The aboveM equations can be written compactly as(9)Hβ=T,where β=β1,…,βNN×qT and T=t1,…,tNTM×q. H is called the hidden-layer output matrix of the ELM learning algorithm [33].The training process of ELM learning algorithm is equivalent to finding a least squares solutionβ^ of the linear system Hβ=T, and the above equation can be translated to(10)β^=H+T,where H+ is the Moore–Penrose generalized inverse of H.ELM algorithms are implemented by a fully coupled framework between the input layer and hidden layer. This fully coupled structure may incur higher computing costs when ELM has many hidden nodes and higher dimensions of input data.


### 3.2.2. Improved Extreme Learning Machine

To reduce the amount of computation, Qu [32] proposed LC-ELM, which was to decouple the input layer and the hidden layer based on ELM. Each hidden node is assigned a parameter in an input space [34]. Given a learning sample, the fuzzy membership function is used to measure the distance between the hidden node and the input sample as the activation degree of the hidden node. A fuzzy membership function and similarity relation are used to realize LC-ELM, which can reduce the complexity of the weight search space and improve the generalization performance. LC-ELM can be expressed as follows:(11)fXj=∑i=1NβigWi⋅Xj+biFSXj,di,i=1,...,N,j=1,2,...,M.In the LC-ELM learning algorithm, the similarity relationSX,di denotes the distance between the input X and the i-th hidden layer neurons, which is expressed as the address di. F⋅ is a fuzzy membership function, and there are many choices, such as a Gaussian, sigmoid, or reverse sigmoid function. The underlying radius parameter r is kept in F⋅ to adjust the width of the activation area, which is also an optimized parameter, to match the address parameter di. Combining the structure of LCFNN with the learning mechanism of ELM, LC-ELM also is a three-step learning algorithm whose network parameters (input weights W, biases b between the input layer and hidden layer, and address d of hidden neurons) are assigned randomly, which is the same as ELM [32].Compared with ELM, LC-ELM reduces the complexity of the algorithm structure, but its parameters are input randomly, and it may not be optimal, which will affect its performance. We propose a local coupled extreme learning machine based on particle swarm optimization [34] to improve the performance. This algorithm uses a particle swarm optimization strategy to optimize the four parameters W,b,d,r in LC-ELM, so as to improve its accuracy and generalization performance.Therefore, the particles in the searching space of the LC-PSO-ELM are composed of a set by the parameter values of input weights, hidden biases, address, and radius, which can be defined as(12)θ∈w11,w12,...,w1L,w21,w22,...,w2L,...,wp1,...,wpL,...,b1,b2,...,bN,d11,d12,...,d1L,d21,d22,...,d2L,...,dp1,dp2,...,dpL,...,r1,r2,...,rL,,where W=Wi|Wi∈Rp,i=1,...,L,b∈RL,d=di|di∈Rp,i=1,2,..,L,andr∈RL. When the optimal parameters of the local coupled extreme learning machine based on the particle swarm optimization algorithm are established based on formula (12), the weights between the hidden layer and the output layer of the algorithm are determined analytically based on formula (10). The flowchart of the LC-PSO-ELM algorithm is shown in Figure 2.Figure 2
Flow chart of local coupled extreme learning machine based on the particle swarm optimization.LC-PSO-ELM improves accuracy and generalization performance, and hence, we use it to recognize and classify gait features after dimension reduction.


## 3.2.1. Extreme Learning Machine

Extreme learning machine (ELM) is a fast learning algorithm that was proposed by Huang in 2006 [31]. Unlike the traditional single-hidden layer neural network, ELM can randomly initialize the input weight and bias and obtain the corresponding output weight. The calculation is relatively simple, the computational complexity is low, and it has been studied and used by many researchers. In this paper, an improved ELM algorithm based on the local coupled extreme learning machine (LC-ELM) [32] and particle swarm optimization (PSO) is used for gait recognition. We first introduce the basic ELM learning algorithm.For a single-hidden-layer neural network, suppose there are arbitrary samplesXi,ti, Xixi1,xi2,...,xipT∈Rp, ti=ti1,ti2,...,tiq∈Rq. A neural network with N neurons in a hidden layer can be expressed as follows:(7)Oj=∑i=1NβigWi⋅Xj+bi,i=1,...,N,j=1,2,...,M,where gx is the activation function, M is the number of the input samples, Wi is the input weight, βi is the output weight, bi is the bias of hidden layer neurons, and Wi⋅Xj is the inner product of Wi and Xj.The goal of ELM neural network learning is to minimize the output error, which can be expressed as follows:(8)∑j=1M||Oj−tj||=0.The aboveM equations can be written compactly as(9)Hβ=T,where β=β1,…,βNN×qT and T=t1,…,tNTM×q. H is called the hidden-layer output matrix of the ELM learning algorithm [33].The training process of ELM learning algorithm is equivalent to finding a least squares solutionβ^ of the linear system Hβ=T, and the above equation can be translated to(10)β^=H+T,where H+ is the Moore–Penrose generalized inverse of H.ELM algorithms are implemented by a fully coupled framework between the input layer and hidden layer. This fully coupled structure may incur higher computing costs when ELM has many hidden nodes and higher dimensions of input data.


## 3.2.2. Improved Extreme Learning Machine

To reduce the amount of computation, Qu [32] proposed LC-ELM, which was to decouple the input layer and the hidden layer based on ELM. Each hidden node is assigned a parameter in an input space [34]. Given a learning sample, the fuzzy membership function is used to measure the distance between the hidden node and the input sample as the activation degree of the hidden node. A fuzzy membership function and similarity relation are used to realize LC-ELM, which can reduce the complexity of the weight search space and improve the generalization performance. LC-ELM can be expressed as follows:(11)fXj=∑i=1NβigWi⋅Xj+biFSXj,di,i=1,...,N,j=1,2,...,M.In the LC-ELM learning algorithm, the similarity relationSX,di denotes the distance between the input X and the i-th hidden layer neurons, which is expressed as the address di. F⋅ is a fuzzy membership function, and there are many choices, such as a Gaussian, sigmoid, or reverse sigmoid function. The underlying radius parameter r is kept in F⋅ to adjust the width of the activation area, which is also an optimized parameter, to match the address parameter di. Combining the structure of LCFNN with the learning mechanism of ELM, LC-ELM also is a three-step learning algorithm whose network parameters (input weights W, biases b between the input layer and hidden layer, and address d of hidden neurons) are assigned randomly, which is the same as ELM [32].Compared with ELM, LC-ELM reduces the complexity of the algorithm structure, but its parameters are input randomly, and it may not be optimal, which will affect its performance. We propose a local coupled extreme learning machine based on particle swarm optimization [34] to improve the performance. This algorithm uses a particle swarm optimization strategy to optimize the four parameters W,b,d,r in LC-ELM, so as to improve its accuracy and generalization performance.Therefore, the particles in the searching space of the LC-PSO-ELM are composed of a set by the parameter values of input weights, hidden biases, address, and radius, which can be defined as(12)θ∈w11,w12,...,w1L,w21,w22,...,w2L,...,wp1,...,wpL,...,b1,b2,...,bN,d11,d12,...,d1L,d21,d22,...,d2L,...,dp1,dp2,...,dpL,...,r1,r2,...,rL,,where W=Wi|Wi∈Rp,i=1,...,L,b∈RL,d=di|di∈Rp,i=1,2,..,L,andr∈RL. When the optimal parameters of the local coupled extreme learning machine based on the particle swarm optimization algorithm are established based on formula (12), the weights between the hidden layer and the output layer of the algorithm are determined analytically based on formula (10). The flowchart of the LC-PSO-ELM algorithm is shown in Figure 2.Figure 2
Flow chart of local coupled extreme learning machine based on the particle swarm optimization.LC-PSO-ELM improves accuracy and generalization performance, and hence, we use it to recognize and classify gait features after dimension reduction.


## 4. Experiment and Analysis

To evaluate algorithm performance, we used the CASIA gait database [35] of the Center of Biometrics and Safety Technology, Institute of Automation, Chinese Academy of Sciences. We conducted three experiments, identified as A, B, and C, based on the CASIA-A and CASIA-B datasets. Experiment A examined the gait data of 20 subjects from the CASIA-A database. Each tester had 12 gait sequences with angles of 0°, 45°, and 90°. There was no angle discrimination. The gait data from all perspectives were examined together and were divided into training and testing sets at a 7 : 3 ratio. This small volume of experimental data was used to verify the superiority of the proposed algorithm. The experimental results are shown in Table 1.Table 1
Performance comparison of ELM and LC-PSO-ELM based on dimensional reduction methods of PCA or LDA on the CASIA-A dataset.
Algorithm
Training accuracy (%)
Testing accuracy (%)
ELM + PCA
99.76
93.10
LC-PSO-ELM + PCA
98.77
93.52
ELM + LDA
98.35
97.82
LC-PSO-ELM + LDA
98.67
98.24The normal sequences in CASIA-B were used in experiment B. The database contained 124 gait sequences. The data volume was large, and there were many cross angles (0°, 18°, ..., 162°, 180°). We did not consider cross view in this experiment. The results are shown in Table2.Table 2
Comparison of gait recognition results on the CASIA-A dataset.
Algorithm
Recognition rate (%)
Liu et al. [4]
97.38
Chai [36]
92.25
Proposed method
98.24The normal sequences in CASIA-B were also used in experiment C, which mainly considered the influence of cross view. The difference between the training and test sets was 18 degrees. The results are shown in Table3. The average results of six trials with different training and test sets are shown.Table 3
Performance comparison of different gait recognition methods on the CASIA-B dataset.
Algorithm
View
Average
0°
18°
36°
54°
72°
90°
108°
126°
144°
162°
180°
[37]
90.52
88.31
84.91
87.93
88.79
87.93
92.24
90.09
87.5
86.64
89.66
88.59
[38]
88.13
87.43
90.21
93.43
94.17
91.54
92.51
93.00
93.38
88.92
90.00
91.12
[39]
90.8
97.9
99.4
96.9
93.6
91.7
95
97.80
98.9
96.8
85.8
95
[40]
96.30
98.20
98.50
95.40
94.30
99.90
98.60
97
97.40
99.20
96.10
97.35
Ours
99.47
97.16
98.21
99.08
98.28
98.80
97.31
98.99
99.48
99.09
99.40
98.66


### 4.1. Experimental Details

First, we compare the performance of the ELM and LC-PSO-ELM learning algorithms based on the gait data, as shown in Table4. We selected the sigmoid function as the activation function for the two algorithms. The wave kernel Sx,y=θ/|x−y|sin|x−y|/θ was selected as the similarity function. The population size NP was set at 200, the maximum iteration time was 50, and the other control parameters are listed in Table 5.Table 4
Parameter configuration of ELM and LC-PSO-ELM learning algorithms.
Configurations
ELM
LC-PSO-ELM
Input weight and hidden layer biases
RN in [−1, 1]
NDRN (normally distributed random numbers)
Activation function
Sigmoid
Sigmoid
Hidden node address and window radius
—
NDRN
Similarity
—
Wave kernel
Fuzzy membership function
—
F
x
=
2
1
+
exp
x
/
rTable 5
Control parameters used in the LC-PSO-ELM learning algorithm.
Algorithm
w
max
w
min
c
max
c
min
c
1
c
2
LC-PSO-ELM
0.9
0.4
2.5
0.5
2
2Because the performance of ELM and LC-PSO-ELM is greatly influenced by hidden neurons, it is necessary to determine the number of these to ensure better classification performance. The experiment was performed by gradually increasing the number of hidden neurons, and the test results were recorded separately. The number of hidden neurons corresponding to the best classification result was identified as the best number for the algorithm. To show the superiority of the proposed method, we ran simulations with the dimension reduction and classification methods. The curve of the experimental results with the number of neurons in the hidden layer is shown in Figure3. For each curve, the red curve represents the training accuracy, and the blue curve is the test accuracy.Figure 3
Generalization performance curves for ELM and LC-PSO-ELM algorithms with different dimensional reduction methods. (a) ELM for PCA dimension reduction data. (b) ELM for LDA dimension reduction data. (c) LC-PSO-ELM for PCA dimension reduction data. (d) LC-PSO-ELM for LDA dimension reduction data.
(a)
(b)
(c)
(d)Figure3 including four subgraphs of Figures 3(a)–3(d) shows accuracy results of dimension reduction using the ELM algorithm with PCA and LDA, respectively. As is known, PCA is also a common dimensionality reduction method. Different from LDA, PCA is unsupervised. It neglects the classification information and only ensures that the internal information of data is maximized after dimension reduction. In the experiments, the input dimension of data based on the LDA method is 123, and the input dimension of data based on the PCA method is 240 in the classification method, respectively.Subgraphs of Figures3(c) and 3(d), respectively, represent the accuracy results of classification using LC-PSO-ELM after dimensionality reduction by the two methods. The accuracy increases with the number of neurons in the hidden layer and then tends to stabilize. The best number of hidden layer neurons differs by an algorithm. Fewer hidden layer neurons are needed for gait recognition from data after dimension reduction by LDA. Comparing Figure 3, the number of hidden layer neurons corresponding to the best classification results in subgraphs of Figures 3(b) and 3(d) is 9 and 6, respectively, which means that LDA is more conducive to dimension reduction classification. After using the same method for data dimensionality reduction, LC-PSO-ELM needs fewer hidden neurons to obtain the best classification results. In addition, it is obvious that in subgraphs of Figures 3(c) and 3(d), fewer hidden neurons are needed to obtain the best classification results, about 145 and 6, respectively.Compared with the traditional ELM algorithm, LC-PSO-ELM can simplify the algorithm structure, optimize the parameters of the neural networks, reduce the influence of random input parameters, and improve classification performance. The subgraph of Figure3(d) shows the best classification performance being obtained most quickly, and the combination of LDA dimension reduction and LC-PSO-ELM classification can obtain higher recognition accuracy, which has certain advantages.Therefore, as shown in Table6 in the experiments of simulations, in pursuit of better generalization performance of different algorithms, the hidden neurons of ELM learning algorithm based on dimensional reduction methods of PCA or LDA are 275 and 10 in CASIA-A dataset, respectively, While the hidden neurons of the LC-PSO-ELM learning algorithm are 140 and 8, respectively. In the CASIA-B dataset with and without considering influence of cross view, the hidden neurons of the LC-PSO-ELM learning algorithm based on LDA are between 8 and 25 for increasing the generalization performance of different cross angles (0°, 18°, ..., 162°, 180°).


### 4.2. Experimental Results

Performance comparison between the ELM learning algorithm and the LC-PSO-ELM learning algorithm based on different dimensional reduction methods of PCA or LDA on the CASIA-A dataset is illustrated in Table1. The recognition accuracies of ELM and LC-PSO-ELM based on LDA are 97.82% and 98.24%, while the algorithms based on PCA are 93.10% and 93.52%, respectively, which shows that the LDA for data dimensionality reduction is more conducive to classification.Compared with ELM, the LC-PSO-ELM algorithm obtains higher recognition accuracy, demonstrating that LC-PSO-ELM has better classification performance with a compact network configuration of fewer hidden neurons.In Table2, the proposed method of LC-PSO-ELM based on the LDA dimensional reduction method and two other algorithms of the references of [4, 36] are compared based on the recognition accuracy of testing process on the CASIA-A datasets. The proposed algorithm structure achieves the highest recognition accuracy of 98.24%, which is, respectively, 5.9% and 0.86% greater than the existing algorithms of references of [4, 36], with 92.25% and 97.38%. The proposed algorithm structure has a certain effectiveness and superiority for gait recognition.To demonstrate the superior performance of the proposed algorithm, Table3 compares it to state-of-the-art methods based on gait recognition accuracy on the CASIA-B datasets. The algorithm achieves almost all the best classification accuracy results. Among the compared methods, Binsaadoon and El-Alfy [37] used the FLGBP method, with 88.59% average accuracy of gait recognition in 11 views. Zhang et al. [38] used the KPCA-LPP method and achieved 91.12% average gait recognition accuracy. Chao et al. [39] achieved 95% average gait recognition accuracy using the method of regarding the gait as a set. Wolf et al. [40] used the 3DCNN method with average gait recognition accuracy of 97.35%. The proposed algorithm improved the average gait recognition accuracy by 10.07%, 7.54%, 3.66%, and 1.31%, respectively, over the studies of Binsaadoon, Zhang, Chao, and Wolf.Table 6
Number of hidden neurons of different algorithms on CASIA problem.
Dataset algorithms
CASIA-A
CASIA-B
CASIA-B considering influence of cross view
ELM + PCA
275
LC-PSO-ELM + PCA
140
ELM + LDA
10
LC-PSO-ELM + LDA
8
8–25
8–25In recent years, Yu et al. [41] proposed a gait recognition framework that is more sensitive to the perspective drawing of a walking gait, which caused the recognition accuracy to differ greatly, with an average recognition rate of 38.53%. Wang et al. [42] proposed a method based on multiview gait sequence fusion, with an average gait recognition accuracy of 88.75%. Wang et al. [43] used the TS-GAN method and achieved 88.1% average gait recognition accuracy. The results of the CASIA-B dataset with considering influence of cross view are shown in Table 7 based on the proposed method and the above gait recognition approaches.Table 7
Performance comparison of different gait recognition methods on the CASIA-B dataset with considering influence of cross view.
Algorithm
View
Training testing
Average
0°
18°
36°
54°
72°
90°
108°
126°
144°
162°
18°
36°
54°
72°
90°
108°
126°
144°
162°
180°
[41]
31.9
39.9
28.8
23.0
81.5
82.3
37.1
26.6
5.6
28.6
—
38.53
[42]
84.69
87.84
88.66
91.92
91.95
90.04
88.20
88.50
89.56
86.25
—
88.75
[43]
86
95
89
88
88
93
89
96
84
73
—
88.1
Ours
95.18
97.81
96.44
95.23
94.60
95.79
95.82
95.48
95.81
95.83
—
95.80Compared with the above methods, the proposed algorithm improved the average gait recognition accuracy by 57.27%, 7.05%, and 7.7%, respectively. The highest recognition accuracy was 99.48%, the lowest was 97.16%, and hence, there is little difference between the highest and lowest accuracy. The proposed algorithm attained almost all of the best classification accuracy figures from different views.In conclusion, in this paper, a gait recognition method based on LC-PSO-ELM and the LDA dimensional reduction method is proposed and the performance is compared with some gait recognition methods on the CASIA dataset with and without considering influence of cross view. Table2 shows results under the same view, while Tables 3 and 7 are under different views. The proposed algorithm has good recognition accuracy of gait and can be effectively applied to gait recognition. In particular, Table 7 shows that cross-view gait recognition based on the proposed method also has high accuracy. The proposed algorithm can effectively avoid the impact of different views on gait recognition.


## 4.1. Experimental Details

First, we compare the performance of the ELM and LC-PSO-ELM learning algorithms based on the gait data, as shown in Table4. We selected the sigmoid function as the activation function for the two algorithms. The wave kernel Sx,y=θ/|x−y|sin|x−y|/θ was selected as the similarity function. The population size NP was set at 200, the maximum iteration time was 50, and the other control parameters are listed in Table 5.Table 4
Parameter configuration of ELM and LC-PSO-ELM learning algorithms.
Configurations
ELM
LC-PSO-ELM
Input weight and hidden layer biases
RN in [−1, 1]
NDRN (normally distributed random numbers)
Activation function
Sigmoid
Sigmoid
Hidden node address and window radius
—
NDRN
Similarity
—
Wave kernel
Fuzzy membership function
—
F
x
=
2
1
+
exp
x
/
rTable 5
Control parameters used in the LC-PSO-ELM learning algorithm.
Algorithm
w
max
w
min
c
max
c
min
c
1
c
2
LC-PSO-ELM
0.9
0.4
2.5
0.5
2
2Because the performance of ELM and LC-PSO-ELM is greatly influenced by hidden neurons, it is necessary to determine the number of these to ensure better classification performance. The experiment was performed by gradually increasing the number of hidden neurons, and the test results were recorded separately. The number of hidden neurons corresponding to the best classification result was identified as the best number for the algorithm. To show the superiority of the proposed method, we ran simulations with the dimension reduction and classification methods. The curve of the experimental results with the number of neurons in the hidden layer is shown in Figure3. For each curve, the red curve represents the training accuracy, and the blue curve is the test accuracy.Figure 3
Generalization performance curves for ELM and LC-PSO-ELM algorithms with different dimensional reduction methods. (a) ELM for PCA dimension reduction data. (b) ELM for LDA dimension reduction data. (c) LC-PSO-ELM for PCA dimension reduction data. (d) LC-PSO-ELM for LDA dimension reduction data.
(a)
(b)
(c)
(d)Figure3 including four subgraphs of Figures 3(a)–3(d) shows accuracy results of dimension reduction using the ELM algorithm with PCA and LDA, respectively. As is known, PCA is also a common dimensionality reduction method. Different from LDA, PCA is unsupervised. It neglects the classification information and only ensures that the internal information of data is maximized after dimension reduction. In the experiments, the input dimension of data based on the LDA method is 123, and the input dimension of data based on the PCA method is 240 in the classification method, respectively.Subgraphs of Figures3(c) and 3(d), respectively, represent the accuracy results of classification using LC-PSO-ELM after dimensionality reduction by the two methods. The accuracy increases with the number of neurons in the hidden layer and then tends to stabilize. The best number of hidden layer neurons differs by an algorithm. Fewer hidden layer neurons are needed for gait recognition from data after dimension reduction by LDA. Comparing Figure 3, the number of hidden layer neurons corresponding to the best classification results in subgraphs of Figures 3(b) and 3(d) is 9 and 6, respectively, which means that LDA is more conducive to dimension reduction classification. After using the same method for data dimensionality reduction, LC-PSO-ELM needs fewer hidden neurons to obtain the best classification results. In addition, it is obvious that in subgraphs of Figures 3(c) and 3(d), fewer hidden neurons are needed to obtain the best classification results, about 145 and 6, respectively.Compared with the traditional ELM algorithm, LC-PSO-ELM can simplify the algorithm structure, optimize the parameters of the neural networks, reduce the influence of random input parameters, and improve classification performance. The subgraph of Figure3(d) shows the best classification performance being obtained most quickly, and the combination of LDA dimension reduction and LC-PSO-ELM classification can obtain higher recognition accuracy, which has certain advantages.Therefore, as shown in Table6 in the experiments of simulations, in pursuit of better generalization performance of different algorithms, the hidden neurons of ELM learning algorithm based on dimensional reduction methods of PCA or LDA are 275 and 10 in CASIA-A dataset, respectively, While the hidden neurons of the LC-PSO-ELM learning algorithm are 140 and 8, respectively. In the CASIA-B dataset with and without considering influence of cross view, the hidden neurons of the LC-PSO-ELM learning algorithm based on LDA are between 8 and 25 for increasing the generalization performance of different cross angles (0°, 18°, ..., 162°, 180°).


## 4.2. Experimental Results

Performance comparison between the ELM learning algorithm and the LC-PSO-ELM learning algorithm based on different dimensional reduction methods of PCA or LDA on the CASIA-A dataset is illustrated in Table1. The recognition accuracies of ELM and LC-PSO-ELM based on LDA are 97.82% and 98.24%, while the algorithms based on PCA are 93.10% and 93.52%, respectively, which shows that the LDA for data dimensionality reduction is more conducive to classification.Compared with ELM, the LC-PSO-ELM algorithm obtains higher recognition accuracy, demonstrating that LC-PSO-ELM has better classification performance with a compact network configuration of fewer hidden neurons.In Table2, the proposed method of LC-PSO-ELM based on the LDA dimensional reduction method and two other algorithms of the references of [4, 36] are compared based on the recognition accuracy of testing process on the CASIA-A datasets. The proposed algorithm structure achieves the highest recognition accuracy of 98.24%, which is, respectively, 5.9% and 0.86% greater than the existing algorithms of references of [4, 36], with 92.25% and 97.38%. The proposed algorithm structure has a certain effectiveness and superiority for gait recognition.To demonstrate the superior performance of the proposed algorithm, Table3 compares it to state-of-the-art methods based on gait recognition accuracy on the CASIA-B datasets. The algorithm achieves almost all the best classification accuracy results. Among the compared methods, Binsaadoon and El-Alfy [37] used the FLGBP method, with 88.59% average accuracy of gait recognition in 11 views. Zhang et al. [38] used the KPCA-LPP method and achieved 91.12% average gait recognition accuracy. Chao et al. [39] achieved 95% average gait recognition accuracy using the method of regarding the gait as a set. Wolf et al. [40] used the 3DCNN method with average gait recognition accuracy of 97.35%. The proposed algorithm improved the average gait recognition accuracy by 10.07%, 7.54%, 3.66%, and 1.31%, respectively, over the studies of Binsaadoon, Zhang, Chao, and Wolf.Table 6
Number of hidden neurons of different algorithms on CASIA problem.
Dataset algorithms
CASIA-A
CASIA-B
CASIA-B considering influence of cross view
ELM + PCA
275
LC-PSO-ELM + PCA
140
ELM + LDA
10
LC-PSO-ELM + LDA
8
8–25
8–25In recent years, Yu et al. [41] proposed a gait recognition framework that is more sensitive to the perspective drawing of a walking gait, which caused the recognition accuracy to differ greatly, with an average recognition rate of 38.53%. Wang et al. [42] proposed a method based on multiview gait sequence fusion, with an average gait recognition accuracy of 88.75%. Wang et al. [43] used the TS-GAN method and achieved 88.1% average gait recognition accuracy. The results of the CASIA-B dataset with considering influence of cross view are shown in Table 7 based on the proposed method and the above gait recognition approaches.Table 7
Performance comparison of different gait recognition methods on the CASIA-B dataset with considering influence of cross view.
Algorithm
View
Training testing
Average
0°
18°
36°
54°
72°
90°
108°
126°
144°
162°
18°
36°
54°
72°
90°
108°
126°
144°
162°
180°
[41]
31.9
39.9
28.8
23.0
81.5
82.3
37.1
26.6
5.6
28.6
—
38.53
[42]
84.69
87.84
88.66
91.92
91.95
90.04
88.20
88.50
89.56
86.25
—
88.75
[43]
86
95
89
88
88
93
89
96
84
73
—
88.1
Ours
95.18
97.81
96.44
95.23
94.60
95.79
95.82
95.48
95.81
95.83
—
95.80Compared with the above methods, the proposed algorithm improved the average gait recognition accuracy by 57.27%, 7.05%, and 7.7%, respectively. The highest recognition accuracy was 99.48%, the lowest was 97.16%, and hence, there is little difference between the highest and lowest accuracy. The proposed algorithm attained almost all of the best classification accuracy figures from different views.In conclusion, in this paper, a gait recognition method based on LC-PSO-ELM and the LDA dimensional reduction method is proposed and the performance is compared with some gait recognition methods on the CASIA dataset with and without considering influence of cross view. Table2 shows results under the same view, while Tables 3 and 7 are under different views. The proposed algorithm has good recognition accuracy of gait and can be effectively applied to gait recognition. In particular, Table 7 shows that cross-view gait recognition based on the proposed method also has high accuracy. The proposed algorithm can effectively avoid the impact of different views on gait recognition.


## 5. Conclusions

As a promising biometric technology, gait recognition has attracted wide attention. We used a Gabor filter to extract multidirectional and multiscale features from gait energy images. Linear discriminant analysis was used to reduce the dimensionality of feature data. The improved extreme learning machine algorithm was used for recognition and classification. We conducted experiments on the CASIA datasets, and the results demonstrate the effectiveness of the proposed method. The algorithm has low complexity and good generalization performance. However, this study does not consider the influence of covariate factors, such as clothes and bags, as well as that of cross view, where the angle difference is more obvious. Hence, subsequent research will focus on recognition and classification under cross view and covariate factors.



---

*Source: 5393058-2020-04-25.xml*