# Gait Recognition Based on Invariant Leg Classification Using a Neuro-Fuzzy Algorithm as the Fusion Method

Authors: Hadi Sadoghi Yazdi; Hessam Jahani Fariman; Jaber Roohi
Journal: ISRN Artificial Intelligence
 (2012)

---

## Abstract

This paper presents a human gait recognition algorithm based on a leg gesture separation. Main innovation in this paper is gait recognition using leg gesture classification which is invariant to covariate conditions during walking sequence and just focuses on underbody motions and a neuro-fuzzy combiner classifier (NFCC) which derives a high precision recognition system. At the end, performance of the proposed algorithm has been validated by using the HumanID Gait Challenge data set (HGCD), the largest gait benchmarking data set with 122 objects with different realistic parameters including viewpoint, shoe, surface, carrying condition, and time. And it has been compared to recent algorithm of gait recognition.


---

## Body

## 1. Introduction

In the last decade, there have been great interests in applying human biometrics for identification and verification purposes, for instance, in video surveillance and human recognition areas. Amongst there have been lots of researches in using ear and face recognition, body tracking and hand gesture recognition, and recently gait recognition using in the human identification areas. But as a comparison between human gait and other various biometrics, such as hand geometry, iris, face, voice, signature, and fingerprint [1], the human gait has some eligible advantages over them that make the gait recognition an ideal method in identification procedures. For instance, there is no need to subject cooperation in gait recognition, and it can operate without interrupting or interfacing with the subject activities [2]. In other words, we can recognize people using human gait regardless of their clothes or the backgrounds [3]. Further it is difficult to conceal or disguise in application scenarios like bank robbery that other biometrics such as face recognition or fingerprint are impossible in detection. Moreover, it is nonobstructive and effective for identifying at long distances like surveillance applications in public places [4].The previous works have been classified under similar covariate conditions (e.g., clothing, surface, carrying, etc.). But in this paper we proposed an improved and also novel method of classification which is only based on different gestures of leg during walking without body parts tracking and invariant to different covariate conditions.As it is indicated in Figure1, with a more careful focus on the sequences of sample energy halation images, we can obviously conclude that because of negligible changes of bust’s organs during walking cadence and little effect of added objects (e.g., carrying a bag, wearing a coat, etc.) on the gait, for obtaining higher recognition rate we can only focus on leg gesture for gait recognition and derive accurately from its classification.(a) A sequence of energy halation images. (b) Underbody changes during walking. (c) Bust (upper body) changes during walking.
(a)(b)(c)As a review to fundamental of the usual gait recognition algorithm, we can express that in the walking process functional versatility of the body joints allows the lower and upper limbs to readily accommodate stairs, doorways, changing surfaces, and obstacles in the path of progression. Efficiency in these endeavors depends upon free joint mobility and muscle activity that is selective with timing and intensity. Energy conservation is optimal in the normal pattern of limb action. A person will perform one’s walking pattern in a fairly repeatable and unique way, and medical research has been trying to apply these gait patterns for the treatment of pathologically abnormal patients [4].As a brief introduction of the approach of this paper we can express the following procedures.Five states of human gait are extracted after background estimation and human detection in the scene. Leg gestures are classified over directional chain code of bottom part of silhouette contour. A spatiot-emporal data base, namely, Energy Halation Image (EHI), is constructed over bottom part of human silhouette from train film sequence for five leg gestures separately. Eigen space of energy halation is applied to multilayer perceptron neural network. Five neural network systems recognize people but with medium recognition rate. A Neuro-fuzzy fusion technique is used for obtaining high recognition rate. Experimental results are performed over a suitable data base. It includes 20 samples for eight people which each sample have 100 frames approximately. 99% recognition rate of the proposed system is obtained over 10 samples test patterns.


### 1.1. Recent Works

Leg gesture studies have various applications. Among this, some interest work indicates importance of leg gesture classification as in [5–7]. In [8], matching between stored prototypes and silhouette images helps for state classification. View point of this paper [8] is based on pattern matching and recognition of state using hidden Markov model; it helps to insert the prior knowledge of gait in state recognition.The infrared thermal imaging was applied to collect gait video, and an infrared thermal gait database was established in [9]. Infrared is useful to detect human body and remove noises from complex background; illumination variations in [10] show that using Principle Component Analysis (PCA) on accelerometer-based gait data gives a large improvement in the performance of gait recognition system.Reference [2] argues that selecting the most relevant gait features that are invariant to changes in gait covariate conditions is the key to develop a gait recognition system that works without subject cooperation. So [2] proposes Gait Entropy Image to perform automatic feature selection on each pair of gallery and probe gait sequences. The performance of gait recognition decreases because of low-resolution (LR) sequences. Reference [11] proposes method for solving this solution. They proposed a new algorithm called superresolution with manifold sampling and backprojection, which learns the high-resolution (HR) counterparts of LR test images from a collection of HR/LR training gait image patch pairs.Reference [12] presents a novel framework for gait recognition augmented with soft biometric information. Geometric gait analysis is based on Radon transforms and on gait energy images. User height and stride length information is extracted and utilized in a probabilistic framework for the detection of soft biometric features of substantial discrimination power.In [13] multiple gait features fusion was explored with the framework of the factorial hidden Markov model (FHMM). The FHMM has a multiple-layer structure and provides an alternative to combine several gait features without concatenating them into a single augmented feature. Besides, the feature concatenation was used to directly concatenate the features and the parallel HMM (PHMM) was introduced as a decision-level fusion scheme, which employs traditional fusion rules to combine the recognition results at decision level.Tactile ground surface indicators installed on sidewalks help visually impaired people walk safely. The visually impaired distinguish the indicators by stepping into its convexities and following them. However, these indicators sometimes cause the nonvisually impaired to stumble. In [5] have been studied effects of these indicators by comparing the kinematics and kinetic variables of walking on paths with and without indicators.Another interest for gait identification is that of reflect gait degeneration due to ageing that might have closer linkage to the causes of falls. This would help to undertake appropriate measures to prevent falls. Like in many other developed countries, falls in older population have been identified as a major health issue in Australia [6]. In [7] automatic recognition of young-old gait types from their respective gait patterns has been studied using support vector machine. Ageing influences gait patterns causing constant threats to locomotor balance control.Biomechanical analysis of gait has been successfully applied in human clinical gait analysis [14]. With regards to gait recognition, a major early result from psychology is by Johansson [15], who used point light displays to demonstrate the ability of humans to rapidly distinguish human locomotion from other motion patterns. Cutting and Kozlowski [16] showed that this ability also extends to recognition of friends.Identification of people by analysis of gait patterns extracted from video has recently become a popular research problem. However, the conditions under which the problem is “solvable” are not understood or characterized as in [17]. The biggest limitation in human motion analysis is the underlying difficulty of tracking the human body for subsequent interpretation [18, 19].As a solution for making it possible to identify human gait from a sequence of segmented noisy silhouettes in low-resolution video, a model-based gait cycle extraction based on the prediction-based hierarchical active shape model (ASM) is presented in [1]. Moreover in [20] there is a presentation of a new gait recognition method that does not presume the existence of strict lab conditions for its operation.As it mentioned, the gait recognition is an effective way for identifying from a distance but there are two different obstacles in this situation. First in the low-resolution case the performance of gait recognition is abated because of noisy images. Furthermore, as a usual procedure of gait recognition the gait sequences are projected onto a nonoptimal low-dimensional subspace to reduce the data complexity which again would lead to decline of gait recognition performance. A new algorithm is proposed in [11] called super resolution with manifold sampling and back projection (SRMS), which learns the high-resolution (HR) counterparts of LR test images from a collection of HR/LR training gait image patch pairs.


### 1.2. Contributions and Motivation

Recognizing gait with body decomposition to details and fusion of them were not observed in the literature. Main contribution of this paper is gesture classification for human gait recognition. But some new notes can be found in this paper as follows.(a)
A new spatio-temporal data base, namely, energy halation.(b)
Five-feature space generation using leg gesture concept.(c)
Human gait recognition based on leg gesture classification.(d)
Neuro-fuzzy-based combiner classifiers (NFCCs).(e)
Presentation of complete system in gait recognition.Low performance in human gait recognition systems is one of motivations of the proposed method. Human detection in the scene, object tracking, and classifiers capability over time-dependent features are some of problems in obtaining low recognition rate. So, we try to present a complete system in human gait recognition which includes many features.


## 1.1. Recent Works

Leg gesture studies have various applications. Among this, some interest work indicates importance of leg gesture classification as in [5–7]. In [8], matching between stored prototypes and silhouette images helps for state classification. View point of this paper [8] is based on pattern matching and recognition of state using hidden Markov model; it helps to insert the prior knowledge of gait in state recognition.The infrared thermal imaging was applied to collect gait video, and an infrared thermal gait database was established in [9]. Infrared is useful to detect human body and remove noises from complex background; illumination variations in [10] show that using Principle Component Analysis (PCA) on accelerometer-based gait data gives a large improvement in the performance of gait recognition system.Reference [2] argues that selecting the most relevant gait features that are invariant to changes in gait covariate conditions is the key to develop a gait recognition system that works without subject cooperation. So [2] proposes Gait Entropy Image to perform automatic feature selection on each pair of gallery and probe gait sequences. The performance of gait recognition decreases because of low-resolution (LR) sequences. Reference [11] proposes method for solving this solution. They proposed a new algorithm called superresolution with manifold sampling and backprojection, which learns the high-resolution (HR) counterparts of LR test images from a collection of HR/LR training gait image patch pairs.Reference [12] presents a novel framework for gait recognition augmented with soft biometric information. Geometric gait analysis is based on Radon transforms and on gait energy images. User height and stride length information is extracted and utilized in a probabilistic framework for the detection of soft biometric features of substantial discrimination power.In [13] multiple gait features fusion was explored with the framework of the factorial hidden Markov model (FHMM). The FHMM has a multiple-layer structure and provides an alternative to combine several gait features without concatenating them into a single augmented feature. Besides, the feature concatenation was used to directly concatenate the features and the parallel HMM (PHMM) was introduced as a decision-level fusion scheme, which employs traditional fusion rules to combine the recognition results at decision level.Tactile ground surface indicators installed on sidewalks help visually impaired people walk safely. The visually impaired distinguish the indicators by stepping into its convexities and following them. However, these indicators sometimes cause the nonvisually impaired to stumble. In [5] have been studied effects of these indicators by comparing the kinematics and kinetic variables of walking on paths with and without indicators.Another interest for gait identification is that of reflect gait degeneration due to ageing that might have closer linkage to the causes of falls. This would help to undertake appropriate measures to prevent falls. Like in many other developed countries, falls in older population have been identified as a major health issue in Australia [6]. In [7] automatic recognition of young-old gait types from their respective gait patterns has been studied using support vector machine. Ageing influences gait patterns causing constant threats to locomotor balance control.Biomechanical analysis of gait has been successfully applied in human clinical gait analysis [14]. With regards to gait recognition, a major early result from psychology is by Johansson [15], who used point light displays to demonstrate the ability of humans to rapidly distinguish human locomotion from other motion patterns. Cutting and Kozlowski [16] showed that this ability also extends to recognition of friends.Identification of people by analysis of gait patterns extracted from video has recently become a popular research problem. However, the conditions under which the problem is “solvable” are not understood or characterized as in [17]. The biggest limitation in human motion analysis is the underlying difficulty of tracking the human body for subsequent interpretation [18, 19].As a solution for making it possible to identify human gait from a sequence of segmented noisy silhouettes in low-resolution video, a model-based gait cycle extraction based on the prediction-based hierarchical active shape model (ASM) is presented in [1]. Moreover in [20] there is a presentation of a new gait recognition method that does not presume the existence of strict lab conditions for its operation.As it mentioned, the gait recognition is an effective way for identifying from a distance but there are two different obstacles in this situation. First in the low-resolution case the performance of gait recognition is abated because of noisy images. Furthermore, as a usual procedure of gait recognition the gait sequences are projected onto a nonoptimal low-dimensional subspace to reduce the data complexity which again would lead to decline of gait recognition performance. A new algorithm is proposed in [11] called super resolution with manifold sampling and back projection (SRMS), which learns the high-resolution (HR) counterparts of LR test images from a collection of HR/LR training gait image patch pairs.


## 1.2. Contributions and Motivation

Recognizing gait with body decomposition to details and fusion of them were not observed in the literature. Main contribution of this paper is gesture classification for human gait recognition. But some new notes can be found in this paper as follows.(a)
A new spatio-temporal data base, namely, energy halation.(b)
Five-feature space generation using leg gesture concept.(c)
Human gait recognition based on leg gesture classification.(d)
Neuro-fuzzy-based combiner classifiers (NFCCs).(e)
Presentation of complete system in gait recognition.Low performance in human gait recognition systems is one of motivations of the proposed method. Human detection in the scene, object tracking, and classifiers capability over time-dependent features are some of problems in obtaining low recognition rate. So, we try to present a complete system in human gait recognition which includes many features.


## 2. The Proposed Method

Block diagram of the proposed method can be abstracted in Figure2. Five parts of this system are as follows and are explained in the next subsections.(i)
Background estimation,(ii)
leg gesture recognizer,(iii)
energy halation image construction (spatio-temporal data base),(iv)
gait recognition in Eigen space,(v)
neuro-fuzzy-based combiner classifier.Figure 2
Block diagram of proposed gait recognition method.


### 2.1. Background Estimation

Several approaches are known to separate foreground from background. If the background is known a simple thresholding yields to the foreground. One suitable way in object detection is background estimation. This paper uses probability density function (PDF) estimation of each pixel [21]. Gaussian PDF can model variation of scene because of flicker, CCD noise, and shadow approximately. For obtaining mean and variance of Gaussian PDF, (1) and (2) are used which can accept scene variations. Results of human detection in the scene are shown in Figure 3:(1)μt(x,y)=(1-α)μt-1(x,y)+αIt(x,y),(2)σt2(x,y)=(1-α)σt-12(x,y)+α(It(x,y)-μt(x,y))T(It(x,y)-μt(x,y)),
where It(x,y), is the pixel’s current value in location (x,y) and μt-1 the previous average, σt-1 the previous variance; T is transpose; α is an empirical weight often chosen as a tradeoff between stability and quick update. At each t frame time, the I, pixel’s value can then be classified as a foreground pixel if the inequality
(3)|It-μt|>kδt
holds, where k, is threshold value.Figure 3
Human detection in the scenes using Gaussian PDF model.


### 2.2. Leg Gesture Recognizer

After background estimation and human detection in the scene, binary human image (blob) is obtained. After cutting a bottom of blob image (waist to sole), distribution function of directional chain code is extracted from blob contour. After normalizing the chain code to its maximum, a multilayer perceptron neural network (MLP-NN) is used for leg gesture recognizing with this feature. Block diagram of leg gesture classifier is shown in Figure3. For training the proposed artificial neural network, we have five states of five people (five images for each person). So after creating images’ database, we named them as the following sequence.(i)
First digit denotes the state of person (1-2-3-4-5).(ii)
Second digit denotes the person (1-2-3-4-5).(iii)
Third digit denotes the number of the image of each person (1-2-3-4-5).Therefore we now have 125 named images in the database for training. Moreover, we considered five different angles in the video sequence of samples for each state like the one in Figure4. Then by considering 5 different angle states (Figure 5), in our program, we would have the angles of every one of the five gait states which is shown in Figure 6.Figure 4
Different states of every sample for the NN trainer.Figure 5
Five angles of each gait state.Figure 6
A sample of recognizing 5 different states of gait using NN recognizer.One of leg gesture classifier parts is gesture data base which is necessary for training of MLP-NN using backpropagation algorithm. Five states are determined for leg gesture which depends on frame rate and type of application. Figure4 shows these five states for number of people. Gesture data base is collected from a set of film which includes 160 sequences of eight people. Obtained manually gesture data base includes five leg states, and for each state 100 images have been collected. Extracted distribution directional chain code is shown in Figures 7(a) and 7(b) shows directional chain codes histogram for difference state.Normalized histogram for state 1 and 2.
(a)(b)However, trained neural network cannot classify leg gestures perfectly but this problem compensates in creation of spatio-temporal data base and using classifier.


### 2.3. Energy Halation Image Construction (Spatio-temporal Data Base)

Spatio-temporal data base use for compact presentation of film sequence and use in many applications as image retrieval, gesture analysis, action recognition, and behavioral recognition in the scene.In this sub-section we propose a spatio-temporal like motion history image (MHI) in [22] which pseudo code is as follow and results is named energy halation images (EHI).Each input frame belong to one of five leg gestures and is used for generation of five energy halation images.(1)
Initializing:LetEHi,i=1,2,…,5, beforced to zeros with dimension 220 × 90.Letj=0; j is frame’s index.(2)
j=j+1.(3)
Ii⟵blob matrix of jth frame with size x×y; i is state of leg (1 to 5).Note:(x,y) is less than (220, 90) for each blob size.(4)
Adding zero rows and columns bilateral ofI(x,y) that become I(220,90) matrix;(5)
EHi=EHi+Ii; i is state of leg gesture.(6)
If it is not end of sequence go to step 2.(7)
End.Obtained results include five images of energy halation for each input sequence. As an example, Figure8 shows five images of energy halation for three people.Figure 8
Five images of energy halation (columns) for three people (rows).


### 2.4. Gait Recognition in Eigen Space

As face recognition and similar applications, we use Eigen space transform for reducing the dimensions of the energy halation images before applying to MLP-neural network. Training MLP-NN is performed over each leg gesture for human gait recognition. So five trained MLP-NNs are created and use for human identification but each network recognized people separately based on different features (these features are energy halation over each leg gesture).Recognition rate of each network does not satisfy the using system as good human gait recognizer so we combine neural networks output using neuro-fuzzy-based mixer classifiers which is followed in the next sub-section.


### 2.5. Neuro-Fuzzy-Based Combiner Classifier

Neuro-fuzzy system has been proved to have significant results in modeling nonlinear functions. Neuro-fuzzy system has been used frequently in the literature as fishing predictions [23], vehicular navigation [24], identifying the turbine speed dynamics [25], radio frequency power amplifier linearization [26], microwave application [27], image denoising [28, 29], prediction in cleaning with high pressure water [30], sensor calibration [31], fetal electrocardiogram extraction from ECG signal captured from mother [32], and identification of normal and glaucomatous eyes [33].In a neuro-fuzzy system, the membership functions (MFs) are extracted from a data set that describes the system behavior. The neuro-fuzzy system learns features in the data set and adjusts the system parameters according to given error criterion. In a fused architecture, NN learning algorithms are used to determine the parameters of fuzzy inference system. Below, we have summarized the advantages of the neuro-fuzzy system technique. Fusion of output classifiers with linear combiner has been pointed in [34]. In this paper, we used a nonlinear mixer classifier which is based on neuro-fuzzy system for the first time in human gait recognition.


## 2.1. Background Estimation

Several approaches are known to separate foreground from background. If the background is known a simple thresholding yields to the foreground. One suitable way in object detection is background estimation. This paper uses probability density function (PDF) estimation of each pixel [21]. Gaussian PDF can model variation of scene because of flicker, CCD noise, and shadow approximately. For obtaining mean and variance of Gaussian PDF, (1) and (2) are used which can accept scene variations. Results of human detection in the scene are shown in Figure 3:(1)μt(x,y)=(1-α)μt-1(x,y)+αIt(x,y),(2)σt2(x,y)=(1-α)σt-12(x,y)+α(It(x,y)-μt(x,y))T(It(x,y)-μt(x,y)),
where It(x,y), is the pixel’s current value in location (x,y) and μt-1 the previous average, σt-1 the previous variance; T is transpose; α is an empirical weight often chosen as a tradeoff between stability and quick update. At each t frame time, the I, pixel’s value can then be classified as a foreground pixel if the inequality
(3)|It-μt|>kδt
holds, where k, is threshold value.Figure 3
Human detection in the scenes using Gaussian PDF model.


## 2.2. Leg Gesture Recognizer

After background estimation and human detection in the scene, binary human image (blob) is obtained. After cutting a bottom of blob image (waist to sole), distribution function of directional chain code is extracted from blob contour. After normalizing the chain code to its maximum, a multilayer perceptron neural network (MLP-NN) is used for leg gesture recognizing with this feature. Block diagram of leg gesture classifier is shown in Figure3. For training the proposed artificial neural network, we have five states of five people (five images for each person). So after creating images’ database, we named them as the following sequence.(i)
First digit denotes the state of person (1-2-3-4-5).(ii)
Second digit denotes the person (1-2-3-4-5).(iii)
Third digit denotes the number of the image of each person (1-2-3-4-5).Therefore we now have 125 named images in the database for training. Moreover, we considered five different angles in the video sequence of samples for each state like the one in Figure4. Then by considering 5 different angle states (Figure 5), in our program, we would have the angles of every one of the five gait states which is shown in Figure 6.Figure 4
Different states of every sample for the NN trainer.Figure 5
Five angles of each gait state.Figure 6
A sample of recognizing 5 different states of gait using NN recognizer.One of leg gesture classifier parts is gesture data base which is necessary for training of MLP-NN using backpropagation algorithm. Five states are determined for leg gesture which depends on frame rate and type of application. Figure4 shows these five states for number of people. Gesture data base is collected from a set of film which includes 160 sequences of eight people. Obtained manually gesture data base includes five leg states, and for each state 100 images have been collected. Extracted distribution directional chain code is shown in Figures 7(a) and 7(b) shows directional chain codes histogram for difference state.Normalized histogram for state 1 and 2.
(a)(b)However, trained neural network cannot classify leg gestures perfectly but this problem compensates in creation of spatio-temporal data base and using classifier.


## 2.3. Energy Halation Image Construction (Spatio-temporal Data Base)

Spatio-temporal data base use for compact presentation of film sequence and use in many applications as image retrieval, gesture analysis, action recognition, and behavioral recognition in the scene.In this sub-section we propose a spatio-temporal like motion history image (MHI) in [22] which pseudo code is as follow and results is named energy halation images (EHI).Each input frame belong to one of five leg gestures and is used for generation of five energy halation images.(1)
Initializing:LetEHi,i=1,2,…,5, beforced to zeros with dimension 220 × 90.Letj=0; j is frame’s index.(2)
j=j+1.(3)
Ii⟵blob matrix of jth frame with size x×y; i is state of leg (1 to 5).Note:(x,y) is less than (220, 90) for each blob size.(4)
Adding zero rows and columns bilateral ofI(x,y) that become I(220,90) matrix;(5)
EHi=EHi+Ii; i is state of leg gesture.(6)
If it is not end of sequence go to step 2.(7)
End.Obtained results include five images of energy halation for each input sequence. As an example, Figure8 shows five images of energy halation for three people.Figure 8
Five images of energy halation (columns) for three people (rows).


## 2.4. Gait Recognition in Eigen Space

As face recognition and similar applications, we use Eigen space transform for reducing the dimensions of the energy halation images before applying to MLP-neural network. Training MLP-NN is performed over each leg gesture for human gait recognition. So five trained MLP-NNs are created and use for human identification but each network recognized people separately based on different features (these features are energy halation over each leg gesture).Recognition rate of each network does not satisfy the using system as good human gait recognizer so we combine neural networks output using neuro-fuzzy-based mixer classifiers which is followed in the next sub-section.


## 2.5. Neuro-Fuzzy-Based Combiner Classifier

Neuro-fuzzy system has been proved to have significant results in modeling nonlinear functions. Neuro-fuzzy system has been used frequently in the literature as fishing predictions [23], vehicular navigation [24], identifying the turbine speed dynamics [25], radio frequency power amplifier linearization [26], microwave application [27], image denoising [28, 29], prediction in cleaning with high pressure water [30], sensor calibration [31], fetal electrocardiogram extraction from ECG signal captured from mother [32], and identification of normal and glaucomatous eyes [33].In a neuro-fuzzy system, the membership functions (MFs) are extracted from a data set that describes the system behavior. The neuro-fuzzy system learns features in the data set and adjusts the system parameters according to given error criterion. In a fused architecture, NN learning algorithms are used to determine the parameters of fuzzy inference system. Below, we have summarized the advantages of the neuro-fuzzy system technique. Fusion of output classifiers with linear combiner has been pointed in [34]. In this paper, we used a nonlinear mixer classifier which is based on neuro-fuzzy system for the first time in human gait recognition.


## 3. Experimental Results

A set of film including 160 sequences of eight people is used as data base. Frame rate per second is 25, and image size is 352 × 288. Some images from data base are shown in Figure9.Figure 9
Some samples of people image.Leg gesture recognizer is a three-layer MLP neural network with eight input neurons and five output neurons and fifteen neurons in hidden layer that can categorize input frames to 5 states. An example of this stage is shown in Figure10.Figure 10
Result of gesture recognizer system.As it was mentioned before, each gesture helps in categorization of frame sequence in five images of energy halation are performed, and five MLP neural networks are trained over 10 film sequences for 8 people. Each network has 50 neurons in input layer, and three hidden layers with 100, 90, 40 neurons and 8 neurons in output layer. In testing phase, captured confusion matrixes for two networks are shown in Tables1 and 2. These tables show that fusion of networks increases performance. As an example, network 2 can recognize people 1 but network 1 cannot perform recognition over this people as well. Confusion matrix after application of neuro-fuzzy combiner is shown in Table 3. Recognition rate increases to 99.8% over test pattern whereas learning of neuro-fuzzy system has been performed over learning patterns.Table 1
Confusion matrix of neural networks 1and 2 related to the 1st and 2nd gestures.
NN1P1P2P3P4P5P6P7P8P170000000P237012001P300900000P400091000P502007000P601100700P7000003100P800000009Table 2
Confusion matrix of neural networks 1and 2 related to the 1st and 2nd gestures.
NN2P1P2P3P4P5P6P7P8P1100000000P208000001P3001000000P400081100P502025130P600001500P700002370P800001009Table 3
Confusion matrix of proposed system as shown in Figure2.
NFP1P2P3P4P5P6P7P8P1100000000P209100000P3001000000P4000100000P5000010000P6000001000P7000000100P8000000010As an approach to evaluate our proposed method (gait recognition based on NFCC), we also analyzed comparison between our method and different algorithms of HumanID Gait Challenge Dataset (HGCD) and compared its result with a recent algorithm of gait recognition which have been evaluated by HGCD (Table4).Table 4
The final result of NFCC performance in gait recognition in comparison with HGCD and ASM algorithms [1, 17].
AlgorithmWidth vectors (UMD)DTW(UMD)HMM(UMD)BodyShape (CMU)HMM (MIT)Body (CAS)BaselineASMProposed method (NFCC)Average performance*25.730.153.246.243.239.249.889.194.2*The average performance under different experiments of HGCD.


## 4. Conclusion

An interesting note was found in this paper “human gait recognition based on leg gestu.” But this paper includes a new spatio-temporal gait data base (Energy Halation Image), neuro-fuzzy-based combiner classifier (NFCC). To overcome the limitation of recognition performance rate, we proposed a system for gait feature fusion. We used five spatio-temporal data bases and applied their features in Eigen space to five neural networks separately. Performance of each NN for test samples was low (about 70% to 80%). Then we used a neuro-fuzzy combiner classifier for mixing the neural networks for the first time in gait recognition. Result of combination of neural network outputs was satisfying.



---

*Source: 289721-2011-11-17.xml*