Gait Recognition Robust to Speed Transition
Using Mutual Subspace Method
Yumi Iwashita1(B) , Hitoshi Sakano2 , and Ryo Kurazume1
1

Kyushu University, Fukuoka, Japanyumi@ieee.org
2
NTT Data Corporation, Tokyo, Japansakanoh@nttdata.co.jp

Abstract. Person recognition from gait images is not robust to speed
changes. To deal with this problem, generally existing methods havefocused on training a model to transform gait features from various
speeds into a common walking speed, and the model was trained withgait images with a variety of speeds. However in case that a subjectwalks with a speed which is not trained in the model, the performancegets worse. In this paper we introduce an idea that an image set-basedmatching approach, which omits walking speed information, has a potential to solve the problem. This is based on the assumption that speed
information may not be critical information to gait recognition, since
speed variations are universal phenomena. To prove the proposed idea,
we apply a mutual subspace method to gait images and show the eﬀectiveness of the proposed idea with the OU-ISIR gait speed transition
database.
Keywords: Gait recognition · Speed transition · Subspace method

1

Introduction

Gait-based person recognition has received an increasing amount of attentions
for monitoring and surveillance applications, thanks to the advantages that gait
does not require interaction with a subject and can be obtained from a distance.
Person identiﬁcation methods that extract features from gait images taken by acamera have been used with good results. Existing methods using appearancebased gait features, such as gait energy image (GEI) [1], active energy image(AEI) [2] and frame diﬀerence frieze pattern (FDFP) [3], reported good performance with publicly available gait databases [4] [5] [6] [7].
However, there are several causes which make the performance of gait recognition worse, and one of them is walking speed change. The walking speed change
causes variations in pitch and stride, which result in non-invariant gait features.
To address this problem existing methods focused on transforming the gait features from various speeds into a common walking speed [8] or extraction ofspeed-invariant gait feature [9]. However, since the existing methods assumedc Springer International Publishing Switzerland 2015

V. Murino and E. Puppo (Eds.): ICIAP 2015, Part I, LNCS 9279, pp. 141–149, 2015.
DOI: 10.1007/978-3-319-23231-7 13

142

Y. Iwashita et al.

that the walking speed is constant in one gait cycle, the performance of gait
recognition is low in cases in which the subject accelerates or decelerates. The
speed transition can happen in daily life, such as acceleration after a ticket gate
at a station and deceleration before stop at red traﬃc signal, and the speedtransition problem was ﬁrstly reported by Mansur et al. [10]. They proposeda method which synthesized constant-speed gait images based on a cylindricalmanifold, which was based on a mapping function trained with an auxiliary
training set including gait images under various speeds. Then experimental evaluations of the proposed method was done with a newly released database forspeed transition.
As we mentioned above, most of existing methods assumed to have auxiliary training sets to train models. However, in case that a subject walks with aspeed which is not included in the auxiliary training set, the performance getsworse. In this paper, we introduce a new idea as follows. Since speed variations
are universal phenomena, there is a possibility that speed information does nothave to be regarded as critical information to gait recognition. Based on thisassumption, we obtain an idea that an image set-based matching approach cansolve the gait recognition problem. To show the eﬀectiveness of this idea, inthis paper we apply a mutual subspace method (MSM) [14] [15] to gait images
under speed transition. Subspace-based methods have been popularly used inface recognition, and the eﬀectiveness of MSM is demonstrated in [16]. Advantages of the use of the image set-based matching approach to gait recognition
are as follows: (i) we can directly apply gait images/gait features to the image
set-based matching approach without any process such as constant-speed gait
image generation [10], and (ii) we do not need any auxiliary training sets. We
conducted experiments with the database which Mansur et. al used in [10], andwe obtained results with higher performance compared with [10].
There are some existing gait recognition methods using subspace techniques.
Liu et al. proposed a gait recognition method which was robust in subject’s walking direction changes with respect to a camera [11]. In [11], GEI, which is one of
popular gait features, was extracted from gait images at each walking direction.
A subspace method was applied to a set of GEI from multiple walking directions,and then a weighted subspace distance was used for person identiﬁcation. Connieet al. proposed a gait recognition method which formulated a gait subspaces onthe Grassmann manifold [12]. This method can be applied to either gait images
with walking direction changes or gait images with speed changes, and it also
used GEI which was extracted from gait images at each condition (e.g. at eachwalking direction/speed).
These existing methods need training datasets which include gait images atmultiple conditions, such as gait images under multiple walking directions/speeds.
We agree that gait images at multiple walking directions are necessary for gait
recognition robust to changes in walking direction. However, as we mentioned
above, speed information may not be critical information to gait recognition,and thus the proposed image set-based matching approach, which ignores speed
information, has a big potential to be eﬃcient to speed changes.

Gait Recognition Robust to Speed Transition

143

The remainder of the present paper is organized as follows. Section 2 describes
gait recognition using the mutual subspace method. Section 3 describes experiments performed to demonstrate the eﬃciency of the proposed idea using theOU-ISIR gait speed transition database, and Section 4 presents our conclusions.

2

Gait Recognition Using Mutual Subspace Method

In this section we brieﬂy review the mutual subspace method, and then weexplain the way how we apply the mutual subspace method as an image setbased matching.
2.1

Mutual Subspace Method

The Mutual Subspace method (MSM) is regarded as one of powerful image set image set matching techniques. The MSM models template images of each classin gallery dataset and input images in probe dataset as subspaces, and images
are embedded to the subspaces. The similarity measure in the MSM is calculated
as a canonical angle between template subspace and input subspace.
Let us assume C class pattern recognition problem. Patterns in class c arerepresented in d dimensional vectors xc 1 , . . . , xc nc , where nc is the number oftraining samples in class c. Bases of the class c template subspace are given by
the following eigenequation;(1)
Γ c φc = λc φc ,where λc and φc are eigenvalue and eigenvector, respectively. Γ c is autocorrelation matrix of class c training samples,ncnc1   c cT
Γ = cx x .
n i=1 j=1 i jc(2)Similarly input images x1 , . . . , xn are represented by eigenvector of autocorrelation matrix ψ. The similarity measure in the MSM is canonical angle between
two subspaces, which are calculated as eigenvalues of following matrix [17].
Z c = (ζij )c =M(φci · ψ m )(φcj · ψ m ),

(3)

m=1

where M is dimensionality of input subspace. Dimensionalities of training andinput images are inﬂuential parameters. Roughly speaking the dimensionality
is proportional to variation of training and input images. However when thedimensionality increase too large, recognition accuracies are worse, because of
the increasing amount of intersections among subspaces. Generally such dimensionalities are deﬁned by cross validation experiments. Finally, class c is chosen,
in case that the maximum eigenvalue of Z c is the highest one among all classes.

144

2.2

Y. Iwashita et al.

Gait Recognition Using an Image Set-Based Matching

Assume that there are gait images of a subject and silhouette images are obtained
as shown in Fig. 1 (a). As a baseline method of the image set-based approach,
we directly use intensity values of each silhouette image as a feature vector, andwe apply the mutual subspace method to a set of feature vectors from silhouette
images. Fig. 1 (b) shows examples of visualizations of principal components.
We projected silhouette images to ﬁrst, second, and third principal components
as shown in Fig. 2. Fig. 2 (a) shows projected results of gait image to 1st-2nd
principal components, and images of frame ID 1 to 37 are for the duration of
the ﬁrst half gait cycle (one step) and those of frame ID 38 to 74 are for theduration of the last half gait cycle (one step). Fig. 2 (b) shows projected resultsof gait image to 2nd-3rd principal components.
#0

#5

#10

#15

#20

#25

#30

#35

#40

(a)
1st

2nd

3rd

4th

5th(b)
Fig. 1. (a) Examples of gait images and (b) visualization of each principal component(color information is scaled in the range from 0 to 255, for the purpose of visualization).

3Experiments

In this section, we implement the mutual subspace method and evaluate its
performance on the OU-ISIR gait speed transition database [10].
3.1

Experimental Settings

Datasets. The OU-ISIR gait speed transition dataset is the only one dataset
which includes gait datasets with speed transition within an image sequence.
The dataset consists of two diﬀerent datasets, dataset 1 and dataset 2. In thedataset 1 the probe set consists of speed transited gait sequences recorded from26 subjects. More speciﬁcally, each subject gradually decreased walking speed
and ﬁnally stop, and the ﬁnal gait period of each subject was selected as theGait Recognition Robust to Speed Transition
#33

#36
#60

#20

#66
#15
#13

#40
#45

#70

#10
#1

#50

#48

1st principal component
(a)

#30

3rd principal component

2nd principal component

#30

145

#1

#26

#3

#23

#5

#20
#13
#10

#15
#18

2nd principal component(b)

Fig. 2. Projection of gait images to 1st-2nd principal components (a) and 2nd-3rd
principal components (b). Frame ID 1 to 37 are for the duration of the ﬁrst half gait
cycle and frame ID 38 to 74 are for the duration of the last half gait cycle.

probe dataset, which contains signiﬁcant change in stride. The gallery dataset
consists of gait sequences from 179 subjects, which include the 26 probe subjects,
and the subjects walked at the constant speed (4 km/h) for a few seconds.
In the dataset 2 the probe set consists of 25 subjects and each subject walked
twice on the treadmill under the following conditions: (i) accelerations from 1km/h to 5 km/h and (ii) decelerations from 5 km/h to 1 km/h. For gallery
dataset, there are 154 subjects, which include the 25 probe subjects, and eachsubject walked at a constant speed (4 km/h) for six seconds.
In addition to the two datasets we explained above, the OU-ISIR gait speedtransition dataset has an auxiliary training set, which includes 24 subjects undervarious walking speeds (2, 3, 4, an 5 km/h). It is worth to be pointed out thatin [10] the auxiliary training set was used for learning a mapping function tosynthesize constant-speed gait images. However, in case that the subject walks
with a speed which is not included in the auxiliary training set, the performancegets worse. On the other hand, the proposed approach does not need the auxiliary
training set, and thus it can work with subjects walking with arbitrary speeds.
Evaluation Setting. We evaluated the proposed approach with the following
two settings.
Setting 1: In general gait features are obtained from a set of images for theduration of one gait cycle. Thus we use gait images for the duration of onegait cycle for both probe and gallery datasets. This is the same evaluation
setting of [10]. In [10], GEI was extracted from each gait sequence which
includes gait images for the duration of one gait cycle. As we mentionedabove, each gallery sequence of dataset 2 is for six seconds and this means
the gallery sequences have images for the duration of multiple gait cycles.

146

Y. Iwashita et al.

Thus each gallery sequence is divided into multiple sequences so that eachof them include gait images for the duration of one gait cycle. The divided
gait sequences were used as gallery datasets.
Setting 2 (for dataset 2 only): We use gait images for the duration of multiple gait cycle for gallery datasets. As we explained in setting 1, there are
multiple gait cycles in the gallery dataset and the proposed approach can use
all of them. The use of multiple gait cycles may result in higher performancecompared with the use of one gait cycle.
3.2

Evaluation

We evaluated the proposed approach with 2 scenarios: (i) a veriﬁcation scenario and (ii) an identiﬁcation scenario. In the veriﬁcation scenario, we plotteda receiver operating characteristics (ROC) curve, which describes how true positive rate and false positive rate changes as a threshold for the similarity measurechanges. In the identiﬁcation scenario, we plotted a cumulative matching characteristics (CMC) curve (i.e., rank-n identiﬁcation rates).
Table 1. EER, rank-1 and rank-5 identiﬁcation rates for dataset 2 (acceleration).
Mansur et. al [10] P roposed P roposed(setting1) (setting2)
EER [%]
Rank-1 [%]Rank-5 [%]

8.0
72.0
96.0

7.0
92.0
92.0

1.0

1.0

0.9

0.9

0.8

0.8

0.7
0.6
0.5
0.4
0.3

dataset 1
dataset 2
acceleration
dataset 2deceleration

0.7
0.5
0.4
0.3
0.2

0.1

0.1
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

dataset 1
dataset 2
acceleration
dataset 2
deceleration

0.6

0.2
0.0

1.0
96.0
100.0

0.0

0

20

40

60

80

100Fig. 3. ROC (left) and CMC curves (right) for dataset 1, dataset 2 (acceleration), anddataset 2 (deceleration) with the setting 1.

Gait Recognition Robust to Speed Transition

1.0

1.0

0.9

0.9

0.8

0.8

0.7

0.7

0.6

dataset 2
acceleration
dataset 2
deceleration

0.5
0.40.30.6
0.4
0.3
0.2

0.1

0.1

0.0

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

dataset 2acceleration
dataset 2deceleration

0.5

0.2

0.0

147

0204060

80100Fig. 4. ROC (left) and CMC curves (right) for dataset 2 (acceleration) and dataset 2(deceleration) with the setting 2.

We ﬁrst evaluated the proposed approach with setting 1. Figure 3 shows
ROC curves and the CMC curves of the dataset 1, dataset 2 (acceleration), anddataset 2 (deceleration). These results show much higher performance comparedto those of [10] (please refer to [10]). These results show higher performance
than the existing method [10]. The results of dataset 1 show lower performancecompared with those of dataset 2. As the reason is explained in [10], the dataset
1 is more challenging since it contains signiﬁcant gait ﬂuctuation and very short
stride lengths.
Next we evaluated the proposed approach with setting 2. Figure 4 shows
the ROC curves and CMC curves of the dataset 2 (acceleration) and dataset 2(deceleration). Compared with the results of Fig. 3 at setting 1, the results of 4
at setting 2 signiﬁcantly improved the performance, thanks to the use of all gait
cycles in dataset.
Tables 1 and 2 show equal error rate (EER) and rank-1 and rank-5 identiﬁcation rates of the proposed approach and [10]. These results show the proposed
approach outperformed the existing method [10].
Table 2. EER, rank-1 and rank-5 identiﬁcation rates for dataset 2 (deceleration).
Mansur et. al [10] P roposed P roposed(setting1) (setting2)
EER [%]
Rank-1 [%]Rank-5 [%]

8.0
84.0
92.0

6.0
96.0
96.0

3.0
96.0
100.0

1484

Y. Iwashita et al.

Conclusions

This paper described a new idea of the use of the image set-based matchingapproach to solve the gait recognition problem in speed variations. The proposed
concept is based on the assumption that speed information may not be criticalinformation to gait recognition, since speed variations are universal phenomena.
We implemented a baseline approach using the mutual subspace method, andexperimental results with the OU-ISIR gait speed transition database showedthat the proposed approach outperformed the existing method. Future work
includes applying the proposed approach to diﬀerent gait datasets, such as OUISIR treadmill dataset which includes various speeds. Moreover, we will utilizea kernel mutual subspace method [13] to see if the non-linear subspace method
can work better than the linear one.
Acknowledgments. This work is supported by a Grant-in-Aid for Exploratory
Research (26630099).

References
1. Han, J., Bhanu, B.: Individual Recognition using Gait Energy Image. IEEE Trans.
on Pattern Analysis and Machine Intelligence, 28(2) (2006)
2. Zhang, E., Zhao, Y., Xiong, W.: Active energy image plus 2DLPP for gait recognition. Signal Processing 90(7), 2295–2302 (2010)
3. Shinzaki, M., Iwashita, Y., Kurazume, R., Ogawara, K.: Gait-based person identiﬁcation method using shadow biometrics for robustness to changes in the walking
direction. In: IEEE Winter Conf. on Applications of Computer Vision, pp. 670–677(2015)
4. Iwashita, Y., Baba, R., Ogawara, K. Kurazume, R.: Person identiﬁcation fromspatio-temporal 3D gait. In: Int. Conf. Emerging Security Technologies (EST),
pp. 30–35 (2010)
5. Iwashita, Y., Ogawara, K., Kurazume, R.: Identiﬁcation of people walking along
curved trajectories. Pattern Recognition Letters 46, 60–69 (2014)
6. CASIA Gait Database. http://www.sinobiometrics.com
7. Iwama, H., Okumura, M., Makihara, Y., Yagi, Y.: The OU-ISIR Gait Database
Comprising the Large Population Dataset and Performance Evaluation of Gait
Recognition. IEEE Trans. on Information Forensics and Security 7(5), 1511–1521(2012)
8. Tanawongsuwan, R., Bobick, A.: Modelling the eﬀects of walking speed on
appearance-based gait recognition. Computer Vision and Pattern Recognition(2004)
9. Liu, Z., Sarkar, S.: Improved gait recognition by gait dynamics normalization.
IEEE Trans. on Pattern Analysis and Machine Intelligence 28 (2006)
10. Mansur, A., Makihara, Y., Aqmar, R., Yagi, Y.: Gait Recognition under SpeedTransition. In: IPSJ Trans. on Computer Vision and Applications, ComputerVision and Pattern Recognition (2014)
11. Liu, N., Lu, J., Tan, Y., Li, M.: Set-to-set gait recognition across varying views andwalking conditions. In: IEEE Int. Conf. on Multimedia and Expo, pp. 1–6 (2011)

Gait Recognition Robust to Speed Transition

149

12. Connie, T., Go, M., Teoh, A.: A Grassmann graph embedding framework for gaitanalysis. EURASIP Journal on Advances in Signal Processing (2014)13. Sakano, H., Mukawa, N.; Kernel mutual subspace method for robust facial image
recognition. In: Int. Conf. on Knowledge-Based Intelligent Engineering Systems
and Allied Technologies, vol. 1, pp. 245–248 (2000)
14. Maeda, K., Watanabe, S.: Pattern matching method with local structure. Trans.
on IEICE (D) (Japanese Edition) 68–D(3), 345–352 (1985)
15. Maeda, K.: From the Subspace Methods to the Mutual Subspace Method. Computer Vision 285, 135–156 (2010)
16. Yamaguchi, O., Fukui, K., Maeda, K.: Face recognition using temporal imagesequence. In: IEEE Int. Conf. on Face and Gesture Recognition, pp. 318–323 (1998)
17. Chatelin, F.: Veleurs propres de matrices. Masson (1988) (In French)