2009 IEEE International Conference on Robotics and Automation
Kobe International Conference Center
Kobe, Japan, May 12-17, 2009

Person identiﬁcation from human walking sequences using afﬁnemoment invariants
Yumi Iwashita and Ryo Kurazume
Abstract— This paper proposes a new person identiﬁcation
method using physiological and behavioral biometrics. Various
person recognition systems have been proposed so far, andone of the recently introduced human characteristics for the
person identiﬁcation is gait. Although the shape of one’s bodyhas not been considered much as a characteristic, it is closely
related to gait and it is difﬁcult to disassociate them. So,
the proposed technique introduces a new hybrid biometric,
combining body shape (physiological) and gait (behavioral).
The new biometric is the full spatio-temporal volume carved
by a person who walks. In addition to this biometric, we
extract unique biometrics in individuals by the following way:
creating the average image from the spatio-temporal volume
and forming the new spatio-temporal volume from differential
images which are created by subtracting an average image fromoriginal images. Afﬁne moment invariants are derived fromthese biometrics, and classiﬁed by a support vector machine.
We used the leave-one-out cross validation technique to estimate
the correct classiﬁcation rate of 94 %.

I. I NTRODUCTION
Person recognition systems have been used for a wide variety applications, such as secure access to buildings, computer
systems, automated teller machines (ATMs). For reliable personal recognition systems, biometrics have received growinginterest. These include physiological biometrics, related tothe shape of the body, the oldest of which are the ﬁngerprints;
and the behavioral biometrics, related to the behavior of aperson, the ﬁrst of which used was the signature. Several
biometric recognition systems are available in the market.
However, these systems need special equipment and requireinteraction with or cooperation of the subject.
One of the recently introduced human characteristics thatare expected to overcome these limitations is gait [1] [10].
Gait is the peculiar way one walks and is a complex spatiotemporal biometric. Gait recognition has the advantage of being unobtrusive because body-invasive sensing is not needed
to capture gait information. Moreover, gait recognition has
the extra advantage that it may be performed from a distance.
One of the other characteristics that has not been considered much is the shape of one’s body. This is probably
because the shape of the body changes with time and withclothing. However, it is difﬁcult to disassociate body shapefrom gait. So, this paper introduces a new hybrid biometric,
combining body shape (physiological) and gait (behavioral).
When a person walks, she/he carves a speciﬁc volume in thespatio-temporal domain. It is the shape of this volume we
Y. Iwashita and R. Kurazume are with Graduate Facultyof Information Science and Electrical Engineering, Kyushu
University, Motooka, Nishi-ku, Fukuoka 819-0395, Japan yumi,

kurazume@is.kyushu-u.ac.jp

978-1-4244-2789-5/09/$25.00 ©2009 IEEE

wish to consider as a biometric. In addition to this biometric,
we consider another biometric, that is, the average image
obtained from the spatio-temporal volume. Moreover, foremphasizing the difference of features in individuals, weform a new spatio-temporal volume from new differential
images which are created by subtracting an average image
from original images. In this paper we propose a novel
person identiﬁcation method which utilizes the full spatiotemporal volume, its average image, and the new spatiotemporal volume.
This paper is organized as follows. Section 2 is a brief
literature survey on the identiﬁcation of individuals using
gait. Research that considers the spatio-temporal shapes
created by a walking person is reviewed in section 2.2.
However none of these papers considers the full volumes
as potential biometrics. Section 3 describes the methodology
we shall use in this paper. Section 4 describes the data weshall use and the experiments performed. Our results andconclusions are presented in section 5.
II. L ITERATURE SURVEY ON GAIT AS A BIOMETRIC
For person identiﬁcation from their gait, several approaches have been proposed. They may be mostly classiﬁed into two classes, model-based and appearance-based
approaches.
A. Model-based approaches
A model-based approach recovers explicit features describing gait dynamics, such as stride dimensions and thekinematics of joint angles. Bouchrika and Nixon [11] described spatial model templates for human gait in a parameterized form using Fourier descriptors. The positions of thejoints of walking people were extracted by using the Hough
Transform.
Cunado et al. [2] extracted the motion of the thigh, anddeﬁned their gait signature by Fourier analysis. Yam et al. [3]extended the system [2] to handle walking as well as running
people. They extracted the motion of the hip, the thigh,and the lower leg by temporal template matching. Phaseweighted Fourier description gait signatures were derived
from the extracted movements.
Urtasun and Fua [12] introduced a model composed of
implicit surfaces attached to an articulated skeleton. The
motion of people was tracked by using the 3D model andclusters of 3D points captured by a stereo camera. By
using the 3D model, they increased the robustness to achanging view direction. Lee and Grimson [13] introduced436

gait representation based on moments extracted from orthogonal view video silhouettes. Their gait appearance feature
vector comprised parameters of moment features of image
regions. Seven ellipses were ﬁtted to different parts of thebinarized silhouette of the person and the parameters of theseellipses, such as location of their centroids, eccentricities,
etc., were used as features to represent the gait of theperson. BenAbdelkader et al. [14] used stride and cadence
for the identiﬁcation of people. The person’s identity wasdeﬁned based on parametric Bayesian classiﬁcation of thecadence and stride feature vector. From their experiments,
the variation in stride length with cadence was found to belinear and unique for different people.
B. Appearance-based approaches
Appearance-based approaches directly extract parametersfrom images without assuming a model of the human bodyand its motion. This approach characterizes body movementby the statistics of the spatio-temporal (XYT) patterns generated in the image sequences by the walking person. Here,the XYT patterns are formed by piling up frames in animage sequence as shown in Fig.1. There are many waysof extracting XYT patterns from the image sequences of awalking person.

bounding contours detected by ﬁtting ‘snakes’. Similarly, Liuet al. [18] extracted the XT and YT projections of the binary
silhouettes. They used a frieze pattern to represent gait motion, that is a pattern created by summing up the white pixels
of a binarized image of a gait along the rows and columnsof an image. BenAbdelkader et al. [19] characterized gait interms of a 2D signature computed directly from the sequenceof silhouettes. The signature consisted of self-similarity plots(SSP), deﬁned as the correlation of all pairs of images in thesequence. Han et al. used the gait energy image (GEI) which
is based on the average image [6].
Little and Boyd [7] used optical ﬂow instead of binary
silhouettes. They ﬁtted an ellipse to the dense optical ﬂow
of the person’s motion, then computed thirteen scalar featuresconsisting of ﬁrst- and second-order moments of this ellipse.
Twelve measurements were provided from thirteen features.
III. M ETHODOLOGY
In this section, we describe the methodology we use inthis paper. Here, we assume that a target region in an image
sequence is extracted.
A. Biometrics from human walking sequences

In the proposed method, we consider three biometrics, thatis, the full spatio-temporal volume carved by a person who
walks, the average image from the spatio-temporal volume,and the new spatio-temporal volume which is created by theaverage image and original images. The above biometrics are
explained in details. Firstly, Fig. 1 (a) shows the 3D volume
in the spatio-temporal (XYT) domain, and it is formed by
X piling up the target region in the image sequences of one gaitYcycle, which is used to partition the sequences for the 3DTvolume. One gait cycle is a fundamental unit to describe theXgait during ambulation, which occurs from the time whenTthe heel of one foot strikes the ground to the time at which(a) XYT image sequence volume(b) Braided pattern in the XT-slice
the same foot contacts the ground again. In this paper, wetaken at the ankle
assume that the 3D volume consists of a number of small
Fig. 1. XYT image sequence volume.
voxels.
Next, the average image I average (x, y) is deﬁned as
The simplest approach is to use the sequence of binary follows:silhouettes spanning one gait cycle and scaled to a certain
uniform size [4]. Murase and Sakai [4] proposed a templatematching method in the parametric eigenspace that was
created from images. Sarker et al. [5] proposed a baseline
algorithm of gait recognition, which estimated the silhouettes
by background subtraction and performed recognition by
spatio-temporal correlation of silhouettes. Collins et al. [15]used silhouettes corresponding to certain gait poses only. Liuet al. [16] adopted dynamics-normalized shape cues witha population HMM which emphasize difference in stanceshapes between subjects and suppresses differences for thesame subject under different conditions.
Other methods use a signature of the silhouette by collapsing the XYT data into a more terse 1D or 2D signal,such as vertical projection histograms (XT), and horizontal
projection histograms (YT) [17]. Niyogi and Adelson [17]extracted XT sheets that encoded the person’s inner and outer

Iiaverage (x, y) =

T

i
1 
Ii (x, y, t),
Ti t=1(1)where T is the number of frames in one gait cycle andI(x, y, t) represents the density of the voxels at time t.
Finally, for emphasizing the difference of features inindividuals, we create new images by subtracting the average
image from the original images, and then the new spatiotemporal volume is formed. Here, the average image I averageis deﬁned as follows:1
I average (x, y) = N

Ti
N 


i=1 Ti i=1 t=1

Ii (x, y, t),(2)where N is the number of subjects. New images Iinew (x, y, t)
are given by

437

Iinew (x, y, t) = max(0, Ii (x, y, t) − I average (x, y)).

(3)

−6μ220 μ02 μ21 μ03 + 9μ220 μ02 μ212
+12μ20 μ211 μ21 μ03 + 6μ20 μ11 μ02 μ30 μ03

Figure2 (a) shows the average image, and Fig.2 (b) shows anexample of the differential images. For characterizing these
2D average images and 3D volumes, we consider the 2D and
3D afﬁne moment invariants as features.

−18μ20 μ11 μ02 μ21 μ12 − 8μ311 μ30 μ03−6μ20 μ202 μ30 μ12 + 9μ20 μ202 μ221

I5I6(a) An average image

Fig. 2.

+12μ211 μ02 μ30 μ12 − 6μ11 μ202 μ30 μ21+μ302 μ230 )1=(μ40 μ04 − 4μ31 μ13 + 3μ222 )μ600
1=(μ40 μ04 μ22 + 2μ31 μ22 μ13
μ900−μ40 μ213 − μ04 μ213 − μ322 )(8)

For a 3D space in which I(x, y, t) represents the densityof the voxels, the 3D moment of order (p + q + r) of a 3D
object O is given by the same procedure with 2D centralized
moments.
μpqr =(b) An example
a new image

An average image and an example of differential images.

(x,y,t)∈O(x − xg )p (y − yg )q (t − tg )r I(x, y, t),(9)

B. 2D and 3D afﬁne moment invariants
In this section, we introduce the 2D and 3D afﬁnemoment invariants. Afﬁne moment invariants are momentbased descriptors, which are invariant under a general afﬁnetransform. The moments describe shape properties of an
object as it appears.
For an image I(x, y) the 2D moment of order (p + q) ofan object O is given byμpq =xp y q I(x, y)dxdy.
(4)where xg , yg and tg are the coordinates of the center ofgravity of an object in the 3D space.
Six 3D afﬁne moment invariants are given in [9] [20], andtwo of them are listed below. For the rest of them, refer to[20] because of their long formulae.

The discrete version of Eq.4 is written asμpq =xp y q I(x, y).

I2I1(x,y)∈O(x,y)∈O(5)

The center of gravity of an object in the image can be
determined from the zeroth and the ﬁrst-order moments by
μ10
μ01
,
yg =
.
(6)
xg =
μ00
μ00
Centralized moments are computed by using the coordinates
xg and yg :μpq =(x − xg )p (y − yg )q I(x, y).
(7)(x,y)∈O

Six afﬁne moment invariants are listed below [8].
I1
I2

I3

I4

1
(μ20 μ02 − μ211 )μ400
1=(μ230 μ203 − 6μ30 μ21 μ12 μ03 + 4μ30 μ312
μ1000
+4μ03 μ321 − 3μ221 μ212 )1=(μ20 (μ21 μ03 − μ212 ) − μ11 (μ30 μ03
μ700−μ21 μ12 ) + μ02 (μ30 μ12 − μ221 ))1=(μ320 μ203 − 6μ220 μ11 μ12 μ03
μ11
00
=

1
(μ200 μ020 μ002 + 2μ110 μ101 μ011
μ5000
−μ200 μ2011 − μ020 μ2101 − μ002 μ2110 )1=(μ (μ μ + 3μ2022 − 4μ013 μ031 )7μ000 400 040 004+3μ202 (μ040 μ202 − 4μ112 μ130 + 4μ2121 )+12μ211 (μ022 μ211 + μ103 μ130 − μ031 μ202=

−μ112 μ121 ) + 4μ310 (μ031 μ103 − μ004 μ130
+3μ013 μ121 − 3μ022 μ112 ) + 3μ220 (μ004 μ220

+2μ022 μ202 + 4μ2112 − 4μ013 μ211 − 4μ121 μ103 )+4μ301 (μ013 μ130 − μ040 μ103 + 3μ031 μ112
−3μ022 μ121 ))(10)

In the proposed method, we extract 2D afﬁne moment
invariants from the average image and 3D afﬁne moment
invariants from the full spatio-temporal volume of each subject at ﬁrst, and the classiﬁer is trained by using the training
data sets. Then in the identiﬁcation phase, the same afﬁnemoment invariants are extracted from the spatio-temporal
volume and its average image, and the subject is identiﬁedby the classiﬁer.
IV. E XPERIMENTS
In this section we describe the experiments. In our experiments, we used a gait database collected by the Universityof Southampton [21]. The database contains 140 raw image
sequences and foreground masks, which contains 140 video438

sequences, which contain 20 different subjects with 7 sequences for every subject. Figure 3 shows an example entry
from the database.
Moreover, we describe the extended method with the aimof obtaining detailed features of the spatio-temporal volume.
This method makes explicit the movement and shape of thearms from those of the legs, since different people move theirarms differently when they walk. To extract these features,
we divide the spatio-temporal volume into upper and lower
volumes.
A. Experiments with the spatio-temporal volume and theaverage image
We carried out four experiments, and the experimental
conditions are as follows: (Exp. A) the binarized spatiotemporal volume, (Exp. B) the average image from thespatio-temporal volume, (Exp. C) the new spatio-temporal
volume created from the differential images, and (Exp. D)
the average image and the new spatio-temporal volume. In
our experiments, the support vector machine (SVM) and the
k-nearest neighbour classiﬁcation were applied to the afﬁnemoment invariants as the classiﬁer. We used the leave-oneout cross validation to estimate the classiﬁcation error rate.
In the ﬁrst experiment the 3D afﬁne moment invariants areused for the classiﬁcation. Figure 4 shows the ﬁrst two afﬁnemoment invariants used for 10 out of the 20 subjects andtheir whitened afﬁne moment invariants. Here, data-speciﬁcvariation can be removed by whitening. The resultant correctclassiﬁcation rate using the SVM and knn were 75 % and
69 %, respectively.
In the second experiment we used the 2D afﬁne moment
invariants. Figure 2 (a) shows an example of the average
images. Here, the images are properly aligned and scaled toa uniform height before calculating the average image. The
resultant correct classiﬁcation rate using the SVM and knn
were 92 % and 89 %, respectively. From this experiment, we
could say that the average images show better performancefor each person than the binary spatio-temporal volumes.
In the next experiment we used the the 3D afﬁne moment invariants of the new spatio-temporal volume, which
are created for emphasizing the difference of features inindividuals. The resultant correct classiﬁcation rate using the
SVM and knn were 84 % and 79 %, respectively, which
was higher than the binary spatio-temporal volume. In theﬁnal experiment, we used 2D afﬁne moment invariants of
the average image and 3D afﬁne moment invariants of the
new spatio-temporal volume. The resultant correct classiﬁcation rate using the SVM and knn were 94 % and 90 %,respectively, which was the highest score in the series of theexperiments. Table I shows the correct classiﬁcation rate of
our experiments.
Using the same database, Nixon et al. [11] used dynamic
and static gait features to yield a feature vector. Staticfeatures include the body height, stride and heights ofdifferent body parts while dynamic features are the phaseweighted magnitudes of the Fourier frequencies for the hipand knee angular motions. The gait signature is derived using

the adaptive forward ﬂoating search algorithm via selecting
the features with higher discriminability values. They used
the k-nearest neighbor rule as a classiﬁer, and their systemachieved a correct classiﬁcation rate of 92 % using the leaveone-out cross validation rule. In their experiment they used
48 features for classiﬁcation, on the other hand, we used
12 features. The comparison of their experiment and ourexperiment is shown in Table II.
B. Experiments with the divided volumes
For obtaining detailed features of the spatio-temporal volume, we divide the 3D volume into upper and lower volumes
by threshold processing. We carried out four experiments,
and the experimental conditions are as follows: (Exp. E) two
volumes of the 3D binarized volume, (Exp. F) two volumesof the new 3D volume from differential images, (Exp. G)
the 3D binarized volume and its divided volumes, and (Exp.
H) the new spatio-temporal volume and two volumes of the
3D binarized volume. In these experiments, the 3D volume
is divided into two volumes at two ﬁfth from the bottom
as show in Fig. 5, and the k-nearest neighbour classiﬁcation
was applied to the afﬁne moment invariants as the classiﬁer.
The resultant correct classiﬁcation rates in (Exp. E ∼ H)
were 87 %, 84 %, 92 %, and 93 %, respectively. From theseexperiments, by dividing the full 3D volume into upper andlower volumes, more detailed features could be obtained than
the full spatio-temporal volume.

Fig. 3.

Samples from the University of Southampton database. [21]V. C ONCLUSIONSWe proposed in this paper the person identiﬁcation method
using the hybrid biometric of physiological and behavioral
biometrics, which is the full spatio-temporal volume carved
by a person who walks. In addition to this biometric, its
average image and new spatio-temporal volume, which was
created from the average image and original images, werealso considered as biometrics. We showed that afﬁne moment invariants in conjunction with an SVM classiﬁer may
produce marginally better results than those based on gait
analysis alone and k-nearest neighbour classiﬁcation. Moreover, for obtaining detailed features of the sptaio-temporal
volume, we extended the proposed method by dividing the3D volume into upper and lower volumes. The calculationof the afﬁne moment invariants is very straight forward,
unlike the individual cues extracted to characterize gait andcertain characteristics of the human body. In addition, the439TABLE I
T HE CORRECT CLASSIFICATION RATE OF OUR EXPERIMENTS
Biometrics

The number of features(Exp.A) 3D volume of binarized images(Exp.B) Average images(Exp.C) 3D volume of differential images(Exp.D) Average images and3D volume of differential images

6
6
6
12

Correct classiﬁcation rate [%]
SVM
knn
75
69
92
89
84
79
94
90

TABLE IIC OMPARISON OF THE EXPERIMENT OF [11] AND OUR EXPERIMENTClassiﬁer
FeaturesClassiﬁcation rate [%]The experiment of [11]The k-nearest neighbor rule
The gait signature
consisting of 48 features
92

afﬁne moment invariants, being integrators, are more robustfeatures than features based on differentiation.
One drawback may be that the spatio-temporal volumes
we consider may be affected by clothing, so their use inunconstrained situations may not be robust. However, they
may be used in conjunction with sensors that can “seethrough” clothing. Moreover, another limitation is the shapedependence on the viewing angle. To deal with this problem,the classiﬁer can be trained by using the training data setsof different viewing angles. Finally, another drawback is thatby the extended method, although the number of features
is increased, it may ignore phase difference between the
movement of arms and that of legs. However, from theexperimental results, the proposed method is promising andmerits further investigation.
VI. ACKNOWLEDGMENTS
The authors are grateful to Professor Maria Petrou (Imperial College London) for her helpful suggestions.
R EFERENCES[1] M.P. Murray, Gait as a Total Pattern of Movement, Am. J. Physical
Medicine, vol. 46, no. 1, pp. 290-329, 1967.
[2] D. Cunado and M. Nixon and J. Carter, Automatic Extraction and
Description of Human Gait Models for Recognition Purposes, CVIU,
vol. 90, no. 1, pp. 1-41, 2003.
[3] C. Yam and M. Nixon and J. Carter, Automated person recognition
by walking and running via model-based approaches, Pattern Recognition, vol. 37, no. 5, pp. 1057-1072, 2004.
[4] H. Murase and R. Sakai, Moving object recognition in eigenspace
representation: gait analysis and lip reading, Pattern Recognition
Letters, vol. 17, no. 2, pp. 155-162, 1996.
[5] S. Sarkar and P. Phillips and Z. Liu and I. Vega and P. Grother and K.
Bowyer, The humanID gait challenge problem: data sets, performance,
and analysis, IEEE Trans. PAMI, vol. 27, no. 2, pp. 162-177, 2005.
[6] J. Han and B. Bhanu, Individual Recognition Using Gait Energy
Image, IEEE Trans. PAMI, vol. 28, no. 2, pp. 316-322, 2006.
[7] J. Little and J. Boyd, Recognizing People by Their Gait: The Shapeof Motion, Videre, vol. 1, no. 2, pp. 1-32, 1998.
[8] J. Flusser and T. Suk, Pattern recognition by afﬁne moment invariants,
Pattern Recognition, vol. 26, no. 1, pp. 167-174, 1993.

Our experiment
The support vector machine
2D and 3D afﬁne moment
invariants consisting of 12 features
94[9] A. G. Mamistvalov, n-Dimensional Moment Invariants and Conceptual
Mathematical Theory of Recognition n-Dimensional Solids, IEEE
Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 8, pp.
819-831, 1998.
[10] M. Nixon and J. Carter, Automatic Recognition by Gait, Proc. theIEEE, vol. 94, no. 11, pp. 2013-2024, 2006.
[11] I. Bouchrika and M. Nixon,, People Detection and Recognition using Gait for Automated Visual Surveillance, Proc. IEE Inter. Symp.
Imaging for Crime Detection and Prevention, 2006.
[12] R. Urtasun and P. Fua, 3D Tracking for Gait Characterization and
Recognition, Proc. 6th IEEE Int. Conf. Automatic Face and Gesture
Recognition, pp. 17-22, 2004.
[13] L. Lee and W. Grimson, Gait Analysis for Recognition and Classiﬁcation, Proc. IEEE Conf. Face and Gesture Recognition, pp.155-161,
2002.
[14] C. BenAbdelkader and R. Cutler and L. Davis, Stride and Cadence asa Biometric in Automatic Person Identiﬁcation and Veriﬁcation, Proc.
IEEE Conf. Face and Gesture Recognition, pp. 372-377, 2002.
[15] R. Collins and R. Gross and J. Shi, Silhouette-based human identiﬁcation from body shape and gait, IEEE Int. Conf. Automatic Face andGesture Recognition, pp. 351-356, 2002.
[16] Z. Liu and S. Sarkar, Improved Gait Recognition by Gait Dynamics
Normalization, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp. 863-876, 2006.
[17] S. Niyogi and E. Adelson, Analyzing and Recognizing Walking Figures
in XYT, IEEE Conf. Computer Vision and Pattern Recognition, pp.
469-474, 1994.
[18] Y. Liu and R. Collins and Y. Tsin, Gait Sequence Analysis using Frieze
Patterns, Proc. the 7th European Conf. Computer Vision (ECCV’02),
2002.
[19] C. BenAbdelkader and R. Cutler and H. Nanda and L. Davis,
EigenGait: Motion-based Recognition of People using Image SelfSimilarity, 3rd Int. Conf. Audio- and Video-Based Biometric Person
Authentication, 2001.
[20] D. Xu and H. Li, 3-D Afﬁne Moment Invariants Generated byGeometric Primitives, Proc. 18th IEEE Int. Conf. Pattern Recognition,
pp. 544-547, 2006.
[21] J. D. Shutler and M. G. Grant and M. S. Nixon and J. N. Carter, Ona Large Sequence-Based Human Gait Database, Proc. 4th Int. Conf.
Recent Advances in Soft Computing, pp. 66-71, 2002.

440

2.5e+006

Subject 15
Subject 23
Subject 24
Subject 25
Subject 27
Subject 30
Subject 31
Subject 57
Subject 102
Subject 115

2.0e+006

1.5e+006

1.0e+006

0.5e+006

00.8e+007

1.2e+007

1.6e+007

2.0+0072.4+007(a) Affine moment invariants

3.0
Subject 15
Subject 23
Subject 24
Subject 25
Subject 27
Subject 30
Subject 31
Subject 57
Subject 102
Subject 115

2.0

1.0

0.0

-1.0

-2.0
-2.0

-1.0

0.0

1.0

2.0

3.0

(b) Whitened affine moment invariants
Fig. 4. The ﬁrst two afﬁne moment invariants plotted against each otherfor 10 of the 20 subjects and their whitened values.

Y

T
Fig. 5.

X

Upper and lower volumes.

441