LIDAR-based Body Orientation Estimation
by Integrating Shape and Motion Information
Masanobu Shimizu† , Kenji Koide†, Igi Ardiyanto‡ , Jun Miura† , and Shuji Oishi†† Department of Computer Science and Engineering
Toyohashi University of Technology‡ Department of Electrical Engineering and Information Technology
Faculty of Engineering, Universitas Gadjah Mada
Abstract— Body orientation gives useful information on assessing a human’s state and/or predicting his/her future actions.
This paper presents a method of reliably estimating humanbody orientation using a LIDAR on a mobile robot by integrating shape and motion information. A shape database isconstructed by collecting body section shape data from various
viewpoints. The result of matching between an input shapewith the database is combined with a UKF-based tracker
which utilizes a relationship between the body orientation andthe motion orientation. The experimental results shows theeffectiveness of the proposed method.

I. I NTRODUCTION
There is an increasing demand for personal service robots
that attend people and provide use-specific services. Such arobot has to have various functions such as person detection
and identification. There are many works on human detection
and tracking using LIDAR (Laser Imaging Detection andRanging) [1], [2], images [3], [4], or depth cameras [5], [6].
These works are mainly concerned with human position andmotion. In addition to these kinds of information, humanbody orientation could provide additional information suchas a possible direction to move. Therefore this paper deals
with body orientation estimation using LIDAR for mobile
robots.
Glas et al. [7] developed a method of estimating bodyorientation by using multiple LIDARs observing a humanfrom various directions. They use a person model with arms
and estimate the pose by combining model-based shapeprediction with laser scan data. Matsumoto et al. [8] proposed
to use sectional-contour for determining human posture suchas standing, sitting, and bowing, obtained from multiple
LIDARs. Since the LIDARs in these works are supposed
to be fixed at different positions, they cannot be directly
applicable to mobile robots. Weinrich et al. [9] developed amethod of classifying human upper body orientation using
HOG [3] and an SVM decision tree. The resolution of imagebased orientation estimation is usually not as high as LIDARbased methods.
When a person walks, his/her moving direction and bodyorientation are correlated to some extent, and this correlation
generally becomes larger at a higher speed. Chen et al.
[10] used this observation in a combined person tracking
and body orientation estimation using particle filter in visualsurveillance. Ardiyanto and Miura [11] dealt with a similar

problem by appearance-motion integration using UKF (unscented Kalman filter). Svenstrup et al. [12] used an observation that a person walking at a higher speed changes its
orientation less probably for making the orientation estimate
more accurate, although they did not use appearance nor
shape information.
In this paper, we propose a method of estimating humanbody orientation by integrating shape information obtainedby a LIDAR and motion information obtained by a UKF (unscented Kalman filter) tracker. The shape-motion integration
part is based on our previous method [11].
The rest of the paper is organized as follows. Sec. II
describes a shape-based orientation estimation using a 2D
LIDAR. Sec. III describes our UKF formulation in shapemotion integration for more reliable body orientation estimation. Sec. IV shows experiments for accuracy validationand those in human tracking situation. Sec. V concludes thepaper and discusses future work.
II. S HAPE -BASED B ODY O RIENTATION E STIMATION
A. Body shape models
We use a visible part of the section of human bodyobserved by a 2D LIDAR for orientation estimation. We
make a set of such data in advance as model data. The
set is generated by observing a human body at a certain
height from omnidirectional viewpoints. Supposing a robot
is following an adult, we put a LIDAR at the height of 85 cm
and at 2 m distance. We take data at 36 viewpoints with
10 deg interval. We compare an input shape data with thosein the database to estimate the body orientation.
We developed a data acquisition system, which is composed of a motorized turn table with a visual angular position
estimation. The accuracy of turn table angle measurement isless than one degree.
B. Body orientation estimation by shape matching
The first step of body orientation estimation is to match an
input shape with those in the model database. All models andinput data are represented as binary images. Fig. 1(a) showsa scene of obtaining a scan and its image representation, in
which one pixel corresponds to a 1 cm × 1 cm square region.
An input image is compared with a set of images in the
database. Since each model image is specific to a certain

weight (normaized)(a) Observation of body shape (green line)(b) Model image afterand its binary image representation.

applying distance transform.

orientation (deg.)

Fig. 2.
Fig. 1.

An example histogram of normalized weights.

Shape model acquisition and representation.

viewing direction, only translation is considered in matching
the input image and each model image. Here we use its
simplified version, that is, calculate the sum of the two
distances: dm is the averaged distance from each input edge
to its nearest model edge, and di is that from each model
edge to its nearest input edge.
After aligning the input and the model image with theircenter-of-mass positions, we search a certain region (currently, 7 × 7 region) for the minimum-distance displacement.
This can be viewed as a simplified version of the Hausdorff
distance-based edge matching strategy [13].
One way to determine the body orientation is to choose
the model image with the lowest distance and thus the corresponding viewing angle. This, however, has two drawbacks.
One is that the estimated angle become discrete and theother is the sensitivity to noise. We therefore introduce aweighting function which converts a distance to a weight andcalculate the weighted mean as an estimate. We also calculate
the variance of the estimation; such a variance informationis necessary for being used in a statistical framework like
Kalman filter. We use the following equations to calculate
the estimated orientation θ and its variance σθ2 :
wi

=

θ

=

σθ2

=

e−κ(dm +di ) ,N
i=1 wi i,
N
i=1 wiN2
i=1 wi (i − θ),N
i=1 wi

(1)
(2)
(3)

where i is a discrete orientation (i = 1, · · · , N ; N = 36), wi
is the weight for i. Fig. 2 shows an example histogram ofnormalized weights for an observation. Note that the weights
are calculated for orientations in the ±40 deg range fromthe previous orientation. At the same time, the periodicity
of angles is taken into account to compute a circular meanfrom directional statistics.
C. Evaluation of body orientation estimation using one scanof LIDAR data
We here evaluate the accuracy of shape-based body orientation estimation, by considering several factors which
may affect to the accuracy, that is, distance, body size, andclothing.

(a) LIDAR scan (2m)Fig. 3.

(b) LIDAR scan (5m)Comparison of LIDAR data for a person at different distances.

1) Effect of distance to the person: The number ofmeasured 3D points decreases as the distance to the person
becomes larger. We compare the accuracy when the person
is at either 2 m and 5 m distant from the robot. Fig. 3 shows
an example of LIDAR data for these distances.
For the person with whom a set of model data was
acquired, we acquired data set for testing for each orientation,
ranging from 0 deg to 350 deg with 10 deg interval (as in
the case of model data acquisition), and performed scan-toscan comparisons to estimate the body orientation. For each
set of input LIDAR scans, the discrete orientation which is
supported by the highest number of estimations is selected as
the result. Considering the estimation is sometimes reversedby 180 deg because the body shape looks very similar observed from the front and the back, we classify the difference
between the input and the model into the five cases: 0 deg,
±10 deg, 180 deg, 180 ± 10 deg, and otherwise. Table I
summarizes the results. Since we can use face recognition
for discriminate 0 degree and 180 degree cases, as explainedlater, we can conclude that even in 5 m cases, the orientation
error is expected to be mostly within 10 deg
2) Effect of clothing: Changing clothing may change thebody shape. We compared three cases shown in Fig. 4.
Using clothing A as the model and the others for testing, weestimate the mean and the maximum absolute error. Table IIsummarizes the results showing clothing change can decrease
the accuracy of the body orientation estimation, however, theproposed method still works well.
3) Effect of person to person difference: We take shapedata for three persons with the height of 160 cm, 170 cm and180 cm. For each person, we collected two sets of shape data
for all orientations, one is for making a model and the otherfor testing. We then calculate the absolute estimation error for
every combination of the model person and the test person.
Table III summarizes the result showing the height difference
between model and test data has a bad influence on the bodyTABLE I

TABLE IIC LASSIFICATION OF ESTIMATION RESULTS FOR DIFFERENT DISTANCE
TO THE PERSON .

T HE MEAN AND THE MAXIMUM ESTIMATION ERROR FOR DIFFERENT
CLOTHING WHEN CLOTHING A IS USED AS MODEL .

difference (deg)person is at 2 mperson is at 5 m

±10
7
10

0
27
16

180
2
6

180 ± 1003others01

Clothing for testing
mean absolute error (deg)maximum absolute error (deg)clothing B
3.21
8.21

clothing C
5.96
17.0

TABLE IIIC OMPARISON OF ERRORS ( IN DEG .) FOR VARIOUS COMBINATIONS OF
MODEL AND TEST PERSON .

model person
A

Bclothing A

Fig. 4.

clothing B

clothing C

C

Three different clothing.

Averagedorientation estimation. However, the table also include thecase where the averaged image of all model images is usedas the model, which is shown to reduce the errors in average.
The result is similar to the case where the subject wears
different clothing.

Integration of shape and motion is performed by introducing body orientation terms and information integration rules
into a human tracker. The tracker uses the Unscented Kalman
Filter (UKF) [14], which is a non-linear statistical filter andknown to be more accurate in many cases than Extended
Kalman filter. Another superior point of UKF is that it does
not require Jacobians, which are sometimes hard to derive.
B. Formulation
We basically follow the formulation by Ardiyanto andMiura [11] but modify it so that the robot-centered coordinate
system is used.
1) State transition equation: State vector xt is defined
by:

T
S
,
xt = xt , yt , ẋt , ẏt , φMt , φt , θt(4)where (xt , yt ) and (ẋt , ẏt ) are the position and the velocity
Sof a person in the robot local coordinate, φMt and φt are
the estimated orientation by the motion information and theshape information, respectively, and θt is the orientationSestimation result that integrates φMt and φt as described later.
Fig. 5 shows the relationships between the robot and theperson positions and the robot motion. Supposing a constantvelocity model, the person position/velocity part of the state

averaged error
2.96
3.98
8.32
4.26
1.70
6.09
6.66
3.95
2.86
3.33
2.32
5.16

maximum error
5.98
8.48
15.1
10.7
3.35
15.3
17.9
7.52
6.41
10.4
4.41
12.3

equation is given by:

III. S HAPE -M OTION I NTEGRATION FOR B ODYO RIENTATION E STIMATION
A. UKF-based integration

test person
A
B
C
A
B
C
A
B
C
A
B
C

x̂t+1

= xt + ẋt Δt,

ŷt+1
ˆ
ẋt+1
ẏˆt+1= yt + ẏt Δt,= ẋt ,

(5)= ẏt ,

where Δt is the cycle time and ˆ· indicates the position/velocity in the robot coordinates at time t. Then, applying the coordinate transformation due to the robot motion
gives the following relationship:
xt+1 = (x̂t+1 − Δx) cos Δθ + (ŷt+1 − Δy) sin Δθ,yt+1 = −(x̂t+1 − Δx) sin Δθ + (ŷt+1 − Δy) cos Δθ, (6)ˆt+1 cos Δθ + ẏˆt+1 sin Δθ,ẋt+1 = ẋˆt+1 sin Δθ + ẏˆt+1 cos Δθ.
ẏt+1 = −ẋCombining eqs. (5) and (6) with additional noise terms, we
obtain:xt+1= (xt + ẋt Δt − Δx) cos Δθyt+1+ (yt + ẏt Δt − Δy) sin Δθ + x ,= −(xt + ẋt Δt − Δx) sin Δθ

ẋt+1+ (yt + ẏt Δt − Δy) cos Δθ + y ,= ẋt cos Δθ + ẏt sin Δθ + ˙x ,ẏt+1= −ẋt sin Δθ + ẏt cos Δθ + ˙y .

(7)For orientation elements, we use the following equations:−ẋt sin Δθ + ẏt cos ΔθφM=
arctan+ φM (vt ),t+1
ẋt cos Δθ + ẏt sin Δθ
φSt+1 = φSt − Δθ + φS ,
(8)
−vt 

1
−
e
θt+1 = φSt + ω
φM − φSt + θ ,1 + e−vt t

Xt+1

xt+1

position at time t+1

Xt
xt

position at time t

Δy

yt+1

Δx

Yt+1
Yt
Fig. 5.

Δθ

Fig. 6.

yt

Accuracy evaluation environment with VICON.

Robot and person motion and coordinate transformation.

where vt is the speed of the person calculated from (ẋt , ẏt )and ω is a constant. The last line in eq. (8) realizes theshape-motion integration; as the person’s speed increases,
motion information is used more for orientation estimation.
Constant ω controls the influence of the speed to orientation
estimation. When ω = 0, only shape information is used,while ω = 1, motion information is maximally utilized. We
will determine the value of ω experimentally, as described
are fixed Gaussian noise
later. x , y , ẋ , ẏ , φS and theta˙
terms. φM (vt ) follows a zero-mean Gaussian with variance
σv2 e−vt ; the error in motion orientation prediction becomes
smaller as the person speed increase.
2) Observation equation: The robot measures the person
LL
position (xLt , yt ) and the body orientation (φt ) from LIDAR
data; the observation vector y t is given by:


L
L T
.
(9)
y t = xLt , yt , φtThen the observation equation is defined as:⎞⎛
xt + Lx⎠y t = ⎝ yt + LyS
φt + φL(10)

φSt and the variance of φL are given by the weighted mean
and the weighted variance as explained in Sec. II-B.
IV. E XPERIMENTAL R ESULTS(a) straight movement.

Fig. 7.

(b) circular movement.

Example motion measurements.

estimation accuracy for the straight and circular motion. Fig.
8 summarizes the averaged error for various ω’s. Larger ω’s
are effective for the straight motion while smaller values
are effective for the circular one. Based on this result, we
determined to use 0.3 as ω.
Table IV compiles the accuracy data for all motion patterns. For the straight and the circular motion, we also
compare the estimation only with shape information and thatwith shape-motion integration. Since the experiments weredone in the environment which is different from the one
for generating shape models, the averaged error for “standstill” is considered as offset between the environments, andsubtracted from the averaged errors in the other motion
patterns. The results show that shape-motion integration iseffective especially when a person walks relatively straightly.

A. Accuracy evaluation with motion capture device
Sec. II-C has described evaluation results for the casewhere one scan of LIDAR data is used, that is, those for the
shape only estimation. To evaluate the effect of shape-motion
integration, we used a motion capture device (VICON M2
Camera system) for measuring the body orientation on-line.
Fig. 6 shows the experimental environment. Three markers
are attached to a subject at the waist level. From the positionsof the markers, we can measure the body orientation as well
as body position. Four motion patterns were tested: standstill, turn, straight, and circular. Example motion measurements were shown in Fig. 7.
We first examined the effect of ω in eq. (8), which controls
how shape and motion information are integrated, on the

B. On-line tracking and orientation estimation experiments
1) Implementation on the robot: We implemented the
proposed method on a real mobile robot, GRACE-I. The
robot can detect and follow a target person while estimating
his/her body orientation. Fig. 9 shows the configuration ofprocessing steps.
GRACE-I is equipped with four LIDARs (Hokuyo UTM30LX) at the heights of 30 cm and 85 cm so that 360 deg
scans at two levels are obtained. These two heights arechosen to correspond to the body and the feet positions. The
robot has cameras for person identification based on face,
clothing, and other appearance features.
We measured the actual processing time in the case whereaveraged error [deg.]

straight

circularshape-basedorientation estimation

Person tracking andshape-motion integration

person detection

odometry

body shape extraction

motion estimation

shape-based estimation

shape-motion integration
using UKF

LIDARsshape model databaseweight ω
0 [deg]10 [deg]

20 [deg]...

Effect of ω on the body orientation estimation accuracy.

Fig. 8.

TABLE IV
E RRORS IN VARIOUS MOTION PATTERNS ( IN DEG .).

forwardcircularmotion pattern
stand stillturnshape only
shape-motion integration
shape only
shape-motion integration

averaged error
6.34
7.40
8.70
7.28
12.2
11.2

std. dev.
0.83
4.89
7.01
5.90
8.88
8.74

three to four persons are exist in the environment. The
average processing time for one frame was 37.6 msec andthat for the shape matching and the UKF update per person
was 2.39 msec.
2) Face recognition and orientation correction: As described in Sec. II-C, we use a face recognition method
to discriminate 0 degree and 180 degree cases. We apply
the Viola-Jones face detector [15] with our illumination
normalization method [16] for detecting faces in various
illumination conditions. When a face is detected in the predicted head region, and when the estimated body orientation
is within the range between −30 deg and 30 deg as shownin Fig. 10, we reverse the orientation, that is, add 180 deg tothe estimation result.
3) Tracking and orientation estimation: Fig. 11 shows
snapshots of tracking a person with estimating his bodyorientation. The left column indicates the images from acamera on the robot, in which cylinders and arrows indicate
the person positions and body orientations, respectively. The
right column indicate the maps with person information.
Circles, red one among them, and arrows indicate detected
persons, the target person, and the target’s body orientation,
respectively. In Fig. 11(a), the target person is standing
still and facing to the robot. In Fig. 11(b)-(d), the robot isfollowing the target person and his body orientations are wellestimated.
Fig. 12 shows the results of tracking and body orientation
estimation of multiple persons. The top-right corner of theimages shows the 2D mapping of persons and obstacles. The
results show the body orientations are well estimated.
V. C ONCLUSIONS AND F UTURE W ORKThis paper has described a body orientation estimation
method by shape-motion information. The shape information
is extracted from a 2D LIDAR scan and then compared with

Fig. 9.

position, velocity, andbody orientation

Outline of tracking and orientation estimation.
ypersonrobot

Fig. 10.

x-30+30 [deg]

Orientation range for reversing the measured orientation.

the shape models to generate an orientation estimation with
its variance. Considering the fact that the body orientationand the motion direction are correlated to some extent,
the shape-based orientation information is combined withmotion information in a UKF framework to produce a betterorientation estimate. We evaluated the method with a motion
capture system as a ground truth and have shown the shapemotion integration increases the accuracy of body orientation
estimation. The method is also implemented on a real robot
and successfully applied to a following robot scenario.
Although we have examined the effect of body size andclothing on the estimation accuracy, the effects of larger
disturbances such as belongings (e.g., shoulder bag or backpack) have not been evaluated. In such a case, if the changeof the body shape at the height of 2D LIDAR scans is large,
the estimation could be much worse. We are also planning
to use a 3D LIDAR so that informative parts of scans canbe selectively utilized for a better estimation. Application ofbody orientation estimate to recognition of various humaninteractions such as two persons talking closely is also future
work.
ACKNOWLEDGMENT
This work was supported by JSPS KAKENHI Grant
Number 25280093.
R EFERENCES[1] K.O. Arras, O.M. Mozos, and W. Burgard. Using Boosted Features
for the Detection of People in 2D Range Data. In Proceedings of the2007 IEEE Int. Conf. on Robotics and Automation, pp. 3402–3407,
2007.
[2] Z. Zainudin, S. Kodagoda, and G. Dissanayake. Torso Detection
and Tracking using a 2D Laser Range Finder. In Proceedings ofAustralasian Conf. on Robotics and Automation 2010, 2010.
[3] N. Dalal and B. Briggs. Histograms of Oriented Gradients for Human
Detection. In Proceedings of 2005 IEEE Conf. on Computer Vision
and Patttern Recognition, pp. 886–893, 2005.

(a) scene 1.
(a) scene 1.

(b) scene 2.
(b) scene 2.

(c) scene 3.

(c) scene 3.

(d) scene 4.

Fig. 12.

Tracking and body orientation estimation of multiple persons.

(d) scene 4.

Fig. 11.

Tracking and body orientation estimation by the robot.

[4] J. Satake and J. Miura. Robust Stereo-Based Person Detection andTracking for a Person Following Robot. In Proceedings of ICRA-2009
Workshop on Person Detection and Tracking, 2009.
[5] L. Spinello and K.O. Arras. People Detection in RGB-D Data. In
Proceedings of the 2011 IEEE/RSJ Int. Conf. on Intelligent Robots
and Systems, pp. 3838–3843, 2011.
[6] M. Munaro and E. Menegatti. Fast RGB-D People Tracking forService Robots. Autonomous Robots, Vol. 37, No. 3, 2014.
[7] D.F. Glas, T. Miyashita, H. Ishiguro, and N. Hagita. Laser-Based
Tracking of Human Position and Orientation using Parametric Shape
Modeling. Advanced Robotics, Vol. 23, No. 4, pp. 405–428, 2009.
[8] T. Matsumoto, M. Shimosaka, H. Noguchi, T. Sato, and T. Mori. Pose
Estimation of Multiple People using Contour Features from Multiple
Laser Range Finders. In Proceedings of 2009 IEEE/RSJ Int. Conf. onIntelligent Robots and Systems, pp. 2190–2196, 2009.
[9] C. Weinrich, C. Vollmer, and H.-M. Gross. Estimation of Human
Upper Body Orientation Estimation for Mobile Robotics using an
SVM Decision Tree on Monocular Images. In Proceedings of 2012
IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 2147–2152,
2012.
[10] C. Chen, A. Heili, and J.-M. Odobez. Combined Estimation of
Location and Body Pose in Surveillance Video. In Proceedings of 2011
IEEE Int. Conf. on Advanced Video and Signal-Based Surveillance, pp.

5–10, 2011.
[11] Igi Ardiyanto and J. Miura. Partial Least Squares-based Human Upper
Body Orientation Estimation with Combined Detection and Tracking.
Image and Vision Computing, Vol. 32, No. 11, pp. 904–915, 2014.
[12] M. Svenstrip, S. Tranberg, H.J. Andersen, and T. Bak. Pose Estimation
and Adaptive Robot Behavior for Human-Robot Interaction. In
Proceedings of 2009 IEEE Int. Conf. on Robotics and Automation,
pp. 3571–3576, 2009.
[13] D.P. Huttenlocher, G.A. Klanderman, and W.J. Rucklidge. Comparing
Images using the Hausdorff Distance. IEEE Trans. on Pattern Analysis
and Machine Intelligence, Vol. 15, No. 9, pp. 850–863, 1993.
[14] S.J. Julier and J.K. Uhlmann. Unscented Filtering and NonLinear
Estimation. Proceedings of IEEE, Vol. 92, No. 3, pp. 401–422, 2004.
[15] P. Viola and M.J. Jones. Robust Real-Time Face Detection. Int. J. of
Computer Vision, Vol. 57, No. 2, pp. 137–154, 2004.
[16] Bima Sena Bayu Dewantara and J. Miura. Fuzzy-based Illumination
Normalization for Face Recognition. In 2013 IEEE Workshop on
Advanced Robotics and its Social Impacts, pp. 131–136, 2013.