Sensor Fusion based Measurement of
Human Head Motion
Satorn E n w a

S~LSWIHL
Tuchi

Research Center for Advanced Scieiice illid Teclinology
University of Tokyo
4 G -1 I<oiiial)a,Meguro-ku, Toky;, 153, JAPANTEL: +81-3 3481-4468, FAX: +81 3-3481 4580
A b s t r a c t : Present V R systems often u s e unc o n s t r a i n e d sensors like P o l h e m u s Tracker. These
have advantages of unconstrainedness and w i d e r
motion range, b u t have deficiencies of low sampling rate(- 60Hz)and critical l a t e n c y (- 100ms).
T h i s paper proposes a method, which raises the
sampling rate and compensates the l a t e n c y of Polh e m u s on r o t a t i o n a l m o t i o n t h r o u g h the “sensor
fusion” of g y r o sensor, and P o l h e m u s measure-ments and a method of evaluating the p e r f o r m a n c eof u n c o n s t r a i n e d motion sensors quantatively. T h evalidity of the methods w a s confirmed b y off-line
c o m p u t a t i o n of a c t u a l data.
1 Introduction
Motion sensors used in present VR systems are classified into two types. One is tlie constrairied link-type
sensor like BOOM, and the other is the uiicoiistrained
seiisor like Polhemus Tracker. Latter lias advantages ofuiicoiistrainedness and wider motion range but lias deficiencies of low sampling rate(- 60Hz) and critical latency(- 100ms). Most present VR systems use raw s&isor datain spite of decrease in interactiveness by tliis latency or
try ad-hoc methods in order t o compensate the latency
such as liiiear extrapolation from both sinootlied iiieasuremeiits mid crude estimates of its instantaneous differentialof user’s motion.
Friedmain et al. [l]pointed out that in such approachuser’s quick motioii causes poor prediction which overshoots or underslioots the actual motion and that users
are forced to make slow deliberate motions. They solved
tliis problem on translational niotioii by mealis of prediction based on optinid linear estimation theory. However
they ignored the rotational motion, which is far criticalfor HMD, and their solution is not directly applicable torotatioiial motion because of its non-linearity.
This paper deals with rotational motion of liuman
head. We propose a nietliotl whirl1 raises the sampling
rate and conipwsates tlie latciicy through “sensor fusion”
of gyro sensor aiitl Polliemiis Tracker based on optimal
prediction. Althogli gyro sensor lias advaiitagcs of highsampling rate ant1 iirgligible latciiry and dificeiicy of drift,
integration of Pollieinus and gyro sensor by the proposed
inetliod has showii tlie bet tcr perforinnrice as tlie total
sensing system coiiipared with Polhemus Tracker or Fastrak only. In order to apply tlie proposed method to actual motion data of 3DOF(Degree of Freedom), we derived tlie dynamic niotlel of liunian head rotaioiial motion
described by Eulcr angle aiid confirnied tlie validity of thismethod by off-line computation.
We also propose a nietliod wliich quaiititatively evaluates sensor’s delay and fidelity by iioriiiaized cross correlation between sciisor signal aid stmiclad sigiid. By thismehtod we caii niatlicniatically define tlie sensor delay astlie shift time to iiiatcli wave foriiis of both signals best.
We applied this iiietliod for quantitativcly evaluating the
fidelity of Polhemus Tracker and Fastrak uiider the effectof nietalic objects a i d various electric waves.
2

Sensor fusion based m e a s u r e m e n t
We define “sciisor fusioii” as realization of a virtual

single sensor, wliicli lias liiglier perforinanre comparedwith each sensor by processiiig singals from multilple variant sensors. Brcause of difference of seiisors’ sampling
frequency coinbiiiatioii of available sensors differs a t eachiiiomeiit. We propose to prrdict llunian liead motion by
optimal estiniatioii based on tlie data of available sensors
at each inoirient. This approcli aims at making full use

IEEE lnternatlonal Workshop on
Robot and Human Communlcatlon
0-7803-2002-6/94 $4.00 0 1994 IEEE
-

124 -

of tlie latest available sensors, combination of wliicli istime-variant.

2.1

Modelling

We adopt ZYX-Euler angle [2] as the expression ofhunilui lieatl motion. Let r = (y p a)’”be the Euler aiigleof a liuman liead a t time t , r’ be the Eulcr angle a t timet (It. Lvt w = (wr wy w , ) ~be angular vclocity of thehuman head in body coordinate.
Define F’ as below.

+

- sin

T’

= r + F’wdt

The Algoritluii [4] consists of two procedures. Procedure 1 is executed when both Polhemus and gyro measurements are available. Procedure 2 is executed when
only gyro measurement is available.
Procedure 1

sin (IF’Algorithm

2.2

(Y

(2)

xr,+l/n = Fxn/n

Then we obtain the process dynamics Eq.(2) expressed byangular velocity in body coordiilate and Euler angle.
Let dt be unit sampling interval and T be time lag

pn+l/nxn/n(12)= FPn/nFT= Xn/n-l+ GQGT(13)+ I~:[Y:- H Pxn/n-~](14)= Pn/ri-l - h-:HPPn/n-I(15)
KE = P,l/,I-l HPTIHPPll/,l-lH P T+ Rp]-‘(16)Pn/nof measureiiient of Euler angle. State vector J consists ofEuler angle and angular velocities like

Procedure 2= (7P a w z w yw e use the notation of T = (y p a)T and w = ( w z wy w , ) ~ .

Xn+l/n = F xi,/,

By this notation state vector I is expressed as (rTIwT)*.
System model is given by Eq.(4), where wn is a zeroniean white noise whose covariance is given by Q. Let
HI’ be ”wireinerit matrix corresponding to siiniiltaneous nieasureiiieiit of both Pollienius and gyro sciisor. Let

pn+1/,1 = FPn/nFT GQGT
xn/n = *xn/n-l + IiR[g: - Hyxn/n-11

(18)
(19)

= Pn/n-~- Ic:HPn/n-l(20)

HY be ii~easuremeiitinatrix corresponding to iiieasurenient of gyro seiisor only. When both Polhe&x, aid gyro
sensor are available, tlie measurement model is given by
Eq.(8), where up is a n additive nieasurerneiit noise and

RP is its covariance. When only gyro sensor is available,
the measurement iiiodel is given by Eq.(9) and U P RP arean additive measurement noise and its covariance.

F

I

(w)

(3)

+

pn/n

K:

= Pll/l,-lH yTIHYPn/,-lHy+ Ry]-‘(21)The algoritlim used iii procedures 1 anti 2 is tlie same asthat of discrete Iialinan filter [3].
3

Evalution of sensor delay

In tills section we propose a nietliod wliicli evaluates
sensor delay by noriiiaiaed rross correlation between sensor signal and staiidard signal. Set f ( t ) be sensor signal and y ( t ) be standard signal. Tlieii cross correlation
function ‘9fy(7)and norniaiacd cross correlation function
p j Y ( 7 ) are cdculatrd as below.

(4)

-

(17)

125 -

p ranges from -1 to 1. If sensor delay is At and f ( t ) Ng ( t + A t ) , p j g ( ~lias) a peak at T w At and tlie peak value
sliows the fidelity of f ( t ) to standard signal y ( t ) .
Suppose the noise on f ( t )be an additive rantloni one
and be uiicorrelatied to standard signal g ( t ) . Set signal
power S slid noise power N . Then 9//(0) E S N and+9,,(0)

Covariant iliatrices Q , RI’ and RY tletermine process
noise and measuremelit noise. Let I3 be 3 x 3 unit niatrix. We assume process and nieasurement noise to be
ilitlependeiit. Wc set, uiiit sainpling interval tlt = IO[nis].

= S. Then peak value is nearlySpeak value

(25)

(24)

By this N/S rate we quantatively estimate tlie effects ofmettalic object and various electric waves around the Polhemus.
4

Noise parameter

4.2

Experiment

4.1 Experimental system

wl,w2 are determined as the standard deviation ofremnant error when fitting the data of constrained highprecision link-type sensor [5] to the system niotlel(Eq.4).
we used w1 = 0.0l[rad], w2 = O.lO[rad/s].
For fine prediction we must roughly estimate 1’1 andw2. Measurement noise of angular velocity

In order t o evaluate performance, we compared theresult of the proposed method with ineasureirient of linktype sensor [5]. The precision of link-type sensor (accuracy 0.0125 deg. resolution 0.025 deg.) is much better
than that of Polhemus Tracker (accuracy 0.1 deg. reso-.;I3= ( w - u’)(u - w*)T(28)

was deteriiiinecl by specification of gyro sensor. We used
v2 = O.U[rad/s].

lution 0.5 deg.) because rotation angle of each joint wasmeasured by high-precision optical encoder slid up-down
counter. Lead-time of this up-down counter 2 ps and CPU
performance determine the delay of this system. This system with 80286 (10MHz) measured operator’s inotion
and controletl the robot a t least 300Hz. So response andcommunication delay of this link-type sensor is negligilbecompared with Polhemus (- SO 111s). We rise measurement of this link-type sensor as standard signd.

We must determine measurement noise 211 taking delay of Polhemus Tracker T = SO[nis] into account. Let
describe measured value by adding * and value after T[s]by adding ‘. For example r‘ is the value TIS] after t
and T’* means the measurement of T’. Within intervalT, head moves froin T to T ‘ = T T F ’ u d r . T’* =++r.*+TF’u*. suppos:c the iiidividuality of r * - - f , u * - u , d r ,we calculate standard deviation of T ’ * - T ’ , wliicli is nothing other than ~ 1 1 3 . Let tlescribe covariaiice of u1 by(,./* - ,’)(,/* - T j ) T.

-

Intel 80286 lUMHzt = to+T~‘=t+TF’w+dt

UU( i: )T‘*- r*+ T F’ w*

Gym P Pdhrmlu(7’*Fig. 1 Configuration of experimental systeni-

126- T’)(T’* - p’)T=

- T ) ( T ’ - 7‘)T + dT d T T+dt2 (w’ - w;F‘TF’(W* - w)T
(P’

I

0.6

........... ............._._......]Link.....-...
Ahemus Tracker -------roposed method..............................................................................

Q- .......................................

0.4........................................

0.2......................................... <........................................

57

E

U

Q)

4

9

......................................

0
-0.2
-0.4

7

8

9

Time [SIPig.2 Standard signal(outputs of link-type sensor), raw Polhenius Tracker data and the result of proposed method - pitch angle-

We approximate P T F ' I because IF'( = I/ C O @ )1. we idso approximate dr drT by random walk.

N- T)(T* - r)T

0.013*1T
dr d r T = - W : I- dt(w' - @)(U' '- W ) T(U' - w)F'TF'(w* - w)T(7'

Table 1 Performance Coinparisoii by R M S Error. "Polhemus" denotes
raw data of Polheinus Tracker, *ICalinan" denotes result of Kalman flllered raw data of Polhenius Tracker, and "Proposed" denotes reault of
the proposed inehod.

RMS ( x lO-*[rad])

N

-

T

($w;

+

Kalinan
ProposedUi)IFinally we got 111 = 0.0252(rad].
4.3

6.65

7.09
4.00

4.19

ResultWe sanipled ineasurenient of gyro at 100Hz, that oflink-type seiisor at 50Hz and that of Polhenius Tracker
a t 50Hz for off-line computation. We used tlie output,of link-type seiisor as the standard signal. We coinparedthis in thc respect of RMS (Root Meau Square) error withthe result of the proposed method, raw data of Polhernus
Tkacker aiid I<dniaii filtered Pollieinus Tracker raw data.

-

Fig.2 sliows standard sigual, raw data of Polhemus
and the result of the proposed Inethod. Average delay of
Polheinus Tracker was nearly 80111s. It is apparent thattlie proposed inetliod compensated this delay well. Table l shows tlie RMS(Root Mean Square) error between
various outputs aiid the standard signal. R M S error of

Iialman filtered Polliclnus Trackrr data was l a g e r than

127 -

Gyro
PolhemusU@&

20

IHead Sensur

0 Time Update

40

[ms]
Kalman Filter

Fig.3 Timing chart of combination of available sensors

that of raw Tracker data because of overshoots and undershoots of its prediction. RMS error of the proposed
method reduced nearly half of raw Tracker data except
roll angle.
4.4

Fig.5 p of Polhemus Tracker arid p of h s t r a k uiider nearly ideal situation(with the IDOF penduluinn device

1

0.95

Evaluation by correlation

0.9

0.85

0.8

0.75

0.7
-0.15

-0.1

-0.05

0

lime (SI

0.05

0.1

0.15

0.2

Fig.6 rho of Polhemus Tracker, Fastrak and the proposed method under
actual situation(with metalic link-type sensor

standard signal. This device was constructed with plastic
parts and the least metalic parts. We designed the special

Fig.4 Original IDOF pendulumn device for performance check of Polhemus

mounts wliich enables to ineasure each element of Euler
angle (y p a) independently with meclianically fixed axisof the peridulurnn. The pendulumn was driven by handbecause electric motor generated much noise. But there is

We also evaluate the performance of Polhemus Tracker.
its improved version Fastrak and the proposed method by
the correlation-based method proposed in 52. We received

no problem because thhe frequency range of hand motion
is much wider than that of head motion.
Second we evaluate the performance of Polhemus Tracker,

data from Tracker and Fastrak via RS232C.
First we check their performaice under nearly ideal

Fastrak and proposed method under acutual situation,
where we use nietalic link-type sensor[5] as standard sig-

situation. We used the device consisted of a pendulumn,
potentiometer and mounts for transmitter and receiver of
Polhemus. We used the output of poteiitionieter as the

nal.

-

We received data from Tracker aiid Fastrak via RS232C.
The dealy of Pollieillus is 120111sand that of Filstrak is

128 -Table 2 Performance evaluation by correlation method under nearly ideal siluation

N/S rate [%)I

Delay [ms]Tracker
Fastrak

Y

P

40
0

-20

80

CY

Y

P

120
20

20.8
1.0

6.5
2.2

CY

G.3
0.4

Table 3 Performance evaluation by correlation method uiider ideal situation

N/S rate [%]Delay [ms]Y

Tracker
Fastrak
Proposed

P

a

Y

P

a

40
40
-20

80
20
0

56

3.3
17

5.2

23
56

2.0

0.2
4.1

20ms in Fig.5. Iii ideal situation we see the improved
performalice of Fastrak. Fig6 sliows the delay and thefidelity of Polliemus Tracker, Fastrak and the proposediiiethod under actual situation. Polliemus delay is about
40111s, that of Fastrak is 201ns and tliat of proposed method
is -10111s. Fastrak is improved in tlie respect of time delay ahout 20111s coinpared with Tracker, but there is still

Referencesollie delay. It is obvious that the proposed irietliod com-

[3] A.Gcll, et al. : Applied Optimal Estimation, THE[l] hl.Friediiianii, T.Starner and A.Peiitlaiit1 : ~ ~ T l C r O l l z % ~ -tion i n Virt,ual Realities, PRESENCE, 1, 1, pp139144, 1992[2] Joliii J. Craig : Robotics, Addison-Wesley, 1980

MIT Press, 1974

peiisates the delay of Pollietiius Tracker by tlie optiiiial
prediction with sensor fusion.
It is guessed from Table 3 that in actual situation rolleleineiit of <lilt&lial a bit. different wavc forin comparedwith stantlard signal because of tlic effect, of the shape ofthe large iii(!tallic liiik-type sciisor.
5[4] S.Emnra aiitl S.TiLchi : Sensor FUsiori Based Measurenierit o f HLlman Head Motion, Proceedings of the
32nd SICE Annual Confewiice, pp805-806, 1993 (inJapanese)[5] S.Titchi, H.Arai aiitl T . M w ~ : L: Tele-Enster~ceMaster Slave System For Renbote Muiaapulntion, IRoS
'90, 1)1>343-348,1990

Conclusion

We proposctl a metliod which iiiiproved the 1)erforniaiice of Pollicmus sensor by optimally integrating thenieasurenient of gyro sensor. We coiifirmed tlic validityof this method by off-line computation of actual data. We
also proposed a nietliod which made it possible to quantitatively evaluate the delay and the fidelity of unconstrained iiiotioii tracking sensor under various conditions.

-

129 -