UBICOMP '14 ADJUNCT, SEPTEMBER 13 - 17, 2014, SEATTLE, WA, USA

A Method for Tracking On-bodySensor Positions Utilizing Prior
Knowledge
Naoto Migita
Sozo Inoue
Kyushu Institute of Technology Kyushu Institute of Technology
1-1 Sensui-cho, Tobata
1-1 Sensui-cho, Tobata
Kitakyushu, 804-8550, JAPAN Kitakyushu, 804-8550, JAPANn.migita96@gmail.com
sozo@mns.kyutech.ac.jp

Takeshi Nishida
Takuya Yumiyama
Kyushu Institute of Technology Kyushu Institute of Technology
1-1 Sensui-cho, Tobata
1-1 Sensui-cho, Tobata
Kitakyushu, 804-8550, JAPAN Kitakyushu, 804-8550, JAPANtakuya.yumiyama@gmail.com nishida@cntl.kyutech.ac.jp

Abstract
In this research, we aim at estimating the positions anddirections of on-body mobile devices such as smartphones
with accelerometers and gyroscopes. We propose amethod utilizing prior knowledge of positions anddirections of sensors. We model the prior knowledge byneighborhood method trained from a motion capturingsystem, and combine with physical principles by Bayes’
theorem. To assess our approach, we developed a systemfor collecting acceleration and position data using an
accelerometer and motion capture, and experimented withdata obtained using it. In contrast to the conventional
method, the experimental result shows that the proposed
method stably follows a trajectory.

Author Keywords
Smartphone, Accelerometers, Gyroscopes, Tracking
On-Body Sensor Positions
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bear
this notice and the full citation on the first page. Copyrights for componentsof this work owned by others than the author(s) must be honored. Abstractingwith credit is permitted. To copy otherwise, or republish, to post on servers or
to redistribute to lists, requires prior specific permission and/or a fee. Request
permissions from Permissions@acm.org.
UbiComp ’14 , September 13 - 17, 2014, Seattle, WA, USA.
Copyright is held by the owner/author(s). Publication rights licensed to ACM.
ACM 978-1-4503-3047-3/14/09...$15.00.
http://dx.doi.org/10.1145/2638728.2641308

757

ACM Classification Keywords
I.4.8 [Scene Analysis]: tracking.

Introduction
Recently, activity recognition with sensor data has become
realistic by the spread of mobile devices such as
smartphones with accelerometers and gyroscopes. In thisresearch, we aim at estimating the positions andUBICOMP '14 ADJUNCT, SEPTEMBER 13 - 17, 2014, SEATTLE, WA, USA

directions of on-body mobile devices such as smartphones
with accelerometers and gyroscopes. In the literature,
many researches for recognizing several types of activities
exist in which the activities are represented in categoricalvariable, but few tackle with numeric ones, that is the
positions and directions of sensors on the body.
Since the sensors output diﬀerential values, estimating thepositions and directions corresponds to the integrated
values, which is challenging under noises. In this paper,
we propose a method utilizing prior knowledge of positions
and directions of sensors. If the device is on the body, we
can assume the the positions and the directions havespecific distributions. We model them by neighborhood
method trained from a motion capturing system, andcombine with physical principles by Bayes’ theorem.
To assess our approach, we developed a system forcollecting acceleration and position data using an
accelerometer and motion capture, and experimented withdata obtained using it. In contrast to the conventional
method, the experimental result shows that the proposed
method stably follows a trajectory.
In this paper, we firstly formulate the physical principles of
positions, directions, and sensor values of mobile devices,and describe the conventional estimation and proposed
method. In addition, we present an approach forcollecting acceleration and position data. Finally, a basic
assessment of the system is provided.

Related Work
Recently, the introduction of inexpensive motion capturesystems such as Microsoft Kinect has opened up a newline of active research [1, 2]. Kinect is used to estimate
the human pose from the depth image. We note that
there are methods that can estimate the position andorientation of sensors using portable sensors andcameras [4], and can detect faster movements with higher

758

accuracy than motion capture using a wearable wristband
device that combines aaccelerometers and gyroscopes [5].
In our research, we aim to estimate the human pose moreeasily using only accelerometers instead of Kinect. In
addition, research is carried out to convert data between
Kinect and IMU (Inertial Measurement Units) [3]. IMU isa device for measuring the movement of a person from asensor attached to the joint. We use Kinect in our
research too. However, our work diﬀers from the previous
research in the manner in which prior distributions aremodeled.
In robotics area, there are many methods to estimate
positions and orientations using prior knowledge [6–8].
We use prior knowledge to estimate positions andorientations too. However, the estimation target of ourresearch is diﬀerent from them.

Formulation
In this chapter, we describe the formulation of the statedquantity of the mobile device.
Coordinate system of the mobile device
We define a world coordinate system as Σ, and acoordinate system fixed to mobile devices as ΣD . Fig. 1
shows these relationships.

WORKSHOP: HASCA

Dh

y

upper

Dh

hz

gyroscope of mobile devices.

sm a r t -ph on ecoor din a t e syst em
z

Dh

D

ϕk = ϕk + v k

Here, the left shoulder D of each variable represents thatthe definition belongs to ΣD . v k ∼ N (0, σv2 ) is a
distribution that models observation noise. v k is theaverage value of 0 and represents that it follows aGaussian distribution of the dispersion σv2 . Next,
acceleration is observed as follows by an accelerometer
mounted on mobile devices as follows.

x

YD

hy
Y

(1)

lower

hx
wor ld coor din a t e syst emD

f k =D ak +D g k + wk

Figure 1: Phone coordinate system.

(2)

wk is the observation noise follow a certain distribution.
When R(θ) is defined as the Rotation matrix with Euler
angles θ and g is defined as the invariant gravitational
acceleration in k, the following equation is satisfied.

Further, the state vectors are defined as follows.
[]Thk ≜ hx,k hy,k hz,k ∈ Σ[]Tsk ≜ sx,k sy,k sz,k ∈ Σ[]Tak ≜ ax,k ay,k az,k ∈ Σ[]Tθ k ≜ θx,k θy,k θz,k ∈ Σ[]Tϕk ≜ ϕx,k ϕy,k ϕz,k ∈ Σ

ak = R(θ k ) · ak

(3)

g k = R(θ k ) · g

(4)

D

D

Here, x, y and z are coordinate axes, and k is discrete
time. Further, hk , sk and ak are the location of thecenter of gravity, velocity and acceleration of mobile
devices, respectively. Moreover, θ k and ϕk are therotation angular velocity and the rotation angle of ΣD toΣ respectively. h0 , s0 , a0 , ϕ0 and θ 0 are assumed to be
known.
Relationship of State Quantity
Angular velocity and acceleration of mobile devices are to
be measured at sampling intervals ∇[s]. First, the angularvelocity of mobile devices in Σ is observed by the759

Further, the velocity and positions of the center of gravity
of mobile devices, and the change of angle can beapproximated using the Euler method by backward
diﬀerence as follows.
sk = sk−1 + ∇ · ak
hk = hk−1 + ∇ · sk

(5)
(6)

θ k = θ k−1 + ∇ · ϕk

(7)

A Method for Tracking Holding Position of
the Mobile Device
In this chapter, based on the formulation statedpreviously, a method for tracking the holding position of

UBICOMP '14 ADJUNCT, SEPTEMBER 13 - 17, 2014, SEATTLE, WA, USA

mobile devices is proposed. In essence, we describe amethod for estimating hk when angular velocity Dϕk andacceleration Df k are observed given that hk−1 , sk−1 andθ k−1 are known. In the following, we present an overview
of the proposed and conventional methods.
Tracking the holding position of the mobile device using
the conventional method:
1. Estimate acceleration ak from angle θ k−1 and theobservations Df k , Dϕk by linear regression.
2. Estimate value of the displacement from acceleration
estimated ak and the displacements in the pasthk−1 , hk−2 by double integration.
Tracking the Holding Position of the Mobile Device using
the Proposed Method:
1. Collect L neighbor samples (hlk−1 , hlk−2 ) ofdisplacements in the past displacements(hk−1 , hk−2 ) from the training data that relate todisplacement.
2.

Calculate alk for the samples (hlk , hlk−1 , hlk−2 ) in eachthe sample l by calculating the diﬀerence between
two times the displacements.

3. Estimate the observed values Df lk , Dϕlk and the angle
θ lk−1 by performing a linear regression for alk ineach sample l.

Mobile Device State Estimation using the Conventional
MethodAssuming that the angular velocity Dϕk and accelerationDf k have been observed in hk−1 , sk−1 , and θ k−1 statesin which a is known. At this time, displacement hk ,velocity sk and angular velocity θ k at time k can bederived analyticallwy in the following manner.
By substituting the ϕk from (1) in (7), we getθ k = θ k−1 + ∇ · ϕk = θ k−1 + ∇ · (Dϕk − v k )By substituting ak from (3) and Dak from (2) in (7) in arow, we get
sk = sk−1 + ∇ · ak = sk−1 + ∇ · R−1 (θ k ) ·D ak= sk−1 + ∇ · R−1 (θ k )(Df k −D g k − wk )= sk−1 − ∇ · g k + ∇ · R−1 (θ k )(Df k − wk )

760

(9)

By substituting this formula in (6), It is possible to obtain
the hk neglecting noise by. However, the recurrence
formula is solved as follows.
θk = ∇ ·k∑(Dϕi − v i )i=1

sk = ∇ ·k∑

R−1 (θ i )(Df i − wi )i=1

hk = ∇ ·k∑i=1

4. Estimate hk by the k-neighborhood method for theobservation values (θ k−1 ,D f k ,D ϕk ) using thetraining data for L (θ lk−1 ,D f lk ,D ϕlk ).

(8)si = ∇2 ·k ∑i∑

R−1 (θ j )(Df j − wj )i=1 j=1

Considering only the influenceof noises v i and wi the∑keﬀect of the order of i=1 v i = O(k) is stored in the
angularθ k and the eﬀect of the order of∑k ∑velocityi−1(θ j ) · wj = O(k 2 ) is stored in the
i=1
j=1 R
displacement hk . This makes it diﬃcult to ensure
accuracy.

WORKSHOP: HASCA

Implementation of the Conventional Method
In the following, we describe the implementation of theconventional method and compare it to the proposed
method as described in the next section.
Given that hk−1 , sk−1 and θ k−1 are known, theacceleration is observed, and the following procedures are
performed.
1. Calculate acceleration ak from θ k−1 and obtain theobserved values Df k , Dϕk as
ak ← αA(θ k−1 ,D f k ,D ϕk ) by linear regression
given that α is the regression coeﬃcient vector and
A is a function defined byA(θ, f , ϕ) = R(θ + ∇ϕ)f .

• a in step 2, using (5)-(6),hk = hk−1 + ∇sk = hk−1 + ∇(sk−1 + ∇ak )= hk−1 + ∇((hk−1 − hk−2 )/∇ + ∇ak )= 2hk−1 − hk−2 + ∇2 ak
Then, explaining using conditional probability.
P (hk |hk−1 , hk−2 , θ k−1 ,D f k ,D ϕk )Using the addition theorem,∫=
P (hk |hk−1 , hk−2 , θ k−1 ,D f k ,D ϕk , ak )ak·P (ak |hk−1 , hk−2 , θ k−1 ,D f k ,D ϕk )dak

2. Ealculate the estimate of the displacement
As seen from the derivation of the above equation, we
assume conditional independence between (hk−1 , hk−2 )and (θ k−1 ,D f k ) under ak ,∫=P (hk |hk−1 , hk−2 , ak )P (ak |θ k−1 ,D f k ,D ϕk )dakak(10)

hk ← 2hk−1 − hk−2 + ∇2 akfrom ak as determined, hk−1 and hk−2 .

The derivation of these equations is as follows.

In (10), the average of the second factor calculated using
step 1 and the first factor is calculated using step 2.

• a in step 1, by (2)-(4),ak = R−1 (θ k )Dak
= R(−θ k )(Df k −D g k − wk )= R(−θ k )(Df k − wk ) − g
From (1) and (7)ak = R(−θ k−1 − ∇(Dϕk − v k ))(Df k − wk ) − g
ak is modeled With the exception of the constantterm and noise.

761

State Estimation of the Positions of On-Body Mobile Devices Utilizing Prior Knowledge
In this section, we propose a method capable of providing
information to estimate the prior distributions of variables
and improve the accuracy of state estimation by theneighborhood method. In advance, it should beconstructed prior distribution of displacement and velocity
using motion capture such as Kinect sensor.
Accumulation of the noise is suppressed by giving the

UBICOMP '14 ADJUNCT, SEPTEMBER 13 - 17, 2014, SEATTLE, WA, USA

estimated time it. It is assumed that hk−1 , hk−2 , θ k−1 ,f k and Dϕk are known. By Bayes theorem,DP (hk |hk−1 , hk−2 , θ k−1 ,D f k ,D ϕk )∝ P (θ k−1 ,D f k ,D ϕk |hk , hk−1 , hk−2 )P (hk |hk−1 , hk−2 )Then, from the addition theorem,∫=
P (θ k−1 ,D f k ,D ϕk |hk , hk−1 , hk−2 , ak )• At P (hk |hk−1 , hk−2 ) which is the third factor,
sampling is executed by the neighborhood method.
If A of one of (hl , hl−1 , hl−2 ) is sampled by
preparing the training data set in advance, this
distribution follows approximately follows
P (hk , hk−1 , hk−2 ). By utilizing this concept, thesamples according to P (hk |hk−1 , hk−2 ) are
obtained by sampling the neighborhood of(hk−1 , hk−2 ) from H.

ak·P (ak |hk , hk−1 , hk−2 )dak
·P (hk |hk−1 , hk−2 )As with the previous section, assuming conditional
independence between (hk−1 , hk−2 ) and (θ k−1 ,D f k )under the ak .
∫=P (θ k−1 ,D f k ,D ϕk |ak )P (ak |hk , hk−1 , hk−2 )dakak

·P (hk |hk−1 , hk−2 )In this formula,• P (θ k−1 ,D f k ,D ϕk |ak ) which is the first factor is
obtained from (θ k−1 ,D f k ,D ϕk ) = βak + β 0 by alinear regression. However, β, β 0 is the regression
coeﬃcient vector.
• P (ak |hk , hk−1 , hk−2 ) which is the second factor is
uniquely obtained by (5)-(6) as follows.
ak = (sk − sk−1 )/∇= ((hk − hk−1 )/∇ − (hk−1 − hk−2 )/∇)/∇= (hk − 2hk−1 + hk−2 )/∇2

Implementation of the Proposed Method using the Neighborhood Method
In the following, we describe the implementation of the
proposed method. When acceleration Df k and angularvelocity Dϕk have been observed and hk−1 , hk−2 , sk−1
and θ k−1 are known, this method performs the following
procedure.
1. Obtain L samples (hlk−1 , hlk−2 ) in the neighborhoodof the (hk−1 , hk−2 ) and hlk corresponding its fromlearning data H.This distribution of hlk follows
approximately the third factor P (hk |hk−1 , hk−2 ).
2. In accordance with the second factor, alculate
alk ← (hlk − 2hlk−1 + hlk−2 )/∇2 for each l.
3. In accordance with the first factor, calculate(θ lk−1 ,D f lk ,D ϕlk ) ← βalk + β 0 for each l by linear
regression.
4. Obtain M samples in the neighborhood of theobserved values (θ k−1 ,D f k ,D ϕk ) from L(θ lk−1 ,D f lk ,D ϕlk ), and to determine the average
1 ∑ m
h
M m k
of hm
k corresponding to it to estimated value of hk .

762

WORKSHOP: HASCA

Evaluation
In this chapter, we describe the data collection system andthe preliminary experiments in three-dimensional space.
Data collection system
We have developed a software for collecting data in theProcessing environment. Fig. 2 shows a screen image ofthe software.

Figure 2: A screen image of the data collection system

The value of the accelerometer is also received by the
UDP communication at the same time as the position
information from Kinect. This received data is stored to a
file is sampled at the same time as the position
information. Simultaneously, the audio and video also
stored. Although our system operates at an acceleration
frequency of 100 Hz, it synchronizes with Kinect, which763

operates at 30 Hz. We use iPodTouch as anaccelerometer. Our system on the iPodTouch incorporates
the HASC Logger1 , which is a toolkit for activity
recognition research.
Experiment
We performed a preliminary experiment to estimate thedisplacement from the horizontal movement of theaccelerometer with one human subject. The experiment
was carried out with a right hand to the acceleration
sensor. The sensor is held with the screen facing outward,that is, the left hand side of the sensor is set to be theX-axis positive velocity. After the start of the recording,the hand is moved at a slow pace in the transverse
direction for approximately one min.
Setup
We estimate prior distributions from acceleration anddisplacement data generated by the mobile sensor and theKinect, respectively. However, both mobile sensor andKinect data are prone deviations caused by sampling time
and the scale. However, we do not know the X-axis
direction of the acceleration sensor to verify to see if it isdirectly perpendicular to the force of gravity. Therefore,
we carried out a pretreatment the following as. We do notknow the X-axis direction of the accelerometer or to see if
it is perpendicular to the force of gravity exactly.
Therefore, Considered easily with gravity component
moving average of every second, it was smoothed(Hereinafter, this is a low-pass filter) by a weighted
moving average of 9:1 of the two samples, after
conducting the removal of the gravity component.
Orientation information obtained from the Kinect, sk is
obtained from the position information of the hand usind(6), and ak is obtained by (5). We used the low-pass
1 HASCTool, http://hasc.jp/

UBICOMP '14 ADJUNCT, SEPTEMBER 13 - 17, 2014, SEATTLE, WA, USA

X−axis Estimate
Y−axis Estimate
Z−axis Estimate

0

5000

X−axis Truth
Y−axis Truth
Z−axis Truth−10000[mm/s]

15000

Velocity (Conventional)

0

500

1000

1500

[k]
Figure 3: Estimated result of the speed according to the conventional method

filter before and after these stages. Further, we shifted
the time of the accelerometers so that thecross-correlation is maximized. Moreover, in order tosatisfy the assumption that hk−1 , hk−2 are known, theinitial value of the estimated value are aligned to h1 , h2 .
Building a Prior Distribution
The prior distribution of a variable is obtained from thedisplacement, the speed and the acceleration of Kinect.
Further, the acceleration of the mobile sensor wascomputed as described previously. To calculate the764

distribution, the first half of the data were used aslearning data. However, in the proposed method, L is set
to 500, and M is set to 125(please see previous chapter).
Experimental Result
Fig.3 shows the estimation result using the conventional
method. Fig.4 shows the estimation result obtained by the
proposed method.In the conventional method, the result
depicts a large displacement error. Therefore, we show afigure of the estimated result of the speed. In both
figures,the X-axis represents size of the data set. The

WORKSHOP: HASCA

−200
−500

[mm]

X−axis

Truth
0

500

600
500

Truth
0

500

1000

Estimate
1500

1900

Z−axis

1750

[mm]

1500

Y−axis

400

[mm]

1000

Estimate

Truth
0

500

1000

Estimate
1500

Figure 4: Estimated result of displacement by the proposed method

Y-axis in Fig.3 represents the value of the speed, whereas
in Fig.4, it represents the value of displacement. In theconventional method, the predicted value (Estimate) has
a large value that is significantly relative to the true value(Truth) in stage the speed. Therefore, when thedisplacement is considered, the misalignment width isexpected to further increase. Moreover, the Fluctuation
not be followed, so It cannot be said to follow well.
In the proposed method, any axes in the first half followed
a certain pattern to some extent. However, signification
deviations are observed near k = 200 at Y-axis. This

765

discrepancy is results from the failure to utilize priorknowledge to discover major changes. The second half,
although it is not able to follow to some extent to the
middle, the value is the slight deviation near the end. Thisis because it is used for learning the first half. Moreover,the direction of the estimated value has a smaller
amplitude on the whole. This is because this method
calculates the average of the estimated values at the end,
and therefore, it tends to yield a small estimate of thedirection-amplitude. In this manner, we confirmed that

UBICOMP '14 ADJUNCT, SEPTEMBER 13 - 17, 2014, SEATTLE, WA, USA

the proposed method can follow in a stable in comparison
with conventional methods

Conclusion
In this paper, we propose a method utilizing priorknowledge of positions and directions of sensors, andperformed an initial assessment. The result shows thatthe proposed method stably follows a given trajectorycompared to conventional methods.
The future plan is to consider high-dimensional data, anddetermine the accuracy of our approach. Moreover, we
expect to improve the accuracy and the computational
speed through the introduction of importance samplingsuch as a particle filter.
We use Kinect as a motion capture this time, so a
phenomenon that fail to capture often happened in case ofhand in front of the body. Another consideration that may
be of interest is to increase the accuracy of the motion
capture procedure, which we believe would be useful formodeling probability distributions. In In futureaddition, we
hope to optimize the L and M parameters.

Acknowledgements
The part of this research is basic research (B) Sensor
context estimation techniques by two levels of meaning
layer and physical layer (Principal Investigator: Sozo
Inoue) . I would like to thank Mr. Ikoma Norikazu of
Kyushu Institute of Technology who had useful advice on
this research.

References[1] Anderson, F., Grossman, T., Matejka, J., &Fitzmaurice, G. (2013, October). YouMove:enhancing movement training with an augmented

766

reality mirror. In Proceedings of the 26th annual ACM
symposium on User interface software and technology(pp. 311-320). ACM.
[2] Bin Umino; Asako Soga; Motoko Hirayama :Choreographic Support for Contemporary Dance using
a Body-Part Motion Synthesis System: Comparison of
Learning Eﬀect between Laptops and Tablets, IPSJSIG Technical Report, ITS, 2013, 2013.19: 1-6[3] Oresti Banos et al., : Kinect=IMU? Learning MIMO
Signal Mappings to Automatically Translate Activity
Recognition Systems Across Sensor Modalities,
ISWC2012.
[4] Yasutaka Shindo, Shingo Kagami, Koichi Hashimoto：
Position and Orientation Estimation of an Acceleration
and Angular-Velocity Sensor with Help of Human Pose
Information, The Institute of Electronics, Information
and Communication Engineers,ASN2013-4(2013-05)[5] Yutaka SAKANE, Masami TAKAHATA,Naofumi
OHTANI, Yoichi TAKEBAYASHI. (2003). An
Experiment of Martial Arts Analysis Using Multimodal
Sensing Techniques. interaction(2003).
[6] Metni, Najib, et al. ”Attitude and gyro bias estimation
for a flying uav.” Intelligent Robots and Systems,
2005.(IROS 2005). 2005 IEEE/RSJ International
Conference on. IEEE, 2005.
[7] Bristeau, Pierre-Jean, et al. ”The navigation andcontrol technology inside the ar. drone micro uav.”
18th IFAC World Congress. Vol. 18. No. 1. 2011.
[8] Kingston, Derek B., and Randal W. Beard. ”Real-time
attitude and position estimation for small UAVs using
low-cost sensors.” AIAA 3rd Unmanned Unlimited
Technical Conference, Workshop and Exhibit. sn,
2004.