Dealing With Sensor Displacement In Motion-Based
Onbody Activity Recognition Systems
Kai Kunze, Paul Lukowicz
Embedded Systems Lab, University Passau,
Innstr 43, 94032 Passau, Germany
www.wearable-computing.org
www.wearcomp.eu
kai.kunze/paul.lukowicz @uni-passau.de
An important question related to motion sensor based
activity recognition is sensor placement and displacement. The vast majority of research in this area assumes well defined, fixed sensor locations. This is particularly important for activity recognition related toarm and hand motions.

ABSTRACT

We present a set of heuristics that significantly increase
the robustness of motion sensor-based activity recognition with respect to sensor displacement. In this paper
placement refers to the position within a single bodypart (e.g, lower arm). We show how, within certain limits and with modest quality degradation, motion sensorbased activity recognition can be implemented in a displacement tolerant way. We first describe the physicalprinciples that lead to our heuristic. We then evaluate
them first on a set of synthetic lower arm motions which
are well suited to illustrate the strengths and limits of
our approach, then on an extended modes of locomotion problem (sensors on the upper leg) and finally ona set of exercises performed on various gym machines(sensors placed on the lower arm). In this example ourheuristic raises the displaced recognition rate from 24%
for a displaced accelerometer, which had 96% recognition when not displaced, to 82%.

Being able to drop the requirement for ’well defined
fixed position’ and build systems that can deal withsensor displacement has two major advantages:• Robustness. During long term deployment sensor
shifts cannot be avoided. Enabling the system tocontinue working correctly despite sensor shift is asignificant improvement to robustness.
• Better usability and user acceptance. Today, many
mobile appliance are already equipped with sensors.
Sensor encapsulation into clothing or unobtrusive attachment e.g, as ’buttons’ has been demonstrated[13]. It is thus often taken for granted that users
can be easily equipped with sensors in every day situations. However, this does not imply that the user
can be expected to reliably and firmly fix the sensors
to narrowly defined on-body locations.

Author Keywords

Sensor Displacement, Opportunistic Activity Recognition, Motion Sensors, Fitness Exercises
ACM Classification Keywords

Problem Specification

I.5.2 Design Methodology, Feature evaluation and selection

The problem of on-body sensor placement can be decomposed into three sub-problems: (1) ’body part’ placement, (2) sensor orientation, and (3) exact position
within a body part,

INTRODUCTION

Motion sensors, in particular accelerometers, are a common type of body worn sensors for activity recognition.
Following the original work by Randell [12], Van Laerhofen [15] and Mantyjarvi [8] there have been numerous publication dealing with applications ranging fromdance [1] through sign language recognition [2] to tracking of every day activities [5, 14] to industrial maintenance [11] and mental health related applications [17].

’Body part’ placement.

A user can carry mobile appliances such as phones andMP3 players in distinctly different body locations. The
devices can be in a front or side pocket, attached to abelt, or in a holder on the upper or lower arm (e.g, during exercise). Except for trivial recognition problems(e.g, distinguishing walking from standing) a motion
based recognition system trained on one body part will
not work on another. However, we show that it is possible to reliably recognize the body part location of anaccelerometer [4]. At the same time, for most appliances, there are a few body parts on which they can beplaced. Thus, a system can be trained for several body
locations and the appropriate version can be selectedon the basis of the recognized location.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.
UbiComp’08, September 21-24, 2008, Seoul, Korea.
Copyright 2008 ACM 978-1-60558-136-1/08/09...$5.00.

20

implemented in a displacement tolerant (within bodypart) way. Thus, within a single body part, we demonstrate reliable recognition at locations different fromthose on which the sensor was trained. The idea behind our approach is based on three observations:Sensor orientation.

It is well known that a static 3 axis accelerometer canbe used to detect its own orientation with respect togravity [10]. This means that with two simple calibration gestures we can determine the orientation of anaccelerometer on a body part. For an arm mounted
device, the user would have to hold the arm verticallyand horizontally for a second or two. For most otherbody parts (upper body, leg) just standing still would
be sufficient. Mizell has also shown that averaging overthe signals on each axis over a reasonable time period
can produce a good estimate of the gravity-related component [10]. During activities such as walking, where
the type of motion performed by different body parts
is constrained and well known, this could be used toautomatically detect the orientation.

1. The signal of an body-worn accelerometer is the sumof three components: acceleration due to rotation, acceleration due to translation and acceleration due toorientation with respect to gravity. Of the three only
the first one: acceleration due to rotation is sensitive
to sensor displacement within a single body part, as
we will explain in the next section exploring the physical considerations our work is based on.

2. It is possible to identify, with high probability, accelerometer signal segments which are dominated byrotation and thus are possibly ’contaminated’ withdisplacement related noise.

In summary, while handling orientation would certainly
have to be carefully considered for each specific application and body part, there are a number of promising
approaches that can be applied. As a consequence thispaper make the assumption that, in most cases, orientation can be estimated with reasonable effort. Thus
we concentrate on the variations of the exact position
within the body part.

3. Gyroscopes are insensitive to displacement within asingle body part but provide only information on rotation ignoring translations and vertical orientation.

Exact position within a body part.

From the above observations, it follows that combining a gyroscope with an accelerometer and having theaccelerometer ignore all signal frames dominated by rotation can remove placement sensitivity while retaining
most of the relevant information. In fact, sometimes
just an accelerometer ignoring the rotation ’contaminated’ frames can be enough for more displacement tolerant recognition. Additional measures that we propose
are the use of heavily low pass filtered acceleration signals as additional features and training the system on
two sensors corresponding to the ’worst possible displacement’.

Most attachment methods leave a lot of room for placement within a body part. Thus, for example, arm MP3
holders often used for jogging can be placed almost anywhere on the upper or lower arm. Integration of sensors
in clothing can ensure that sensors end up on a certain
body part. However, it cannot ensure a specific placement on that body part. Even a tight fitting sleeve
can be rolled up or twisted, completely changing theplacement of any integrated sensors.
Unfortunately the within body placement issue cannot
be solved with simple calibration gestures. As explainedin the next section, the gravity component (=orientation) does not depend on the position within a bodypart. Thus, a static calibration gesture is not sufficient.
Instead motions would have to be performed with different, exactly defined speeds and trajectories. In general
we cannot expect the user to be able to perform suchexactly defined motions with sufficient reliability.

The main limits of the validity of our heuristics are (1)a rigid body approximation of human body parts and(2) the assumption that the bulk of the discriminative
information is not in signal segments that contain simultaneously performed fast rotations and significant
translations or changes in vertical orientation.
In the rest of the paper, we first described how ourheuristics can be derived from basic physical considerations. Next we apply them to a set of ’synthetic’ gestures that are well suited to demonstrate the strengths
and limits of our approach. Finally, we present an evaluation on two real life recognition tasks. The first task is
an extended modes of locomotion problem using upper
leg mounted sensors. The second is a set of gym exercises classified using sensors mounted on the lower arm.
On this set our heuristics improves recognition rates fordisplaced sensors from 24% using a displaced accelerometer, which had 96% recognition when not displaced, to82%. On other examples the displaced recognition rate
raises from 63% to 90%.

In summary, regarding the three sub-problems described
here, displacement within a body part is the most difficult to handle. Dealing with it is a so far unsolved
problem. It is this paper’s topic.
Paper Idea and Contributions

Although we did not discover an exact, always valid solution, we present a set of heuristics that significantly
increase the robustness of motion sensor-based activity
recognition with respect to sensor displacement within
a single body part. We show how, within certain limits and with modest quality degradation, our heuristics
allow motion sensor based activity recognition to be

21


d1,3

p1

d1,2

€

€

€

d2,3


x2

p1

€


d1,2

€
€
€

θ1

€


d1,3

α1,2

€
€

€


r1

€

€

α 2,3


d2,3

(a)
p1


θ3

p2

€

€


r2

 €
r3

Rigid Body Translation


d1,3


d1,2

€

€ €

€

p3

p2


d1,3
p3
p2€ d
2,3

€
d1,2
α1,3
€
€

Note, that although human joints have only rotational
degrees of freedom, a motion combining simultaneousrotation at two different joints can have the effect ofa translation (e.g. shifting your lower arm through a
combined elbow and shoulder motion). It is also important to keep in mind that motions involving morethen one joint can lead to rotations around axis thatare not identical with any of the involved joints. Regarding arm motions, such axis are often close to thetorso, as we move our arms around the body.


x3

€


θ2

€

p1

p3

p2 


x1

€

regardless of external forces exerted on it. Any motionof a rigid body can be described as a combination of atranslation and a rotation.

α1,3


d2,3

During a translation every point in a rigid body is movedby exactly the same vector with exactly the same speed
and acceleration. This is illustrated in figure 1a. We
have a rigid body with three arbitrary points p1 , p2 , p3 .
The relative positions of those points are given by thedifference vectors d~1,2 , d~1,3 , d~2,3 . We assume that the
body is translated (=moved in a straight line) randomly
which results in p1 , p2 , p3 being moved by a corresponding vectors x~1 , x~2 , x~3 . Per definition of a rigid body the
relative positions given by d~1,2 , d~1,3 , d~2,3 must remainunchanged. This is only possible if all the points are
moved by exactly the same vector:

p3

α€1,2
α

2,3
 
r1 r2 
r3

€

€

(b)

Figure 1: Rigid body translation 1a and rotation 1b

x~1 = x~2 = x~3

Related Work

(1)This is valid independently of the translation distanceand the time it took. Thus, given a translatory motion,at any point in time during this motion, all points of arigid body will have been moved by exactly the same
vector. This is also valid for infinitesimaly small timeintervals which implies that at any given point in time
the speed and with it the acceleration vectors will also
be the same for all points.

To our knowledge there is no other work directly targeting the problem of within body part displacement
for motion sensors. However, there has been some indirectly related work. Van Laerhoven presents a study
to explore the trade-offs between single on-body sensors with predefined, well-known placement and an increasing quantity of sensors with decreasing information
quality (placement accuracy) [16]. Zinnen presents aninnovative way to use rest periods in accelerometer signals for detection of non-repetitive tasks which is basedon some of the principles presented here [18] . Lester
uses acceleration signatures to determine that a set ofdevices is being carried by the same person [6]. There isalso some work on evaluating the suitability of different
on body locations for activity recognition [9]. A platform with multiple sensors (in addition to mere motion
sensors) has been investigated with respect to on body
location invariance of activity recognition [7].

Rigid Body Rotation

In an analogous way it can be shown that angular velocity vector (and angular acceleration) are the same for all
points of a rigid body during a rotation around an arbitrary point in space. To illustrate this figure 1b showsa rigid body in which three arbitrary points p1 , p2 , p3
and an arbitrary center of rotation r have been marked.
The vectors connecting each point to the center of rotation are marked as r~1 , r~2 , r~3 , their relative angles as
α1,2 , α1,3 , α2,3 . We consider a rotation around r which
results in p1 , p2 , p3 being rotated by θ1 , θ2 , θ3 . Since per
definition of a rigid body after the rotation the relative positions of the three points given by d~1,2 , d~1,3 , d~2,3
must be unchanged, the relative angles between vectors
connecting them to the center of rotation α1,2 , α1,3 , α2,3must also be unchanged. This is only possible if all threepoints have been rotated by the same angle:

PHYSICAL CONSIDERATIONSThe Rigid Body Approximation

A common approximation used in modeling human body
motion is that of rigid segments connected by joints
which allow rotation around one (e.g. elbow) or more(e.g. wrist) axis. In simple words such an approximation is equivalent to a ’stick figure’ representation usedin many animations. In exact terms, a rigid body is an
ideal solid body of finite size for which the relative position of any two given points remains constant in time

θ1 = θ2 = θ3

(2)

As for translation considering infinitesimal time periods, the angular speed ω and acceleration must also be
the same for all three points.

22

In summary, during a rotation of a rigid body aroundan arbitrary point in space a gyroscope will produce
the same signal no matter where in the rigid body it isplaced. As will be explained later, this does not apply
to accelerometers since different points in a rigid bodyin general experience a different, non zero acceleration
vector during a rotation.

The important thing to remember when looking at theabove equation is that the v designates the speed traveled along a circle. This means that, although the scalarvalue of the speed is constant (if ω remains constant), tofollow the circle each point of a rotating rigid body constantly needs to change its direction1 . Such a change ofdirection requires an acceleration. The direction of theacceleration is parallel to the radius of the circle. The
magnitude of this acceleration depends on the speed(the faster a point travels the more force is required tochange direction) and with it on the distance for thecenter. The magnitude of linear acceleration aω due toconstant angular velocity ω in a point at the distance r
from the center of rotation is given by:

Limits of the Rigid Body Approximation

Obviously ,the individual segments of the human bodyare not really rigid bodies. Deformation of soft tissue,
skin motion and muscle activity associated with mostmotions all lead to deviations. However, as will be
underscored by subsequent experiments (see the next
section ), for many sensor positions and motions it isa valid approximation. The main deviations from therigid body approximation can be observed in the following situations.

aω = ω 2 r

This gives us the first source of acceleration during arotation of a rigid body. It is often referred to as centripetal acceleration. The second potential source stems
from changes in the rotation speed. Since the linear
speed is proportional to the angular velocity and the
distance from the center (equation 3), it follows thatthe linear acceleration aα associated with a change ofangular velocity is proportional to the angular acceleration (α) and the distance from the center r:

1. During short, intensive acceleration and follow up vibrations soft ’wobbly’ parts (fat, soft muscles) are
deformed in a non rigid way. To deal with such deviations the system might discard such vibrations.
2. When active muscles change shape. In particularlarge muscles will cause motion signals incompatible
with the rigid body approximation. Thus, one shouldfor example avoid placing sensors directly on top ofa well developed biceps. Fortunately, such placement
is often not very comfortable and is likely to be automatically avoided by many users.

aα = αr(5)This component is called tangential acceleration.
Since the centripetal acceleration and the tangential acceleration are perpendicular, not parallel, the scalar values given above can not be just added to get the total
magnitude of acceleration (which is the euclidian normof the acceleration vector). For the sake of simplicity
we will just deal with each of them separately 2 Anothersimplification is to ignore the coriolis force which acts
on objects moving along the rotation axis. By moving
more than one joint at a time, it is certainly possible
to construct motions of human body parts for which
the coriolis acceleration plays a significant role. Yet,motions where this is a relevant component are seldom
and will not be discussed it in this paper.

3. The lower arm rotation parallel to the axis of thearm will affect sensors fixed to the wrist in a significantly different way than sensors near the elbow.
The wrist sensor will rotate perfectly with the wrist,
whereas the elbow sensor will do so to a much lesser
degree. Gestures, for which such rotations are an important discriminative information are a problem forour location invariant recognition, are illustrated by
our ’synthetic gestures’ evaluation in Section .
Acceleration during Rigid Body Rotation

During a pure translation gyroscopes will provide no
signal at all (there is per definition no rotational component) while accelerometers will all give the same readings no matter where they are placed.

Consequences for Displaced Sensors

What does the above mean for the noise introduced bydisplacing a sensor within a single, rigid body segment?
As already state a gyroscope signal is displacement invariant so it need not be considered further. For an acceleration signal we need to differentiate between threecontributions: (1) the contribution caused by orientation with respect to gravity, (2) the contribution causedby translations and (3) contribution caused by rotation.
As explained above, the first two are location invariant
and only the rotation component is location sensitive.

As already said, in a rigid body all points are rotatedwith the same angular velocity (ω) and experience thesame angular acceleration α. Thus, the gyroscope signal
is invariant with respect to sensor displacement.
To understand the effect of sensor displacement during
rotation on the accelerometer signal we need to revisit
some basic physics. During a rotation with the angularvelocity ω, the linear velocity v of each point of therigid body depends on the distance from the center ofrotation r. The further the point is from the center, thelarger the circle it needs to travel and, consequenctly,
the faster it needs to move. For the speed we have:v = ωr(4)

1

The vector ~v is parallel to the tangent of the circle in eachpoint of the rotation2
Since the two are perpendicular their contribution
to the√norm of the acceleration acombined is given by ω 4 r2 + α2 r2

(3)

23

of the acceleration vector computed from the acceleration sensor signal to determine if the acceleration signal
is rotation dominated or not. A high acceleration witha relatively low measured angular velocity is a good
indication of the signal not being dominated by rotation. On the other hand, high angular velocity withlow or moderate acceleration is an indication of a rotation dominated motion.

Displacement Noise in Rotation Related Acceleration

Given two different points of a rigid body: one witha distance r1 from the center of rotation and the second one with a distance of r2 , we can compute the acceleration components resulting from constant rotationaω,1 , aω,2 and from angular acceleration aα,1 , aα,2 fromequations 4 and 5. Thus, the signal difference attributed
to sensor displacement can be computed asaω,1 − aω,2 = ω 2 r1 − ω 2 r2 = ω 2 (r1 − r2 )aα,1 − aα,2 = αr1 − αr2 = α(r1 − r2 )

(6)
(7)

The ratios of angular velocity to acceleration norm signifying the transition between rotation and translation
dominated signal depend on the typical rotation radius
and with it on the motions relevant to the specific recognition task. They have to be learned during training.
We will show an example in the next paragraph.

How relevant this difference is to the recognition depends not on its absolute magnitude, but on the signal
to noise ratio. This is the ratio of the original signal(aω,1 or aα,1 ) to the difference caused by displacement(aω,1 − aω,2 or aα,1 − aα,2 . It can be computed fromequations 6 and 7:
ω 2 (r1 − r2 )r1 − r2aω,1 − aω,2==aω,1
ω 2 r1r1
aα,1 − aα,2α(r1 − r2 )r1 − r2
=
=
aα,1
αr1
r1

The above means that while we can not separate the individual components of a given acceleration signal, it ispossible to estimate with reasonable probability which
signal frames are dominated by rotation and which arenot. We can then throw away the rotation dominatedframes, which are sensitive to displacement and use only
the ones dominated by translation and or vertical orientation. In a sensor setup with a gyroscope we can try to
substitute the rotation for the thrown away acceleration
frames to retain rotation related information.

(8)
(9)

The above is a very compelling result. It shows that
sensor displacement noise during rotational movement
depends only on the amount of displacement with respect to the center of rotation). It is independent of theactual angular velocity or angular acceleration.

Another interesting consideration relates to the verticalorientation component of the acceleration signal. Any(non free falling) object on earth is subject to a constant 9.81 m/s2 acceleration. This means that if thenorm of the acceleration signal is close to 9.81, then thesignal is likely to be dominated by the vertical orientation component. Clearly this also is a heuristic that is
not always valid. We can imagine a situation when an
object is free falling while experiencing a 9.81 m/s2 side
acceleration. However this is a rare occurrence, and theabove assertion is mostly valid (as will be underscoredby the experiments in the next section).

Consequences for the Recognition

Previous paragraph dealt with the distortion of the acceleration signal related to rotation, as the other components are not affected by displacement. A naive idea
for the design of an displacement invariant recognition
system would be to try to ignore the rotation related
component of the acceleration signal and use only thetranslation and vertical orientation related components.
Unfortunately in general 3 , it is theoretically not possible to decompose an acceleration signal into the above
three components. Note that this remains true evenif we combine an acceleration sensor with a gyroscope.
The gyroscope will indicate the presence and speed ofrotation. However, as shown in equation to computethe acceleration we need the distance from the centerof rotation, which we do not know.

In Summary
Signal Level Summary

The results of the discussion presented in this section
can be summed up in the following points:
1. Gyroscopes are insensitive to sensor displacement within
a single rigid body segment. However they capture
only information about the rotational motion component. They fail to capture information about translational motions and the vertical orientation (orientation with respect to gravity).

Fortunately, although we do not know the exact radiusof the rotation, we know that it is bounded by the dimensions of the human body. While rotational motions
with very high radius can be constructed, for most human limb motions the center of rotation is somewhereclose to the torso. This means, that for a given rotation
speed, the acceleration is unlikely to exceed a certain
value. This in turn means that we can use the ratio ofrotation velocity measured by a gyroscope to the norm

2. The accelerometer signal is a sum of acceleration due
to rotation, acceleration due to translation and acceleration due to orientation with respect to gravity.
3. Acceleration due to translation and orientation withrespect to gravity are independent of sensor placement within a rigid segment of the body.

3
The general case assumes that there is no additional information such as further sensors in different locations on thesame body part or appropriate high level knowledge about
the form and constraints of the motion

4. Acceleration caused by rotational motion is location
sensitive. The ratio of the corresponding acceleration24

signal to the ’noise’ introduced by sensor displacement is proportional to the ratio of to the amount of
displacement with respect to the center of rotation.

(norm of the acceleration vector) to the total rotation (norm of the angular velocity vector). The
accelerometer signal is used for classification if it
dominates both ratios. Otherwise it is ignored(e.g acceleration input to the classifier set to 0).

5. Using an acceleration (and possibly gyroscope) sensor
at one location only, it is not possible to separate thethree above mentioned acceleration components (rotation caused, translation caused and gravity caused).
Thus, given an acceleration signal we are not able
to remove the rotation related component (which issensitive to displacement noise) and just use for classification the two other components (which are notdisplacement sensitive).

The above procedure ’looses’ information in two cases.
First, if we have a motion that combines fast rotation or large angular acceleration with a significant
amount of linear acceleration then the above ruleleads to the acceleration signal being ignored. Thisis the price that we have to pay for location invariance and there is nothing that can be done about it.
Second, in all cases where there is a large rotation
we loose information about vertical orientation. Using strongly low pass filtered acceleration signal as anadditional feature can, in most cases, retain at least
some of the vertical orientation information.

6. However, given an acceleration and a gyroscope measurement (from the same location taken at the sametime), we can estimate the contribution of each of thethree components to in the following way• If the norm of the acceleration vector is close
to 9.81 (earth gravity) then the signal is mostprobably dominated by the gravity component(vertical orientation).
• If the norm of the acceleration vector is not close
to 9.81 then we look at the ratio of the normof acceleration minus 9.81 to the angular velocity and the angular acceleration. If the angularvelocity or angular acceleration dominate the ratio, we know that the acceleration signal is dominated by the rotation related components. Thus
the acceleration signal is strongly location dependent. If the acceleration norm (minus 9.81)
dominates, then we know that the acceleration
signal is determined by translation related acceleration. In this case the acceleration is reasonably location independent.
If none of the above applies then the acceleration signal is an mixture of the three contributions with none
clearly dominating.

3. If only an accelerometer is available, then the best we
can do is to identify the segments of the signal thatare dominated by the gravity component and base therecognition solely on the information about verticalorientation. This may sound like loosing a lot of information, however previous work ( [3,18]) has shownthat many activities are to a large degree determinedby vertical orientation and changes thereof.
4. Independent of the recognition modality training thesystem with two sensors as far displaced as possible
should encourage the classifier to focus on location
invariant parts of the signal.
EVALUATION ON SYNTHETIC MOTIONS

As initial evaluation we look at following 8 ’synthetic
motions” of the forearm:

2. For general activities the best location insensitive sensor setup consists of an accelerometer and a gyroscope. The procedure can be summarized as follows

a move upb move straight out
c move from left to rightd close elbow joint
e move back (closing elbow joint) and turn wrist in one
motionf turn around shoulder joint (screw-driving)g turn large circles around shoulder
h turn smaller circles around elbow
The above motion set was put together to contain both’easy’ and ’hard’ gestures and illustrate the strengths
and weaknesses of our approach. Thus, for example,
gestures e and d differ mostly in the turning of the wrist.
As has been discussed in the previous section, wrist
turning is especially displacement sensitive because ofdeviations from the rigid body approximation. On theother hand gestures a and b are likely to be well suitedfor our approach. Note that many typical arm activities
are likely to contain motions from the above set.

(a) If there is a significant gyroscope signal then weuse it as primary source of information(b) To decide what to do with the accelerometer signal we look at the ratio of the total acceleration

The lower arm was chosen for two reasons. First, it
is a likely place to wear accelerometers (watch etc.).
Second, the forearm has the most degrees of freedom
and is the body part that is able to move fastest.

7. Low pass (pass frequency below Hz) filtered acceleration signal is likely to be dominated by the gravity
component (see [3]).
Recommendation for recognition

For the design of on body activity recognition system
based on motion sensors that is as insensitive as possible
to sensor displacement, the following recommendations
can be made:1. If the relevant activities are mostly determined byrotational motions then placement invariance (withina rigid segment of the body) can be achieved by using
gyroscopes instead of accelerometers.

25

Modality
Acceleration
Gyroscope
Cut Off
Combined

Sensor Setup

We use XBus Master System together with 6 MTx motion sensors equipped with a 3-axis accelerometer, gyroscope and magnetic field sensors. As stated in the
introduction we focus on the location within a single
body part and ignore the question of sensor orientation. Thus, for all sensors, the x-axis orientation is thesame (pointing towards the ground if the arm is in rest).
The 6 sensor are placed as follows, (1) wrist outside, (2)wrist inside, (3) middle of segment outside (y axis orientation same as 1), (4) middle of segment on top ofarm (y axis orientation 90 degrees to 3 and 1), (5) close
to elbow inside and finally (6) close to elbow outside.

Same
100 %
65%
-

Trained on 1
33%
43%
42%
78%

Trained on 2
35%
44%
47 %
85%

Table 1: Classification comparison for the synthetic
motions using a majority decision over the motions
based on a Knn classifier. Acceleration cut-off Norm
- 9.81 at larger than 0.8. Decision Boundry for combining accelerometer and gyro at 300.
a
100
0
0
0
0
0
0
0

Signal Level Evaluation

Before we proceed to classification experiments we would
like to use the synthetic gestures data to validate the basic assumptions behind our approach. First we check if
leaving out signal segments with large angular velocity
to acceleration ratio does indeed reduce the displacement related noise in the acceleration signal. To thisend in Figure 2 (left) we have plotted the difference insignals between all sensors locations (in percent of thesensor signal) against the acceleration norm divided byangular velocity norm. In the rest of this paper we willrefer to the difference in signals between all sensors locations in percent as displacement noise. It can be clearlyseen that as long as the ratio is large (above 300) thesignal difference is very close to zero. This means that
displacement has nearly no effect on the signals. As theratio gets smaller and angular velocity starts to dominate we begin to get a spread in the displacement noise
and for very small values there is significant noise. Thisconfirms our basic assumption.

b
0
100
0
0
0
0
0
0

c
0
0
100
39.1
28.6
14.3
0
0

d
0
0
0
60.9
0
0
0
0

e
0
0
0
0
71.4
0
0
0

f
0
0
0
0
0
85.7
0
0

g
0
0
0
0
0
0
100
0

h
0
0
0
0
0
0
0
100

←
a
b
c
d
e
f
g
h

Table 2: Combined Accelerometer and Gyro trainedon 2 evaluated on 4 Sensors Accuracy 85 %, Decision
Boundary at 300.
tions this is extremely simple, as most of the sensors
have the same orientation anyway. For the later two
evaluations two calibration gestures are performed between recording the motions. This allows us to determine two normalized axes from the accelerometers due
to gravity. For all classifications we use the 2 normalized axis (defined as x and y) for feature extraction andonly the magnitude of z, as we cannot determine its
direction using the acceleration.
Using the entropy measure also applied in the C4.5 decision tree, we reduced our feature set from 40 to 8
(mean, variance, number of peaks, median peak height,
FFT center of mass, RMS, and frequency range power)
depending on the evaluation. Each feature is calculated
over the accelerometer and gyro data. The gyro data isnormalized the same way as the accelerometer.

Next, we check the assumption that frames where thenorm of the acceleration is close to 9.81 (gravity) arelikely to contain mostly orientation information andthus no displacement related noise. This is illustrated
in Figure 2 (right). We can see that for accelerations
norm values within 1g from 9,81 the noise is negligible.
In summary, our assumptions hold well on the test dataset.

We classified all examples using several frame-by-frame
classifiers( C4.5, KNN, BayesNets). As all of them show
more or less comparable results, we pick KNN for the
analysis for the remainder of the paper.

Recognition Experiments

Next we test if the validity of our assumptions will actually translate into recognition results. To this end
we first train the system on two locations. We use two
locations to be able to verify the claim that training
different locations helps the system learn the displacement invariant features. We then test the system on
the locations that it was trained on as well as on three
additional locations. We do it with and without ourheuristics and compare the results.

Classification Results

The results are summarized in table 1. Training theclassifiers on training data and test data from one distinct sensor we reach a classification rate of 100 % using both frame by frame classification and a majority
decision window over complete gestures. Testing thetrained system on locations that it was not trained on
reduces the recognition rate to 33% (on the accelerometer only). Having trained the system not on one, but
on two widely displaced sensors improves the recognition rate to 35% only. Getting rid off frames with highacceleration improves the recognition by about 10% but
the performance remains poor.

Classification Method

Applying 1 sec. sliding window we extract 45 standard
pattern recognition features for each accelerometer andgyroscope axis. Concerning sensor orientation, we use
normalized axis. For the evaluation of synthetic mo-

26

45
40
35

1.2

1
25

Norm Difference (in %)Norm Difference (in %)

30

20
15
10

0.8

0.6

0.4
5
0.2

0
!5

0

100

200
300
400
N Accelerartion / N Angle Vel + N Angle Accel

500

600

0

0

0.5

1

1.5

2

2.5

3

3.5

4

Norm ! 9.81 (m/s2)

Figure 2: Left: Difference in Percent plotted against the Norm Acceleration divided by, the Norm Gyro Vector:
Right Difference in Percent against acceleration norm - 9.81
Locomotion
i walkingj runningk running uphilll biking
m rowingn stairs
o skiing
p crosstrainer

The gyroscope performs significantly worse then the accelerometer (65% on the same location) confirming ouranalysis that it fails to capture all relevant information.
When tested on a different location it drops to 43%.
This is much less of a dramatic drop then for the accelerometers but still significant. We expected the gyro
to be invariant with respect to displacement. The explanation is the inclusion of gestures with wrist rotation,
which violates the rigid body assumption.

Gym Exercises
q lat machiner pectorial
s shoulder presst upper backu arm extension
v arm curl
w pull down
i chestpress

Table 3:
Motions classified in the Locomotion and
Gym exercise scenarios

As expected best location invariant recognition resultsfrom a combined accelerometer/gyro based approachwith all rotation dominated accelerometer frames being ignored. Trained on one sensor we reach 78%, on
two we come up to 85%.
In summary the initial experiment confirms that ourheuristic works well. Clearly 85% is far from perfect,
but for many applications it may be acceptable (as opposed to 33%). The result is particularly significant because we were working with large displacements. Small
displacements typical of ’slipping sensor’ are likely tolead to a much less significant reduction in recognition
rate (we have shown, that the noise is proportional tothe displacement with respect to the center of rotation).

Figure 3: Random generated sensor placement and orientation for the leg(front and back).
Experiment Setup

The subjects upper leg is equipped with 6 MTx Sensors
3 mounted on the front and two on the back as seen in
Figure 3. The placement is generated using a uniform
random distribution. We use bandages to attach thesensors. Overall 8 locomotion classes (Table 3) wererecorded on fitness machines in a fitness center. One
test subject performed them each for 5 min.

Another important thing to observe is the confusion
matrix that corresponds to the 85% recognition rate
(Table 2). It can be seen that out of the 8 gestures 5
achieve 100% recognition. The confusions involve gestures with significant wrist rotations. We have identified such rotations as one of the cases where the rigid
body assumption underlying our heuristics is not valid.

Results
MODES OF LOCOMOTION EXPERIMENTS

The results are summarized in table 5. Training andclassifying on the same acceleration sensor gives an accuracy between 95 and 100 % using 10 fold cross validation or 66 % percentage split using a KNN and a majority decision window. As expected a gyroscope performs
worse with 80 percent accuracy on the same location.
Note that the leg motions are very much rotation de-

Towards more realistic evaluation we first look at an
extended modes of locomotion problem with the sensors
placed on the upper leg. We differentiate 8 activities (table 3) on the left. Note that this is not the trivial
walking/standing/sitting modes of locomotion problem
but an experiment involving fairly subtle differences.

27

i
100
0
0
0
0
0
8.7
6.1

j
0
76.9
20.3
0
0
0
0
0

k
0
23.1
79.7
0
0
0
0
0

l
0
0
0
90.4
0
0
0
0

m
0
0
0
9.6
100
0
0
0

n
0
0
0
0
0
91.1
0
0

o
0
0
0
0
0
8.9
91.3
0

p
0
0
0
0
0
0
0
93.9

←
i
j
k
l
m
n
o
p

(a)

Table 4: Joint Accleerometer and Gyro trained on 2
Sensors eval on 3 90 % decision boundary at 150
Modality
Acceleration
Gyroscope
Cut Off
Combined

Same
100 %
80%
-

Trained on 1
63%
72%
72%
87%

(b)

Figure 4: Two pictures form the gym experiment data
recordingTrained on 2
65%
75%
76%
90%

turned from an orientation that is parallel to the arm
pointing towards the ground, the second on the wrist,
with approximately 50◦ , the third placed at the insideof the arm around 8 cm away from the wrist with 0◦ ,the forth placed also on the inside closer the the elbow
at 10◦ . Again we picked 8 gym exercises to record, as
shown in Table 3. One test subject performed each
exercise 20 -25 times. Two runs were conducted.

Table 5: . Classification comparison for the locomotion exercises using a majority decision over the motions
based on a Knn classifier. Acceleration cut-off Norm 9.81 at larger than 0.6. Decision Boundry for combining
accelerometer and gyro at 150.

The feature extraction follows the approach laid out
for snythetic motions and leads to the same features.
We use a 1 sec. sliding window. The recognition task ismuch harder than the modes of locomotion problem andthe majority decision using the acceleration trained andevaluated on the same location gives only an accuracyof 85%. We have thus turned to a continuous HMM
based approach. On top of the features extracted as
mentioned above we apply another 15 sec. sliding window using 3 gaussians for each feature and 4 hidden
states. In case of combining the gyro and accelerometer data, we picked the decision boundary at 300 for
the ratio. If it is below the boundary we use the gyrofeatures and set the accelerometer features all to zero.

termined, so the performance reduction for the gyro isless pronounced than for the synthetic gestures from theprevious paragraph.
Testing the same methods on a location on which they
have not been trained leads to a recognition rate of 63%
on the accelerometer and 72% on the gyro. As we haveless deviations from the rigid body assumption the drop
in gyro recognition rates is smaller than for the synthetic arm gestures. That it exists at all, is probably
due to muscle motions (which are significant in somelocations on the upper leg). Training on two locations
brings minimal improvement. Restricting the acceleration frames to those with a norm close to 9,81 improves
the recognition by 10% bringing it to 76% when trainedon two sensors. This relatively good recognition rate(for a displaced, acceleration only system) is due to thefact that most of the relevant motions are largely determined by changes in vertical orientation of the upper
leg.

Results

The results are summarized in table 6. When training
and testing on the same location (again 66 % percentage split) we reach 96 % on the acceleration signal alone.
Testing the acceleration only system on a location that
it was not trained with drops the recognition rate to 24%. This was to be expected, as the classification problem is fairly complex. The rate can be raised to 31%
by training the system on 2 sensors, which is significant but not really useful. A gyro trained and testedon the same location gives an accuracy of 62% again
confirming that the gyro signal ’looses information’. 4
However, a significant improvement on displaced sensor
is achieved with our combined gyro/accelerometer approach. Trained on one sensor we reach 74% on two wecome up to 82%. The confusion matrix for this case isshown in table 7.

The combined accelerometer gyro heuristic (throwing
away rotation related acceleration frames) brings therecognition rate to 90% (trained on two sensors).
In summary, the modes of locomotion experiment also
confirms the validity of our methods. For many practical applications the 90% recognition rate might be sufficient. Again one should keep in mind that we were
working with large displacements and that the noise is
proportional to the displacement.
GYM EXPERIMENTS WITH SENSORS ON FOREARM

The most challenging evaluation focuses on muscle strength
exercises conducted at fitness center machines.

Considering the confusion matrix of the combined accelerometer and gyro case, the really significant miss4
Since the problem is harder than the previous ones it was
not to be expected that dropping high acceleration frames
form the accelerometer classification will lead to reasonable
performance. Since the HMM evaluation was more timeintensive than the majority decision from previous examples
we did not take the time to evaluate this approach.

Experiment Setup

The sensor placement and orientation are generated
random. There are 4 sensors at the forearm placed asfollows. The first around 10 cm away from the elbow
on the outside of the arm , x axis angle around 90◦28

Modality
Acceleration
Combined

Same
97%
-

Trained on 1
24%
74%

Trained on 2
31%82%

2. Brashear, H., Starner, T., Lukowicz, P., Junker, H.: Using
multiple sensors for mobile sign language recognition.
Wearable Computers, 2003. Proceedings. Seventh IEEE
International Symposium on (2003) 45–52

Table 6: Classification comparison for the gym exercises using a continuous HMM. Decision Boundary for
combining accelerometer and gyro at 300.
q
75.6
0
0
0
0
0
12.0
0

r
0
81.6
0
0
0
0
0
0

s
0
0
88.6
0
13.3
0
0
0

t
0
0
0
100
0
0
0
20.8

u
0
0
11.4
0
76.7
22.2
8.0
0

v
0
0
0
0
0
77.8
0
0

w
0
18.4
0
0
10.0
0
80
0

x
24.4
0
0
0
0
0
0
79.2

3. Kern, N., Schiele, B., Junker, H., Lukowicz, P., Tröster, G.:
Wearable sensing to annotate meeting recordings. Personaland Ubiquitous Computing 7 (2003) 263–274

←
q
r
s
t
u
v
w
x

4. Kunze, K., Lukowicz, P., Junker, H., Troester, G.: Where
am i: Recognizing on-body positions of wearable sensors.
LOCA’04: International Workshop on Locationand Context. . . (2005)
5. Krause, A., Smailagic, A., Siewiorek, D.: Context-aware
mobile computing: Learning context-dependent personal
preferences from a wearable . . . . IEEE Transactions on
Mobile Computing (2006)

Table 7: Confusion Matrix Joint Accleerometer andGyro trained on 2 Sensors eval on 2 82 % decisionboundary at 300

6. Lester, J., Hannaford, B., Boriello, G.: Are you withme?–using accelerometers to determine if two devices arecarried by the same person. Pervasive Computing: Second
International Conference (2004)

classifications happen between movements that train
the complementary muscles, for example arm extension
and arm curl.

7. Lester, J., Choudhury, T., Borriello, G.: A practical
approach to recognizing physical activities. Proceedings ofPervasive (2006)

CONCLUSION AND FUTURE WORK

8. Mantyjarvi, J., Himberg, J., Seppanen, T., Center, N.R.:
Recognizing human motion with multiple acceleration
sensors. Systems, Man, and Cybernetics, 2001 IEEE
International Conference on 2 (2001)

We have shown that a combination of an accelerometer that ignores rotation dominated signal segments
and a gyroscope to compensate for the lost rotation
information is reasonably robust with respect to sensor displacement within a single body part. Combinedwith two sensor training to force the classifier to ignore
location artifacts we have shown that randomly, significantly displaced sensors can reach up to abut 90% of
the recognition rate of a non displaced sensor. Compared to testing an unmodified classification system ona different location we can improve the recognition rateby over 300% !

9. Maurer, U., Smailagic, A., Siewiorek, D., Deisher, M.:
Activity recognition and monitoring using multiple sensors
on different body positions. Proceedings of the International
Workshop on Wearable and . . . (2006)
10. Mizell, D.: Using gravity to estimate accelerometerorientation. Wearable Computers, 2003. Proceedings.
Seventh IEEE International Symposium on (2005) 252 – 253
11. Stiefmeier, T. Roggen, D. Troster, G. Ogris, G. Lukowicz,
P.: Wearable activity tracking in car manufacturing. To.
Appear, IEEE Pervasive Computing 7 (2008)

Clearly coming to 90% of the non displaced sensor will
not be sufficient for all applications. Also the need to
add a gyroscope (which is more expensive than an accelerometer) may not always be acceptable. However
we believe that our heuristics are a significant improvement over the current state of the art and may be goodenough in many cases.

12. Randell, C., Muller, H.: Context awareness by analysing
accelerometer data. The Fourth International Symposium on
Wearable Computers 1 (2000) 175–176
13. Roggen, D., Bharatula, N., Stager, M., Lukowicz, P.,
Troster, G.: From sensors to miniature networked
sensorbuttons. Proceedings of the 3rd International
Conference on Networked Sensing Systems (INSS06) (2006)

This paper has focused on large displacement that would
be typical of a user being given no instructions on whereto place the device. We also wanted to test the limits of our ideas. Next we will investigate in more detail smaller displacements that my be more typical ofshifted sensors. As described in the paper displacement
noise is proportional to displacement distance. Thus forsmaller displacements and with some further improvements we believe that our methods could work satisfactorily even for accelerometer only systems. Under suchcircumstances accelerometer/gyro system might be able
to mask out displacement noise entirely.

14. Van Laerhoven, K., Aronsen, A.: Memorizing what you didlast week: Towards detailed actigraphy with a wearable
sensor. Proceedings of the 27th International Conference on. . . (2007)
15. Van Laerhoven, K., Cakmakci, O.: What shall we teach ourpants? Wearable Computers, 2000. The Fourth International
Symposium on (2000) 77–83
16. Van Laerhoven, K., Gellersen, H.: Spine versus porcupine: astudy in distributed wearable activity recognition. (ISWC
2004)
17. Westeyn, T., Vadas, K., Bian, X., Starner, T., Abowd, G.:
Recognizing mimicked autistic self-stimulatory behaviors
using hmms. IEEE International Symposium on Wearable
Computers (2005) 164–169

REFERENCES

18. Zinnen, A., van Laerhoven, K., Schiele, B.: Toward
recognition of short and non-repetitive activities fromwearable sensors. (European Conference on AmbientIntelligence 2007)

1. Aylward, R., Paradiso, J.: Sensemble: a wireless, compact,
multi-user sensor system for interactive dance. Proceedingsof the 2006 conference on New interfaces for musicalexpression (2006) 134–139

29