Experimental Evaluation of Variations in PrimaryFeatures Used for Accelerometric Context Recognition
Ernst A. Heinz1 , Kai S. Kunze1 , Stefan Sulistyo,1
Holger Junker2 , Paul Lukowicz2 , and Gerhard Tröster2
1

Entertainment and Pervasive Computing Group, International University (IU) in Germany{ernst.heinz, kaisteven.kunze, stefan.sulistyo}@i-u.de
http://www.i-u.de/schools/heinz/

2

Wearable Computing Laboratory, Institute for Electronics (IfE), ETH Zürich, Switzerland{junker, lukowicz, troester}@ife.ee.ethz.ch http://www.wearable.ethz.ch/

Abstract. The paper describes initial results in an ongoing project aimed at providing and analyzing standardized representative data sets for typical context
recognition tasks. Such data sets can be used to develop user-independent feature sets and recognition algorithms. In addition, we aim to establish standard
benchmark data sets that can be used for quantitative comparisons of different
recognition methodologies. Benchmark data sets are commonly used in speech
and image recognition, but so far none are available for general context recognition
tasks. We outline the experimental considerations and procedures used to record
the data in a controlled manner, observing strict experimental standards. We thendiscuss preliminary results obtained with common features on a well-understood
scenario with 8 test subjects. The discussion shows that even for a small sample
like this variations between subjects are substantial, thus underscoring the needfor large representative data sets.

1

Introduction

Using simple sensors distributed over the user’s body has recently emerged as a promising
approach to context recognition in mobile and wearable systems. In particular, motion
sensors such as accelerometers or gyroscopes have been shown to provide valuable
information about user activity. Systems based on a 3 axes have been successfully trained
to distinguish between everyday activities like walking, standing, and sitting [6,7,4]. It
has also been shown that appropriate combinations of sensors attached to different limbs
can provide information necessary to recognize specific complex activities like getting
up and greeting an arriving person with a handshake [3] or operating a particular home
appliance [5]. An important question that needs to be resolved before such systems
can move towards real-life applications is their generalization capability. So far, mostsuccessful experiments have been performed in special settings such as a laboratory
with a single or just a few randomly selected subjects. At this stage it is not clear how
well those results generalize to different locations and other subjects. Thus, a systemtrained to differentiate between walking up or down a particular staircase with a young
female subject might fail for another staircase and / or an older male subject. To a degree,
E. Aarts et al. (Eds.): EUSAI 2003, LNCS 2875, pp. 252–263, 2003.
c Springer-Verlag Berlin Heidelberg 2003⃝

Experimental Evaluation of Variations in PrimaryFeatures Used for Accelerometric Context Recognition
Ernst A. Heinz1 , Kai S. Kunze1 , Stefan Sulistyo,1
Holger Junker2 , Paul Lukowicz2 , and Gerhard Tröster2
1

Entertainment and Pervasive Computing Group, International University (IU) in Germany{ernst.heinz, kaisteven.kunze, stefan.sulistyo}@i-u.de
http://www.i-u.de/schools/heinz/

2

Wearable Computing Laboratory, Institute for Electronics (IfE), ETH Zürich, Switzerland{junker, lukowicz, troester}@ife.ee.ethz.ch http://www.wearable.ethz.ch/

Abstract. The paper describes initial results in an ongoing project aimed at providing and analyzing standardized representative data sets for typical context
recognition tasks. Such data sets can be used to develop user-independent feature sets and recognition algorithms. In addition, we aim to establish standard
benchmark data sets that can be used for quantitative comparisons of different
recognition methodologies. Benchmark data sets are commonly used in speech
and image recognition, but so far none are available for general context recognition
tasks. We outline the experimental considerations and procedures used to record
the data in a controlled manner, observing strict experimental standards. We thendiscuss preliminary results obtained with common features on a well-understood
scenario with 8 test subjects. The discussion shows that even for a small sample
like this variations between subjects are substantial, thus underscoring the needfor large representative data sets.

1

Introduction

Using simple sensors distributed over the user’s body has recently emerged as a promising
approach to context recognition in mobile and wearable systems. In particular, motion
sensors such as accelerometers or gyroscopes have been shown to provide valuable
information about user activity. Systems based on a 3 axes have been successfully trained
to distinguish between everyday activities like walking, standing, and sitting [6,7,4]. It
has also been shown that appropriate combinations of sensors attached to different limbs
can provide information necessary to recognize specific complex activities like getting
up and greeting an arriving person with a handshake [3] or operating a particular home
appliance [5]. An important question that needs to be resolved before such systems
can move towards real-life applications is their generalization capability. So far, mostsuccessful experiments have been performed in special settings such as a laboratory
with a single or just a few randomly selected subjects. At this stage it is not clear how
well those results generalize to different locations and other subjects. Thus, a systemtrained to differentiate between walking up or down a particular staircase with a young
female subject might fail for another staircase and / or an older male subject. To a degree,
E. Aarts et al. (Eds.): EUSAI 2003, LNCS 2875, pp. 252–263, 2003.
c Springer-Verlag Berlin Heidelberg 2003⃝

Experimental Evaluation of Variations in Primary Features

253

generalization issues can be addressed through additional user- and situation-specifictraining. However there are limits to how much training effort a user is willing to put up
with. Hence, it is desirable to be able to design and train systems to work under a wide
variety of circumstances. To this end, at least three issues need to be addressed.
1. For given recognition tasks, it must be determined how sensor signals change fordifferent relevant situations and users.
2. Sets of features as insensitive as possible to the variations above must be found.
3. Standard data sets with the relevant factors varied in a controlled manner over somedesired value range must be generated to allow for situation-insensitive training.
This paper describes some early results in an ongoing project addressing the above
issues. The project aims to collect a representative data set for different typical context recognition tasks. The data is to be used to derive person-independent features andrecognition algorithms. In addition, we intend to establish a publicly available benchmark database of standard context recognition tasks which can be used to objectively
quantify and compare different approaches. Such benchmark databases are widely andsuccessfully used in automatic speech and image recognition. In the remainder of thispaper, we first outline the experimental considerations and procedures used to record
the data in a controlled manner while observing strict experimental standards. We thendiscuss preliminary results obtained on simple scenario with 8 test subjects. While nostatistically valid conclusions can be drawn from such a small sample, interesting effects
can still be observed. In particular we show that variations between subjects do indeed
have a strong impact on feature selection and recognition performance. This illustrates
the importance of establishing standard data sets as intended by our project.

2

Experimental Procedure

The setup of the experiments was carefully planned and documented based upon trial
runs performed in advance. We also explained the whole procedure in considerable detailto the test subjects beforehand, such that they could easily follow the prescribed routeand tasks. The overall objective of the experiment was to gather acceleration data of
selected body parts in different action contexts for multiple test subjects with varying
physical characteristics. Of course, the whole experimental setup should only have aminimal effect on the normal physical movements of the test subjects if at all. The video
recordings of the test subjects in action show that we fully achieved this aim.
2.1

Hardware Setup

The sensor system employed in the experiments draws its power from a standard USB
connection and interfaces to any RS-232 serial port for control and data signal transmission. We used a Xybernaut Mobile Assistant IV (MA-IV) wearable computer with a
233 MHz Pentium-MMX CPU, 64 MB of RAM, and a 6 GB hard drive running SuSELinux 6.2 to drive the sensor system. Because of the USB and serial ports needed, the full
Xybernaut MA-IV system consisting of both the main unit and the port replicator had
to be strapped to the test subjects. Fortunately, there was enough space left here to also

254

E.A. Heinz et al.

(a) on Right Leg(b) on Left Leg(c) on Right Hip

Fig. 1. Sensor Placement

affix the central components of the sensor system. Due to the professional belt packaging
provided by Xybernaut for its wearable computers, this worked quite smoothly and did
not hinder the test subjects in any perceivable way during the course of the experiment.
Finally, three acceleration sensors were affixed to the test subjects as depicted by thethree photographs shown in Figure 1:(a) on right leg about 1 cm above knee cap, tightly fixed, exactly moving like the leg;(b) on left leg about 1 cm above knee cap, lightly fixed, moving like the trousers;(c) on right hip, tightly fixed, exactly moving like the hip.
The sensors on the legs were oriented with their z-axes in the direction of the movement
and their y-axes pointing up. The sensors on the hip had their x-axes facing up and theirz-axes to the right (i.e., their final orientation differed slightly from the picture above).

Fig. 2. A Single Flight of Stairs

Experimental Evaluation of Variations in Primary Features

255

2.2 Action Sequence
After being fully wired and connected, the test subjects were placed at the start of theexperimental “trail” that led them through parts of Leibniz Hall, the building housing the
School of IT at the International University (IU) in Germany. A small custom program(written in ANSI-C and running directly on the wearable computer) recorded the sensor
readings throughout the whole trail by essentially dumping the raw data from the serial
input port of the Xybernaut MA-IV. The list below characterizes in detail the stages andactions that the test subjects moved through.
1. Start signal: 3x tapping on right knee.
2. Walk straight for about 9.09 m.
3. Open a door with right hand, walk through, and close it (door swings inwardly with
its handle on the left and the hinges on the right side).
4. Turn right and walk straight for about 6.59 m to a staircase as depicted in Figure 2.
5. Climb down 6 flights of 11 stairs each with rightward turns after the first 5 flights(stair width: 30 cm, stair height: 15 cm, stair angle: 27° from the horizontal).
6. Walk straight for about 51.34 m to opposite staircase.
7. Climb up 6 flights of 11 stairs each with left-ward turns after the first 5 flights (samestair specifications as for downward climb).
8. Walk straight at fast pace for about 44.13 m back to the door of the starting room.
9. Stop signal: 3x tapping on right knee again.

2.3

Data Recording and Processing

In order to gain a better understanding of the test subjects themselves and their physical characteristics in particular, we also requested brief individual profiles from themspecifying the following details:– age, gender, height, weight, and right- or left-handedness;– frequency of sports activities (daily, weekly, monthly, yearly, or never on average);– additional information such as injuries, handicaps, nationality, and other specials.
The whole procedure was supervised by at least one experiment conductor, who wasalso responsible for filming the whole sequence with a video camera. Post-experimentactivities included verifying that the recorded files actually contained valid data, visuallychecking the video film for inconsistencies regarding the prescribed sequences of actions,and – last but not least – processing the recorded raw data and transforming it intomeaningful visual representations. This was done using IU Sense [1,2] and MatLab(release 12). IU Sense, developed by IU students and written in Java, is an extensible
real-time application with graphical data displays in various views and user-configurable
layouts (see Figure 3). It also provides numerous possibilities to perform off-line datavisualization from files and conversion into several different file formats.

256

E.A. Heinz et al.

Fig. 3. IU Sense: Graphical Data Display

Experimental Evaluation of Variations in Primary Features

2.4

257

Problems and Future Improvements

The trickiest task of the whole experimental setup is the proper fixation of the sensor
equipment. This still takes too much time and remains somewhat sensitive to humanerror. In the long run, the sensors and their interconnect will be woven into some clothes
directly. Until then, however, we need to be content with less professional yet nevertheless reliable alternatives. Towards this end, we have discussed and tried many different
approaches based on a variety of materials and fixation schemes: strong plaster tapewrapped around the sensors and the test subjects’ body parts, strings tied around theleg and knotted together, so-called “velcro straps” as used for portable music players,
rubber bands shifted over the leg, etc. Solely the first of these approaches, namely thestrong plaster tape, worked reliably enough in practice to be considered adequate by us.
Despite its ad-hoc look, the tape solution works surprisingly well. Its main shortcomings
are prolonged setup times, minor imprecisions in sensor placement, and some issues ofholding strength depending on the test subjects’ type of clothes. A better solution than
the taping might be achieved by means of an elastic kind of bandage to be strapped overthe clothes, with the sensors and their connecting cables permanently affixed to it. Thisis certainly worth a try.
Future experimental runs may also be improved by wireless access to the wearable
computer for remote real-time control and display of the sensor data readings and recordings. Moreover, the Xybernaut MA-IV is still somewhat bulky. A smaller platform, suchas a PDA like the HP-Compaq iPAQ for instance, surely impacts the test subjects evenless. In fact, the iPAQ platform was our initial first choice. Unfortunately, it did not
really work out in practice because some of the available iPAQ devices suffered from
hardware problems connecting to the sensor system. A more light-weight setup would
also allow for more precise and specific body measurements and sensor fixation within
the test subjects’ hip areas.

Central Master
MSP
430F149

Subnetwork
Master in
Subnetwork

Control
LEDs

Filters

3-wire bus/
USART

Gain Amplifiers

Sensor Signal Inputs

Front

Fig. 4. The PadNET Wearable Sensor System

258

E.A. Heinz et al.

3

Sensor Platform

The experiments were conducted using PadNET (“Physical Activity Detection Network”, see Figure 4), a sensor platform developed for user activity recognition [5]. It
has been designed as a wearable system and allows for the easy distribution of multiple
sensors over a person’s body while being flexible. The platform consists of multiple sensor nodes interconnected in a hierarchical network. The purpose of a sensor node is toprovide a physical interface for different sensor types (accelerometers, gyroscopes, magnetic field sensors, etc.), to read out the corresponding sensor signal, to provide certain
computational power for signal pre-processing, and to enable communication with allother network components. Figure 4 shows such a sensor node with its logical block diagram. For the experiments, three 3D-accelerometers ADXL202E from Analog Devices
were used. The analog signals from the sensors were low-pass filtered (fcutoff = 50 Hz)
and A/D-converted with 12-bit resolution at a sampling rate of 100 Hz.

4

Initial Results

In the initial experimental phase of our ongoing project, we recorded data for 8 different
test subjects. Of these 8, one data set has proven unusable due to a technical problem.
The remaining 7 data sets were then examined with respect to three features typicallyused in accelerometric activity recognition:– root mean square (RMS) of the signal, giving the average power of the signal;– cumulative sum over the signal (sums);– variance of the signal.
Of course, with just 7 subjects no statistically valid conclusions may be drawn. However,even with such a small data set useful observations can be made. In particular, proving
the existence of certain phenomena such as feature variations and their consequences onclassification requires isolated examples only.
4.1

Feature Variations

As a first step we investigate the statistical distribution of features for all subjects andfour related context classes: walking (class #2), going down stairs (class #3), going up
stairs (class #4), and walking fast (class #8). This is done for the downward-pointing axis
(which is the most relevant one for all types of walking) of the hip and upper leg sensor.
The results can be seen in Figures 5 to 7. The figures show the the mean value (a point)
and the variance (an error bar) of each feature for all test subjects and context classes.
It is interesting to note that the RMS and sums features seem to show little variations,both for each test subject individually and also between them. The only exception istest subject 6, which is totally out of range. Interestingly, from Table 1 it can be seenthat nothing seems to be particularly special about this test subject except for his lefthandedness which does not provide a plausible explanation for such a different walking
style in our opinion. Taken together, Figures 5 to 7 indicate that variations between
subjects can be expected to be a serious problem for context recognition.

Experimental Evaluation of Variations in Primary Features

Feature: Rms
1.4

1.3

1.2

Feature Value

1.1

1

0.9

0.8
Class: 2
Class: 3
Class: 4
Class: 8

0.7

0.6

Fig. 5. Variance in the RMS Feature

Feature: Sums
240

220

Feature Value

200

180

160

140

120

Class: 2
Class: 3
Class: 4
Class: 8

100

Fig. 6. Variance in the Sums Feature

259

260

E.A. Heinz et al.

Feature: Variance
0.4

0.35

Class: 2
Class: 3
Class: 4
Class: 8

0.3

Feature Value

0.25

0.2

0.15

0.1

0.05

0

Fig. 7. Variance in the Variance Feature
Table 1. Characteristics of the 8 Test Subjects
No. Gender
1
2
3
4
5
6
7
8

4.2

Age Height Weight Sports Handed- Special Notes(Years) (cm)
(kg)
ness
male
24
173
75
weeklyrightflat feet
male
24
182
78
weekly
right
male
36
180 90–100 monthly
rightslightly out-of-sync right foot(injury)
male
65
188
86
weeklyrightfemale23
171
62–65 weekly
right
X-like legs, asymmetrical kneecaps, wearing flip-flop sandals
male
21
172
70
daily
left
male
23
191
110
weeklyright
wearing flip-flop sandals
female23
170
50
weeklyright
wearing skirt

Usefulness for Classification

The question of usefulness of a set of features for person-independent classification
involves at least two prominent aspects. First, we need to find out whether the set isuniversal enough to provide separation between the relevant classes for all subjects. As
a stronger second requirement, we then need to see if a single separation plane can befound for all subjects enabling a person-independent system design. In our experiment,
we have found that for the recognition of any two classes only (in particular, fast walking

Experimental Evaluation of Variations in Primary Features

261

with any other) reasonable separation is achieved for all subjects by a combination of
the variance and sums features. However, combining data from all subjects and thenusing single separation has not produced satisfactory results. When considering all 4classes we have found the features to produce excellent separation for test subject 6. For
all the others the separation was mixed to poor as illustrated in Figures 8 to 10. Thesefigures show the values of the variance and sums features plotted in different colors using
distinct shapes for each class. The main lesson of the above is that features that work
very well for some people might not work at all for others. It also shows that widelyused features such as RMS or variance are not directly suitable for person-independent
systems. Instead, representative data sets are needed to derive more appropriate features
sets.

5

Future Work

Our ongoing research project and collaboration intends to collect further data samples
by repeating the exact experimental procedure described above with many more test
subjects. In the end, we plan to collect enough samples to allow for statistically confident
recognition schemes of the features over the whole test population. Moreover, we will
make the full data and documentation of all the experiments publicly available on theInternet as soon as possible. Due to the careful design and in-depth documentation of
our experimental procedure, we are convinced that our data are of common interest andvalue. They may even serve the purpose of a general benchmark set.
Trial 6190

Class: 2
Class: 3
Class: 4
Class: 8180

170

Feature: Sums

160

150

140

130

120

110

100

0

0.02

0.04

0.06

0.08
0.1
Feature: Variance

0.12

Fig. 8. Good Variance / Sums Classification

0.14

0.16

E.A. Heinz et al.

Trial 8230Class: 2
Class: 3
Class: 4
Class: 8

225

Feature: Sums

220

215

210

205

200

0

0.05

0.1

0.15
0.2
Feature: Variance

0.25

0.3

0.35

Fig. 9. Mixed Variance / Sums Classification

Trial 7235

Class: 2
Class: 3
Class: 4
Class: 8

230
225
220

Feature: Sums

262

215
210
205
200
195
190
185

0

0.05

0.1

0.15
0.2
Feature: Variance

0.25

Fig. 10. Poor Variance / Sums Classification

0.3

0.35

Experimental Evaluation of Variations in Primary Features

263

References
1. A. Caracas, E.A. Heinz, P. Robbel, A. Singh, F. Walter, and P. Lukowicz. Real-time sensor
processing with graphical data display in Java. Submitted to 3rd International Symposium on
Signal Processing and Information Technology (ISSPIT), takes place in December 2003
2. H. Junker, P. Lukowicz, A. Caracas, K.S. Kunze, P. Robbel, A. Singh, S. Sulistyo, F. Walter, andE.A. Heinz. Demonstration of integrated CMOS acceleration sensors and real-time processing
software with graphical data display in Java. Description of live demonstration at 1st European
Symposium on Ambient Intelligence (EUSAI), November 2003
3. N. Kern, B. Schiele, H. Junker, P. Lukowicz, and G. Tröster. Wearable sensing to annotate meeting recordings. In Proceedings of the 6th International Symposium on Wearable Computers(ISWC), pp. 186–193, IEEE Press, October 2002
4. K. van Laerhoven and O. Cakmakci. What shall we teach our pants? In Proceedings of the 4thInternational Symposium on Wearable Computers, pp. 77–83, IEEE Press, October 2000
5. P. Lukowicz, H. Junker, M. Stäger, T.v. Büren, and G. Tröster. WearNET: A distributed multisensor system for context-aware wearables. In Proceedings of the 4th International Conference
on Ubiquitous Computing (UbiComp), G. Borriello and L.E. Holmquist (eds.), pp. 361–370,
Springer-Verlag, LNCS 2498, September 2002
6. A. Madabhushi and J. Aggarwal. Using head movement to recognize activity. In Proceedingsof the 15th International Conference on Pattern Recognition, Vol. 4, pp. 698–701, IEEE Press,
September 2000
7. C. Randell and H. Muller. Context awareness by analysing accelerometer data. In Proceedingsof the 4th International Symposium on Wearable Computers, poster paper, pp. 175–176, IEEE
Press, October 2000