Activity Recognition and Nutrition
Monitoring in Every Day Situations
with a Textile Capacitive Neckband
Jingyuan Cheng
Sebastian Wille
German Research Center for Microelectronic Systems
Artificial Intelligence (DFKI) Design Research Group
Trippstadter Str. 122, 67663, Erwin-Schrödinger-Str., 67663,
Kaiserslautern, Germany
Kaiserslautern, Germany
jingyuan.cheng@dfki.dewille@eit.uni-kl.de
Bo Zhou
Norbert Wehn
German Research Center for Microelectronic Systems
Artificial Intelligence (DFKI) Design Research Group
Trippstadter Str. 122, 67663, Erwin-Schrödinger-Str., 67663,
Kaiserslautern, Germany
Kaiserslautern, Germany
bo.zhou@dfki.de
wehn@eit.uni-kl.dePaul Lukowicz
Kai Kunze
Department of Computer
German Research Center for
Science and Intelligent Systems Artificial Intelligence (DFKI)
Osaka Prefecture University
Trippstadter Str. 122, 67663,
1-1 Gakuencho, Naka, Sakai Kaiserslautern, Germany
Osaka 599-8531, Japan
paul.lukowicz@dfki.de
kai.kunze@gmail.com
Carl Christian Rheinlnder
Microelectronic Systems
Design Research Group
Erwin-Schrödinger-Str., 67663,
Kaiserslautern, Germany
carl@cc-software.de
Copyright is held by the author/owner(s).
UbiComp ’13 Adjunct, Sept 8-12, 2013, Zurich, Switzerland.
ACM 978-1-4503-2139-6/13/09...$15.00.

Abstract
We build on previous work [5] that demonstrated, in
simple isolated experiments, how head and neck related
events (e.g. swallowing, head motion) can be detectedusing and unobtrusive, textile capacitive sensor integrated
in a collar like neckband. We have now developed asecond generation of the band that allows long term
recording in real life environments in conjunction with alow power Bluetooth enabled smart phone. It allows thesystem to move from the detection of individual swallows
which is too unreliable for practical applications to ananalysis of the statistical distribution of swallow frequency.
Such an analysis allows the detection of ”nutrition events”
such as having lunch or breakfast. It also allows us to see
the general level of activity and distinguish between justbeing absolutely quiet (no motion) and sleeping, asperiodic ”empty swallows” occur more seldom when aperson is sleeping.

Author Keywords
Activity recognition, nutrition monitoring, capacitive
sensing, wearable computing

ACM Classification Keywords
I.2.m [Artificial Intelligences]: Miscellaneous.

Introduction
In [5] we have proposed a novel sensing system that uses
textile electrodes integrated in an unobtrusive collar tomeasure the capacitance of the users’ neck. In a nutshell
it works like a capacitive touchpad except that theelectrodes are looking ”inwards” into the neck reacting tochanges in neck shape and inner composition (e.g. whenfood is swallowed). In constrained lab experiments wehave demonstrated the detection of head and neck related
events ranging from head motions, through speaking andcoughing to swallowing.
In this demo we build on the above work presenting an
improved version of the systems that supports long termmonitoring of selected behaviours with a neckband
connected to t smart phone using a low power Bluetooh
module. A core application idea is that while thedetection of individual swallows and head motion works
poorly in real life data streams, a statistical distribution ofswallow frequency, time between swallows and headmotion can be detected reliably enough to be a good
indication of certain activities. In an initial expriment witha data set 138 hours from 3 subjects we were able todetect meals, sleeping (including the distinction between
just being absolutely quiet (no motion) and actuallysleeping), and spot three activity levels: fully quiet (e.g.
watchingTV), normal (e.g. working on a computer), andbeing highly physically active (e.g. walking).
Related WorkA widely used wearable sensing modality are arm/wrist
mounted motion sensors which allow the detection ofnutrition related gestures [1]. Unfortunately nutrition
related gesture can vary widely and resemble to manyother gestures (e.g. scratching ones’ head). To overcome
such problems multi modal approaches are often

successful. Thus, for example, Liu et. al., used a
microphone and an egocentric camera to detectfood-intake [6]. Of course, one cannot wear a camera
everywhere (privacy problems) and it might be tricky totell from the recorded images if the user really eats somefood or just looks at it [4]. More reliable is chewing
detection using bode conducted sounds from the ear [2],which however involves a more invasive sensor system.
Another more obtrusive approach is a sensor worn in themouth to detect chewing [7]. In lab environments,
electrodes mounted on the neck can also reliably detectswallowing and with it eating [3].

System design
As shown in Figure 1 the system consists of a textile
neckband with 4 capacitive sensors, operating at asampling rate of 25Hz. Compared with the systemdescribed in [5], it has been modified in several ways.
First, instead of flexible but not stretchable metal textile,
we made the sensor pads out from stretchable conductive
material (Silver plated 92% Nylon 8% Dorlastan fabric)and embedded them in a stretchable neckband withbuttons for length adjustment. Connecting wires aresewed to the pads with conductive thread (Silver Plated
Nylon 117/17 2ply.). Thus the band is more comfortable.
Second, the sensor has been equipped with a Bluetooth
4.0 interface for data transmission to a dedictated
IPHONE App.
Third, we use one common ground for all the fourchannels to reduce noise. Because the system is battery
powered its ground is floating. The bigger the area of thepads’ ground planes, the better the system is coupled tohuman body. We put two sensor pads in the center, wherethe larynx moves through when swallowing, and anothertwo across sternomastoids, which also moves withswallowing, and at the same time changes when turning
the head aside.
Finally, the 2nd and 3rd stages of the amplification wereremoved and di↵erent settings for the Colpitts oscillator
were chosen to reflect that we are now mostly looking for
swallowing, which is a mechanical change at the surface
level and thus much stronger then the type of signals that
we were interested in [5] (e.g. pulse).

Single swallows A model of swallowing is built for eachsubject based on the training set, simply by aligning alllabeled swallows then calculate their average (shown as
thick black lines in Figure. 2, the other lines are thealigned labelled swallows). Half of the maximum value ofthe sum of all 4 channels has been determined from thetraining data as a threshold to select segments possiblycontaining swallows. To those segments we apply a J48
tree classifier trained with the training set. The features
include: the RMS, the mean value, the highest peak’s
channel index, its peak value and the width, thebandwidth, frequency centroid, the amplitude di↵erence
to the model.

Figure 1: The material, the sensor pad design and the readyneckband worn by a test subject

Data Processing
The data stream is evaluated in 3 consecutive steps. Firstis the spotting of single swallows. Next is the extractionof short windows (1.5min) classified on the basis ofswallow frequency and some additional features as (1)eating, (2) sleeping and (3-5) three levels of physicalactivity (quiet, normal, active). Finally we extend theanalysis to longer windows (up to 8min) and the detectionof nutrition events. In doing so we distinguish between
major meals (breakfast, lunch, dinner) that should be
spotted reliably and short snacks which are less distinct.

Figure 2: Swallow model of a female (top) and male (bottom)
subject.

Short window analysis Running a 1.5 min window
moved in 20 sec steps over the detected swallows wecompute the swallow rate and the time interval between
swallows. The classification of the 1.5 min windows is
performed using 10-fold cross-validation method with J48Tree classifier. The features are: the RMS of eachchannel, the percentage of signals higher than the overall
RMS, the numbers of swallows, the mean and RMS oftime interval between swallows, the mean of the shortest

and longest swallow intervals (bottom/top 1/3).
Event based meals spotting The event spotting isessentially based on smoothing the 1.5 sec recognition
results, merging events that are closer then a certain
threshold and throwing away all segments where theresulting eating events are shorter then a certain time
threshold. The remaining segments are the meal events.
These predicted meals are then compared to the realeating periods noted down by subjects.
A threshold is then set from 0 minute to 8 minutes, ateach threshold, only real meals longer than this threshold
are taken into consideration for the evaluation. When thethreshold is 0, all food/water intakes are considered, andwhen the threshold is as high as 7 minutes, only the major
meals (breakfast, lunch, dinner) are considered.
Initial Results
3 healthy subjects (2 males, 1 female, aged 24-34 years)
took part in the experiment. Each has worn the neckband
throughout 3 days. The data includes 3 bigmeals (breakfast, lunch and dinner) as well as light snacks
like banana, apple and yoghurt. The subjects have carriedout their usual daily activities and note the time andduration of food intakes and ”big” drinks (up 5-6 sips,small drinking is neglected) and key activities performed
during the day (work in front of computer, shopping,walk to mensa and etc.). Overall the data encompasses
138 hours with 25 big meals (eating longer than 6minutes), and 39 smaller food or water intakes (theintaking process takes less than 6 minutes). One subject
has also worn the neckband at night (2 nights). For
training an laboratory data set was recorded for eachsubject that includes eating bread, apple, banana anddrinks a cup of water. The training data are companiedwith video record and all swallows are labeled.

The overall accuracy for all 3 subject and 5 states in the
1.5min windows is 84.4%, with class 1 and 2 missing for
subject 3. Over all 3 test subjects, we are able torecognize 24 out of 25 big meals (recall 0.96), with 3 falsepredications (precision 0.89); and 46 out of all 64
food/water intakes, with 136 false predictions.

References[1] Amft, O., Junker, H., and Troster, G. Detection ofeating and drinking arm gestures using inertialbody-worn sensors. In Proc. IEEE ISWC (2005),
160–163.
[2] Amft, O., Stäger, M., Lukowicz, P., and Tröster, G.
Analysis of chewing sounds for dietary monitoring. In
UbiComp 2005: Ubiquitous Computing. Springer,
2005, 56–72.
[3] Amft, O., and Troster, G. On-body sensing solutions
for automatic dietary monitoring. Pervasive
Computing, IEEE 8, 2 (2009), 62–70.
[4] Arab, L., Estrin, D., Kim, D. H., Burke, J., andGoldman, J. Feasibility testing of an automatedimage-capture method to aid dietary recall. European
journal of clinical nutrition 65, 10 (2011), 1156–1162.
[5] Cheng, J., Amft, O., and Lukowicz, P. Active
capacitive sensing: Exploring a new wearable sensing
modality for activity recognition. In Pervasive
Computing. Springer, 2010, 319–336.
[6] Liu, J., Johns, E., Atallah, L., Pettitt, C., Lo, B.,
Frost, G., and Yang, G.-Z. An intelligent food-intake
monitoring system using wearable sensors. In Proc.
BSN 2012, IEEE (2012), 154–160.
[7] Stellar, E., and Shrager, E. E. Chews and swallows
and the microstructure of eating. The American
Journal of Clinical Nutrition 42, 5 (1985), 973–982.