Journal of Robotics, Networking and Artificial Life, Vol. 1, No. 1 (June 2014), 68-72

Heart-pulse Biofeedback in Playful Exercise
using a Wearable device and Modular Interactive Tiles
Tomoya Shimokakimoto*

Graduate School of Systems & Information Engineering, University of Tsukuba,
1-1-1 Tennodai, Tsukuba 305-8573, JapanHenrik Hautop Lund

Centre for Playware, Technical University of Denmark,2800 Kgs. Lyngby, Denmark

Kenji Suzuki

Center for Cybernics Research, University of Tsukuba,
1-1-1 Tennodai, Tsukuba 305-8573, Japan/ Japan Science and Technology Agency,Kawaguchi Center Building 4-1-8, Honcho, Kawaguchi 332-0012, Japan*

Corresponding author. E-mail: shimokakimoto@ai.iit.tsukuba.ac.jp
Tel: +81-29-853-5679

Abstract
We developed a playful biofeedback system using a wearable device and modular interactive tiles. In this approach
we suppose that patients could regulate exercise intensity on t heir own through biofeedback. We propose
biofeedback play system called “bioToys” based on exercise with the modular interactive tiles. The system consistsof a wearable device that measures heart-pulse via ear-mounted sensor, and modular interactive tiles which are used
for physical rehabilitation exercise through playing a game. The wearable devise enables detection of heart pulse inreal-time and therefore provides heart beat rate during playful activities, even if the heart pulse wave have motion
artifacts. The tiles are designed to build flexible structures and to provide immediate feedback based on the users’
physical interaction with the tiles. We combine the two systems to provide users with heart pulse biofeedback in
playful exercise. We show that using the developed system it is possible for the users to regulate the exerciseintensity on their own with biofeedback, and also possible to analyze exercise activity using number of steps on thetiles and heart beat rate.
Keywords: playware, biofeedback, exercise, rehabilitation

Published by Atlantis Press
Copyright: the authors68

1. Introduction
In order to improve the health and quality of life
(QOL) of elderly people and patients with metabolic
disorders, stroke and cardiac diseases, physical training
and exercise is required in daily life. Heart Beat Rate(HBR) is usually used as a standard for regulating theexercise intensity1. Thus, there are some commercial
wearable devices of measuring HBR. And some
exercise systems that control the exercise intensity
based HBR are worth consideration2, 3.
For practical implementation of wearable sensing
technologies in physical exercise, there is a n eed formethods to simplify measurement of bio-signals, as wellas signal processing techniques to reduce motion
artifacts in order to provide stable biofeedback. Systems
such as those in Refs.4 – 5 try to take thoseconsiderations into account.
In physical exercise, it is not easy to maintainparticipants’ motivation for joining a sport. 6 Recent
researches also utilized games as means for exercise,showing a number of cases where gaming may bebeneficial for health.
In this paper, we propose the hypothesis that patients
could regulate exercise intensity on their own throughbiofeedback during playful activity, and we propose abiofeedback play system called “bioToys” based on
exercise with a wearable device and the modular
interactive tiles.
2. System Configuration
2.1. Wearable Device
We developed a wearable device to measure heart
pulse and generate biofeedback in real-time. The
wearable device consists of an ear-mounted sensor (earsensor, Combi Wellness Corp.), a processor(LPC1752FBD80, NXP), a light emitting diode forbiofeedback and Bluetooth module for communication
with PC. Fig. 1 shows the overview of the wearable
device for heart pulse measurement and biofeedback.
Heart pulse measurement via ear-mounted sensor isprone to noise resulting from body movements, thus it isimportant to process the heart pulse waveform in realtime to generate a s table biofeedback output. In ourdevice we implemented an algorithm based on linear
prediction and an adaptive template matching method toFig.1. (a) Overview of the wearable device in a sports
wrist pocket (b)the circuit board and heart pulse
sensorovercome noise in the waveform 5. The algorithm iscapable of detecting actual edges in the heart pulse
waveform in real-time, even if the pulse waveform
include motion artifacts similar to heart pulse. Thismethod utilizes human physiological features togenerate a correct cyclic pulse from noisy signals. Thisalgorithm is mainly constructed by two equations (seeequation 1 a nd 2). The equation 1 represents adifferential equation of linear prediction and theequation 2 means a template matching.
In equation 2, P(t) is the pulse waveform vector
measured by the A/D converter with the mean value
normalized to zero. When P includes a cyclic motion
artifact such as those resulting from locomotion or
jumping, a mere thresholding method is not adequate todistinguish such artifacts from heart pulse. The
developed algorithm, by linear prediction, avoids
detecting artifacts similar to heart pulse. In the linearprediction method we designed a Kalman filter
according to equation 1, which is a differential equation
model for calculating an estimate time interval of theheart pulse[s] depending on previous detected
time intervals. This linear prediction method is capableof approximately predicting the next heart pulse interval
according to the pattern of the heart pulse7. Therefore,an edge of heart pulse waveform is detected after[s], which is the sum of estimated time anddifference between the previous estimated time andactual detected pulse interval.
In addition, for reducing noise effect around an edgeof heart pulse waveform, this algorithm uses ideal pulse
waveform vectors for heart rates (τs[bpm]) that don’t
include noise: I(t;τs). Then, using the correlation value
between I and P; M(τs) in equation 2, when the value ofM(τ) is higher than a threshold, the system detects anedge of heart pulse waveform.

Published by Atlantis Press
Copyright: the authors
69

Using the linear prediction and template matching
methods, this wearable device is able to detect edges ofheart pulse and calculate HBR stably. LEDs on thedevice blink in synchronization with the detected heart
pulse, and the HBR data is transmitted to PC via
Bluetooth in real-time.

(1)

Fig.2. Modular interactive tiles used for playful
physiotherapy with feetinteractive tiles can register whether they are placed
horizontally or vertically, and they can adapt thesoftware games to behave accordingly.

(2)

2.2. Modular Interactive Tiles
We developed a system for playful biofeedback
based on modular interactive tiles. The tiles attach toeach other to form the overall system, and they aredesigned to be flexible for different arrangements. Thetiles are also motivating to the user since they provide
immediate feedback based on the users’ physicalinteraction with the tiles, which is a design principle ofmodular playware8. F ig. 2 shows a scene of playing
with the tiles.
Each tile comprises a sensor and transmitter forcommunicating with other tiles, has a q uadratic shapemeasuring 300mm*300mm*33mm, a force sensitive
resistor (FSR) and eight RGB LEDs that emit light withequal spacing in between on a ci rcle shape for gamefeedback.
On the circuit board of each tile there are also
connectors to mount an XBee radio communication
add-on board, including the MaxStream XBee radio
communication chip. Hence, there are two types of tiles,
those with a radio communication chip (master tiles)
and those without one (slave tiles). The master tile may
communicate with the game selector box and initiate thegames on the built platform. Every platform has to haveat least one master tile if communication is needed togame selector box or a PC. The tile becomes a mastertile only when it has XBee radio communication
connected to it.
The modular interactive tiles can be easily set up onfloor. They can be simply attached to each other as a
jigsaw puzzle, and there are no wires. The modular

2.3. Real-time biofeedback for exercise system
The games on modular tiles are designed forphysical exercise. B asically when users touch or stepon the tiles, the tiles generate feedback through LEDs’
color and intensity according to the game’s rules thatare installed in tiles. We developed a playful
biofeedback system by combining a wearable device formeasurement of heart pulse rate with the modular
interactive tiles. In this paper, we modified the games
installed in modular interactive tiles9 for some new
purposes as will be illustrated.
The wearable device transmits the calculated HBR
to PC via Bluetooth as the wearable device detects
users’ heart pulse. After the PC receives data of HBR,
data is logged in the PC and transmitted to the mastertile wirelessly via XBee. When the master tile receives
the heart beat data, it in terprets it a nd light the LEDs
according to the game rules. This system has six colors
for biofeedback. Fig. 3 shows an example of colorassignment based on HBR in reference to ratings ofperceived exertion scale, which is a scale of perceived
physical exertion relating to HBR10.
For biofeedback, HBR is assigned by color and thegame level or score point. Modular interactive tiles havegames based on counting steps or touches, or based ontime constraints. For example, there are “color racegame” and “final countdown”. Color race is based on
counting steps. When the user steps on an activated orilluminated tile, user gets score. Finally when the user
acquires an adequate score the game finishes
successfully. By adding the wearable heart beat device,
the user is able to know his heart pulse and HBR andshare it with audience.

Published by Atlantis Press
Copyright: the authors
70Fig.4. The scene in the playing using a wearable device and modular interactive tiles: Tiles’ LED color
was changed from green to yellow depending on user’s HBR value.
Final countdown game is based on time constraint.
In this game, all tiles are activated and the eight LEDs
on each tile count down independently. When any tilefinishes to count down it will send a “game over”
message to the master tile and the game will finish.
When the user steps on a t ile, count down time in thistile will reset and the eight LEDs on this tile are lit
again. For this game, in normal case, count down speed
parameter is fixed depending on game difficulty. For
regulating the exercise, we designed a g ame where theuser keeps playing as long as proper HBR is observed.
In this case, we designed the countdown parameter tochange depending on the HBR. At the first level that isunder 70 bpm (color is purple), the counts down interval
is 2s. T hen for the following levels (light blue, blue,green) the countdown time decreases 0.32s respectively.
Considering the fourth level (green, 100bpm) as thedesired level, count down timer will increase for laterlevels (yellow and red) also by 0.32s respectively, inorder to keep the user at the desired level.

3.

Experiment

proposed system and to go up t o the third level (greencolor level) and keep this level. We logged heart beatpulse by 120Hz, HBR, the number of steps and alsorecorded these experiments on video for analysis.
3.2. Result and analysis method
Fig. 4 shows a scene during the experiment. We
confirmed that the proposed system provide color
feedback dependent on HBR during the game. Fig. 5
shows all participants’ results of this experiment. Herewe compare the number of steps and HBR amongparticipants. In this experiment, the results of participant
A and B were similar. Participant C (who exercises onregular bases) did not get tired during the experiment.
On the other hand, participant D who does not usually
exercise was tired from the point of HBR.
Then we focused on the time series of steps and HBR.
Fig. 6 and 7 show the results of two participants (A andB). The top graph shows the events of detected heart
pulse, the HBR is expressed in the color of biofeedback.
The bottom graph shows the sum value of number ofsteps for each 10 seconds.
In Fig.6, when participant A reached green

3.1. Regulating exercise
In order to evaluate the performance of theproposed system, we conducted an evaluation
experiment to verify the feasibility of the proposedsystem as a p layful biofeedback system for regulating
exercise. In this experiment, we used six tiles and final
countdown game. We asked four healthy maleparticipants to exercise for two minutes with theFig.3. Example of a color assignment based on HBR

Published by Atlantis Press
Copyright: the authors
71

Fig.5. The result of all participantslevel, he intentionally decreased his pace to the desired
HBR. In participant B’s case, first he overshoot to thered level for around 20s, then he decreased his pace tokeep the desired HBR. Finally, he was able to find theproper pace to stay between green and yellow levels.
The proposed system showed the possibility for theusers to regulate the exercise intensity during playful
activity on their own with biofeedback.
4. Conclusion
In this paper we proposed a biofeedback play systemcalled “bioToys” based on exercise with modular
interactive tiles and a wearable device. We consider thatgames as a mean for exercise are beneficial for health,and for maintaining the motivation for exercise.
We developed a wearable device that measures heart
pulse via ear-mounted sensor, and we implemented analgorithm for biofeedback in the real time which detects
heart pulse in intense activity. We also developed asystem for playful feedback game based on modular
interactive tiles. Then we combined the two systems anddeveloped games to provide users with heart pulse
biofeedback in playful exercise.
We devised an evaluation experiment to verify
the feasibility of the proposed system for regulatingexercise. We showed that it is possible to use thedeveloped system to regulate the exercise intensity on
one’s own with biofeedback, and it is also possible toanalyze exercise activity using number of steps on thetiles and HBR.
References
1. Karvonen, J., and Vuorimaa, T., Heart Rate and Exercise
Intensity During Sports Activities., Sports Medicine, 5(5)(1988) 303–311.
2. Su, S., W., and Wang, L., and Celler, B., G., and Savkin,A., V., Heart rate control during treadmill exercise. Proc.
of Annual Intl. Conf. of the IEEE EMBS (China,Shanghai, 2005), pp. 2471–2474.
3. ElSamahy, E. and Genedy, A. and Abbass, M.A. andGaddallah, M., A Computer–based System for Safe
Physical Fitness Evaluation, Proc. of Annual Intl. Conf.
of the IEEE BMEI (USA, Boston 2011), pp. 1443–1447.
4. Watanabe, M., and Tsukamoto, S., and Hoshino, H., A
handle electrode system for measuring heart rate whileriding a bicycle, Proc. of Annual Intl. Conf. of the IEEE
BMEI (Canada, Vancouver, 2008), pp. 542–545.
5. Shimokakimoto, T., and Ayuzawa, S., and Suzuki, K.,
Real-time Pulse Detection for Physiothrapy and its
Application toWearable Device, IPSJ Journal, 54(4)(2013) 1480–1488 (in Japanese).
6. Dishman, R.K., Exercise adherence: Its impact on publichealth, Human Kinetics Publishers. (1988).
7. Kobayashi, M., and Musha, T., 1/f Fluctuation ofHeartbeat Period, IEEE transactions on biomedicalengineering, 29(6) (1982) 456–457.
8. Lund., H., H., and Marti, P., Designing modular robotic
playware, the IEEE Int. Workshop Robots Human
Interactive Commun (Japan, Toyama, 2009), pp. 115–121.
9. Lund., H., H., Modular Interactive Tiles forRehabilitation: Evidence and Effect, Proc. of the 10th
WSEAS Intl. Conf. on ACS, (Japan, Iwate, 2010), pp.
520–525.
10. Borg, G., Perceived exertion: a note on "history" andmethods., Med. Sci. Sports., 5(2) (1973) 90–93.

Fig.6. The result of participant A in two minutes exercise

Fig.7. The result of participant B in two minutes exercise

Published by Atlantis Press
Copyright: the authors
72