System for Detecting Kindergartners’ Potential Emergency Situations
M asatoshi HAM ANAKA
Department of Intelligent Interaction Technology,University of Tsukuba
Tsukuba,Ibaraki,Japan
Yuichi M URAKAM I
Department of Intelligent Interaction Technology,University of Tsukuba
Tsukuba,Ibaraki,Japan
Dahyun KIM
Department of Kansei,Behavioral and Brain Science,University of Tsukuba
Tsukuba,Ibaraki,Japan
Yuji M IURA
Department of Intelligent Interaction Technology,University of Tsukuba
Tsukuba,Ibaraki,Japan
Kazuya Atsuta
Department of Intelligent Interaction Technology,University of Tsukuba
Tsukuba,Ibaraki,Japanand
SeungHee LEE
Department of Kansei,Behavioral and Brain Science,University of Tsukuba
Tsukuba,Ibaraki,Japan

ABSTRACTThis paper describes a system for detecting kindergartners’
potential emergency situations. The system consists ofwearable devices for kindergartners and a computer server inthe kindergarten.W e designed and produced a wearable device
that incl
udes sensors for detecting potentialemergencies.W hen
a potentialemergency is detected, the device sends a photo thatparents can view to check what is happening at thekindergarten.W e use verticalacceleration and heart rate torecognize the activity modes ofthe children, and experimental
results show that our system enables the recognition ofchildren’s activity modes with an F-measure ofover 0.
8.
Keywords:PotentialEmergency Situation, W earable Device,
Activity M ode, Acceleration Sensor, Hear Rate Sensor andGPS(GlobalPositioning System)Receiver.

For example, ifparents receive an emergency e-mailfrom thekindergartener’s device through a server in the kindergarten,
they can check the situation by viewing photographs in realtime.Ifthe situation is really an emergency, the parents cancallemergency personnel.In another example, ifthe parents
notice a scratch on the kindergartner’s skin after he or she
returns home, they may not be able to discover the reason due
to the limited communication abilities ofthe kindergartner.By
using our system, the parents can investigate the reason for thescratch.
A previous security camera for deterrent the crime is only forrecord the movies and that is not enough to remove the parent’s
fear ofan accident ofthe kindergartner [1-3].On the other hand,Child W atch Systems [4-6] enable to detect the position ofchild by using RFID.However, they [4-6] only detect the
information of the position and cannot detect a potential
emergency ofthe child.

1.INTRODUCTION
W e have constructed a system for detecting potential
emergencies that kindergartners may experience by using
wearable devices that enable the acquisition of not only
activity information but also the biologicalinformation ofthe
wearer. Kindergartners may find it difficult to explain asituation because they do not have the verbalability.Our
system enables photos to be taken of a kindergartner’s
emergency situation and sends the photos to the parents inreal-time.After the fact, the system can also share with theparents information about incidents the kindergartner
experienced.

ISSN: 1690-4524

To detect potentialemergencies, it is important to acquire thekindergartner’s activity mode, such as walking, running, sports
training, playing outside, playing inside, or eating.For example,
during sports training, ifonly one kindergartner stops moving,
we can estimate that the kindergartner may have a problem andthat it may be an emergency.
A previous system for identifying human activity modes candistinguish the modes of running, walking, staring up, andstaring down [7].The automatic segmentation ofgroup activity
can detect the timing of changes in the group activities ofkindergartners [8, 9], but these systems [7-9] cannot
differentiate between inside playing and eating.In contrast,

SYSTEMICS, CYBERNETICS AND INFORMATICSVOLUME 9 - NUMBER 2 - YEAR 201139

human action recognition using acceleration with physiological
data can differentiate between these activity modes [10-11].
However, it uses a wristwatch-shaped device, which is difficult
to attach to kindergartners because their arms are usually very
short and slim.
We developed a wearable device that has sensors, a camera,
triple-axis accelerometer with dual-axis gyroscope, GPSreceiver, and heart rate monitor. The main advantage of oursystem is that the server in a kindergarten automaticallyrecognizes each kindergartner’s activity mode by using an
acceleration spectrogram and standard deviation of the heartrate. The server detects a kindergartner’s potential emergency
situation by searching for a child with a different activity mode
from the other kindergartners. Experimental results show that
our system enables the recognition of children’s activity modes
with an F-measure of over 0.8.
This paper is organized as follows. We present the overview ofour system in Section 2, explain the development of thewearable device in Section 3, and describe the method forrecognizing a child’s activity mode in Section 4. We thenexplain the Event Checker in our system in Section 5 andpresent experimental results and our conclusion in Sections 6
and 7 respectively.

2. SYSTEM FOR POTENTIAL EMERGENCIES
An overview of our system for detecting a kindergartner’s
potential emergency situation is shown in Fig. 1. The wearable
device regularly sends activity information and biologicalinformation from its sensors to the server in the kindergarten. If
the server detects a potential emergency, the device starts totake photographs and sends them to the server. The server thensends an emergency e-mail to the parents and the parents cancheck the situation by downloading the photograph from theserver. The parents can also check the situation after returninghome with the kindergartner by using a viewer that weconstructed for graphically showing the incidents in a day.

3. DEVELOPING W EARABLE DEVICE
In constructing a wearable device for a kindergartner, we
made a point of using a Kids-friendly design that akindergartner may delight in putting on by him or herself.
The device must also be unobtrusive in long-time wear
because a kindergartner may not understand the mechanismof and the reason for wearing the device. As a result ofinvestigating children’s body sizes and a questionnaire survey
to parents, we decided to construct a device that is below 100
g and has a radius under 10 cm.
Kids-friendly design
To construct a device having a Kids-friendly design, we
followed three requirements.
-

The device will not obstruct children’s actions.

-

The camera on the device can take accurate photographsof what a kindergartner is seeing.

-

The device will fit closely to the kindergartner’s bodyto properly detect acceleration of the kindergartner’sbody.

We sketched many designs and narrowed it down to threedesign candidates. Figure 2(a) shows a design called Banana.
A cloth or rubber strap fits over the child’s shoulder, and the
body of the device (shaped like a banana) fits on the flank of
the child. The Banana fits the body well, but the position of
the camera is a little low, making it difficult to takephotographs of situations or of the faces of adults in front of
the kindergartner because a kindergartner is about 1 m tall.
Figure 2(b) shows a design called Dolphin that improves on
the Banana by adding a strap that wraps around the child’s
body, enabling the camera to be positioned higher. However,the Dolphin is difficult to adjust for different body sizes.
Figure 2(c) shows a design called Onigiri. In Japanese, an
Onigiri is a triangular rice ball. The Onigiri improves on theDolphin as the lengths of its straps are adjustable. Figure 3 isa snapshot of the Onigiri device that we made being worn.

Figure 1:Overview ofoursystem fordetecting potential emergencieskindergartners mayexperience.

40

SYSTEMICS, CYBERNETICS AND INFORMATICSVOLUME 9 - NUMBER 2 - YEAR 2011

ISSN: 1690-4524

(c) Banana

Figure 3: Onigiri device.
Wireless Mesh Network
We use XBee RF modules [12-13] in wearable devices forcommunicating with a server. In comparison with Wi-Fi
modules, XBee RF modules are lightweight (4 g) and have low
power consumption (40 mA). The main advantage of using
XBee RF modules is that they can organize a mesh network
dynamically.

(a) Dolphin

In addition, use of a Wi-Fi network is problematic because WiFi networks have blind spots near building walls, restricting thecoverage. A wireless mesh network does not have this problem.
Another advantage of XBee RF modules is that, by use of theirmesh network function, information from a wearable device
located far from the server can be communicated to the server
by using a wearable device near the server as a relay point(Figure 4).

(b) Onigiri

Figure 4: Mesh network function of XBee.

Device circuit
We use the Funnel I/O board to integrate the information fromsensors. The Funnel I/O board can run on low voltage (3.3 V)
and has a circuit for charging a lithium polymer battery. The
Funnel I/O board also has an XBee socket.

Figure 2: Kids-friendly design.

ISSN: 1690-4524

We connected the Funnel I/O board to a dual-axis gyroscopewith a triple-axis accelerometer (IMU 5 Degrees, SparkFun),heart rate monitor interface (SEN-08661, SparkFun), GPS(CXD2951GA, Sony), VGA camera module (SEN-09334,
SparkFun), XBee RF module, and lithium polymer battery. All
the sensors work on the 3.3-V voltage supplied by the constantvoltage circuit on the Funnel I/O board (Figure 5). The device
weighs 97 g.

SYSTEMICS, CYBERNETICS AND INFORMATICSVOLUME 9 - NUMBER 2 - YEAR 2011

41

Funnel I/O

VGA camera module

Start
XBEE RF module

No
3HzӍFs>1Hz

GPS

Yes

Lithium
Polymer battery
Heart rate monitor intercace

No5Hz>Fs>3Hz

Gyroscope withaccelerometerYes

H ave  H ave or H std ! H std

Figure 5: Circuit in device.

No

Yes
The heart rate monitor interface converts the signal from thePolar Heart Rate Monitor used, which is belt-shaped and needs
to be fastened very tightly to the chest. Because the heart ratemonitor may be too uncomfortable for kindergartners, wereformed it to enable it to be attached to the skin like a sticking
plaster.

Outdoorplaying

The four types of activity modes were as follows.
Walking: Children walk between the university and park while
holding their parent’s hand.
Outdoor playing: Children play in the park with a ball or
playground equipment.
Eating: Children eat lunch while sitting on chairs in theuniversity meeting room.
Indoor playing: Children fold origami while sitting on chairs in
the university meeting room.
Our proposed algorithm for recognizing activities is shown in
Figure 6. First, we differentiate walking and outdoor playing䛂ᩓṌ䛃(a) Walking䛂ᐊእ㐟䜃䛃(b) Outdoor playing

EatingIndoorplaying

Fs:Strongestfrequencyofverticalacceleration.
Have:Averageheartrate.
Have:N orm alaverageheartratew hileeatingor
indoorplaying.
Hstd:Standarddeviationofheartrate.
Hstd:N orm alstandarddeviationof heartratew hileeatingorindoor playing.

4. ACTIVITY RECOGNITION
We differentiated four types of activity mode, walking, outdoor
playing, eating, and indoor playing, by using the data from theaccelerometer and heart rate monitor in the wearable device.
We measured the acceleration and heart rate of five five-yearold children (three girls and two boys). These measurements
were conducted in a meeting room of a university, a park near
the university, and a walkway in between.

W alkingFigure 6: Algorithm of activity recognition.
from eating and indoor playing by using the data from theaccelerometer. Then we differentiate between eating and indoor
playing by using the data from the heart rate monitor.
Recognition by acceleration
We use vertical acceleration to differentiate the activity mode.
Figure 7 shows fast Fourier transform (FFT) spectrograms [14]of walking, playing outside, eating, and playing inside. The
vertical axes indicate the frequency, and the horizontal axes
indicate the time.
The spectrogram for walking (Figure 7(a)) has a peak near 2
Hz. The spectrogram for outdoor playing (Figure 7(b)) has
intermittent peaks near 4 Hz. The spectrograms for eating䛂㣗஦䛃(c) Eating䛂ᐊෆ㐟䜃䛃

(d) Indoor Playing

㻔㼐㻮㻕

Figure 7: Spectrograms of acceleration for four activity modes.

42

SYSTEMICS, CYBERNETICS AND INFORMATICSVOLUME 9 - NUMBER 2 - YEAR 2011

ISSN: 1690-4524

(Figure 7(c)) and indoor playing (Figure 7(d)) do not havepeaks.
To acquire the strongest frequency Fs, we use theautocorrelation method [15]. This method estimates thefundamental frequency (F0) from a speech signal. It is robust
for signals with noise.

kˆk X t  X tk ,(1)argmaxwhere X is one-minute-long vertical acceleration data.
The optimized time lag k̂ is one cycle of the strongest
frequency Fs.
We differentiate among the activity modes of playing outside,
walking, eating, and playing inside with the following
expression.

Ma­1 5Hz ! Fs ! 3HZ°®2 3Hz t Fs ! 1Hz° 3 Other frequency¯(2)Here, Ma is an activity mode by acceleration. Ma=1 means
playing outside, Ma=2 means walking, and Ma=3 means eatingor playing inside.
Recognition by heart rate
We differentiate between eating and playing inside by using theone-minute average of a heart rate Have and the standard
deviation of the heart rate Hstd. Heart rate is measured in beats
per minute (BPM).
Table 1 shows the average and standard deviation of the heart
rates of three children. All the subjects show the sametendencies: the average heart rate when playing inside is higher
than when eating, and the standard deviation when playing
inside is smaller than when eating.
Table 1. Average and standard deviation of heart rate
EatingChild A

Have
100.9

<

IndoorplayingHave
110.1

EatingHstd
5.699

Indoorplaying
Hstd> 4.781Child B

100.8

<

110.0

5.710

> 4.817

Child C

97.0

<

101.1

8.863

> 5.910

To differentiate between the activity modes of eating andplaying inside, we use the following expression.

Mh

­1
®
¯2

H ave  H ave or H std ! H std (3)H ave t H ave or H std d H std

Here, Mh is the activity mode by heart rate;Mh =1 means eating,and Mh =2 means playing inside. H ave is the normal averageheart rate in a day when Ma= 3. H std is the normal standard
deviation of heart rate in a day when Ma= 3.

ISSN: 1690-4524

Recognition of group activity mode
The activity recognition as described in sections 4.1 and 4.2
does not always output correct results because heart ratedepends on the condition of the child.
To solve this problem, we introduce a group activity mode.
This assumes that children are in the same group activity mode
if they are in the same classroom. For example, if most of thekindergarteners in a classroom are in the eating activity mode,
we can estimate that the remaining kindergarteners in theclassroom will also be in the eating activity mode. We can thenestimate that the group activity mode is eating.
However, the activity mode of a kindergarten class may
sometimes be divided almost equally into two modes, making it
difficult to determine the group activity mode. For example,
when eating lunch, some kindergarteners eat very quickly andfinish before their classmates, while others eat very slowly andare still eating when their classmates have finished;the former
will be recognized as playing indoors, and the latter will berecognized as eating.
Accordingly, we use changing mode as the group activity mode
when more than 40% of the kindergarteners are in a different
activity mode.

5. EVENT CHECKER
If a parent receives an emergency e-mail, he or she can checkthe situation in the kindergarten by using the Event Checker inour system. The Event Checker has two modes: viewing modeand configuring mode. The viewing mode shows the events ina day structured by activity mode (Figure 8).
Viewing mode
Figure 8 shows a screenshot of the viewing mode. In theviewing mode, the vertical axis indicates time divided by group
activity modes. Each group activity mode has more than one
thumbnail photograph. Each parent only sees the photographsfrom the device attached to his or her own child for privacy
reasons.
On the right of the thumbnail photograph is a circle with stars
indicating the distance and position of other children when thephotograph was taken. The distance between children ismeasured by the radio field intensities of the XBees in thedevices. The maximum communication distance between
XBees is about 30 m. Therefore, distances of 0–30 m can bedetected. However, if there is an obstacle such as a concrete
wall between two children with devices, the XBees cannot
communicate and the distance cannot be measured.
Configuring mode
In the configuring mode, parents can configure whichsituations should be detected as a potential emergency andwhen it is necessary to send an emergency e-mail. For example,
a parent may configure the following: an emergency e-mailwill be sent if the child is more than 30 m from classmates
while eating, and an emergency e-mail will not be sent if thechild is more than 30 m from classmates while playing
outdoors. When a change of group activity mode occurs, thedevice automatically takes a photograph and sends it to theserver. The parent can also configure when the camera willtake photographs and when photographs are sent to the server.

SYSTEMICS, CYBERNETICS AND INFORMATICSVOLUME 9 - NUMBER 2 - YEAR 201143

Viewing mode

Login page

Login page

Figure 8: Event checker

44

SYSTEMICS, CYBERNETICS AND INFORMATICSVOLUME 9 - NUMBER 2 - YEAR 2011

ISSN: 1690-4524

6. EXPERIMENTAL RESULTS
We evaluated the performance of our system in recognizing
four types of activity mode (walking, outdoor playing, eating,and indoor playing) using the F-measure, which is given by
the weighted harmonic mean of precision P (proportion of
selected activity modes that are correct) and recall R(proportion of correct activity modes that are identified). For
the evaluation, we used 10 min of data for each type ofactivity mode from the measured data described in section 4
and divided it into 1-min intervals. We attempted torecognize the activity mode of each 1-min interval by our
method and evaluated the performance.
Table 2 shows the results of our experiment with threesubjects. For each activity mode, the F-measure is over 0.80,
indicating that our system is fairly accurate in recognizing
activity modes.

7. CONCLUSION
We have constructed a system for detecting kindergartners’
potential emergency situations and that enables emergency emails to be sent from a server in the kindergarten. Thissystem includes a device we constructed called Onigiri; it isunder 100 g and fits well to a kindergartner’s body. If
something happens to a kindergartner wearing Onigiri, aparent can check the situation by viewing photographs taken
by Onigiri. By using vertical acceleration data and heart ratedata, we can recognize four types of activity mode: outdoor
playing, walking, eating, and indoor playing. Experimental
results showed that the F-measure of each activity mode of 3
subjects was more than 0.8.
We plan to construct about 20 devices and test this system inan actual kindergarten.

8. ACKNOWLEDGEMENT
This is a contract project with Strategy Information andCommunication R&D promotion programme (SCOPE) by
Ministry of Internal Affairs and Communications for 20092010.

9. REFERENCES
[1]

[2]

[3]

Ismail Haritaoglu, David Harwood and Larry S. Davis.
"W4: real-time surveillance of people and theiractivities", IEEE Transactions on Pattern Analysis andMachine Intelligence, Vol. 22, No. 1, pp809-830,
2000.
Ali Benkhalil, Stanley Ipson and William Booth.
"Real-time video surveillance system using a field
programmable gate array", International Journal of
Imaging Systems and Technology, Vol. 11, No. 2,
PP.130-137, 2000.
Bangjun Lei and Li-Qun Xu: "Real-time outdoor
video surveillance with robust foreground extraction
and object tracking via multi-state transition
management". Pattern Recognition Letters, Vol. 27,
Issue 15, pp. 1816-1825, 2006.

ISSN: 1690-4524[4]

Yukio Okamura, Toru Yamano, Hiroaki Kubota andTakeshi Doumen."An Information System forChildren Tracking on the Way to and Back from
School",IEEJ
Transactions
on
Electronics,
Information and Systems, Vol. 129, Issue 7, pp. 13711376, 2009.
[5] Dirk Haehnel, Wolfram Burgard, Dieter Fox, Ken
Fishkin and Matthai. Philipose. "Mapping and
Localization with RFID Technology". IEEE
International Conference on Robotics and Automation(ICRA), pp. 1015-1020, 2004.
[6] Ari Juels. "RFID Security and Privacy: A Research
Survey". Journal of Selected Areas in Communication(J-SAC), Vol. 24, No. 2, pp381-395, 2006.
[7] Hiroshi
Kanasugi
and
Ryosuke
Shibasaki."Measurement and Analysis of Human
Behavior Using Wearable Sensors",Proceedings of the25th Asian Conference on Remote Sensing,Vol. 2,
pp.1218-1223, 2004.
[8] Kazuya Kousaka, Kazuhiro Imaki, Kazuhiro Imaki,
Hirohide Haga and Shigeo Kaneda."Classification ofChildren'
S Group Activity from acceleration data byusin Wavelet tranformation", Information Processing
Society of Japan SIG Technical Notes, Vol. 2009, No.
13, pp. 1-8, 2009 (in Japanese).
[9] Kazuya Kousaka, Kazuhiro Imaki, Kazuhiro Imaki,
Hirohide Haga and Shigeo Kaneda."Automatic
segmentationofgroup
activity
basedonacceleration",Proceedings of the 23rd Annual
Conference of the Japanese Society for Artifcial
Intelligence,2C3-4, 2009.
[10] Makoto Sato, Chie Morita and Miwako Doi,"Human
actionrecognitionusingphysiologicalandacceleration data", Information Processing Society ofJapan 72nd National Convention, 3T5B-2, 2003 (inJapanese).
[11] Kazushige Ouchi, Takuji Suzuki, Miwako Doi,"LifeMinder : A Wearable Healthcare Support Systemwith Timely Instruction Based on the User's Context",
IEICE transactions on information and systems, E87D(6), pp.1361-1369, 2004.
[12] Digi internatinal. "Xbee Zigbee/Mesh RF Modules",
http://www.digi.com/pdf/ds_xbeemeshmodules.pdf[13] Digi internatinal. "White paper: Demystifying
802.15.4
andZigBee”,
http://www.digi.com/register/index.jsp?mu=/pdf/wp_z
igbee.pdf[14] Sean A. Fulop and Kelly R. Fitz."Algorithms forcomputing the time-corrected instantaneous frequency(reassigned)
spectrogram,withapplications",
Acoustical Society of America, 2006.
[15] Bracewell, R. "The Autocorrelation Function." The
Fourier Transform and Its Applications. New York:
McGraw-Hill, pp. 40-45, 1965.

SYSTEMICS, CYBERNETICS AND INFORMATICSVOLUME 9 - NUMBER 2 - YEAR 201145