System for Detecting Kindergartners’ Potential Emergency
Situations
Masatoshi Hamanaka, Yuichi Murakami, Atsushi Usami, Yuji Miura, and SeungHee Lee
University of Tsukuba
Abstract This paper describes a system for detecting kindergartners’ potential emergency
situations. The system consists of wearable devices for kindergartners and a computer
server in the kindergarten. We designed and produced a wearable device that includes
sensors for detecting potential emergencies. When a potential emergency is detected, thedevice sends a photo that parents can view to check what is happening at the kindergarten.
We use vertical acceleration and heart rate to recognize the activity modes of the children,and experimental results show that our system enables the recognition of children’s
activity modes with an F-measure of over 0.8.

1.

Introduction

We have constructed a system for detecting potential
emergencies that kindergartners may experience byusing wearable devices that enable the acquisition of
not only activity information but also the biologicalinformation of the wearer. Kindergartners may find it
difficult to explain a situation because they do nothave the verbal ability. Our system enables photos tobe taken of a kindergartner’s emergency situation andsends the photos to the parents in real-time. After thefact, the system can also share with the parents
information about incidents the kindergartner
experienced.

A previous system for identifying human activity
modes can distinguish the modes of running, walking,staring up, and staring down [7]. The automatic
segmentation of group activity can detect the timing ofchanges in the group activities of kindergartners [8, 9],
but these systems [7-9] cannot differentiate between
inside playing and eating. In contrast, human action
recognition using acceleration with physiological data
can differentiate between these activity modes [10-11].
However, it uses a wristwatch-shaped device, which isdifficult to attach to kindergartners because their arms
are usually very short and slim.

For example, if parents receive an emergency e-mailfrom the kindergartener’s device through a server inthe kindergarten, they can check the situation byviewing photographs in real-time. If the situation isreally an emergency, the parents can call emergency
personnel. In another example, if the parents notice a
scratch on the kindergartner’s skin after he or she
returns home, they may not be able to discover thereason due to the limited communication abilities of
the kindergartner. By using our system, the parents
can investigate the reason for the scratch.

We developed a wearable device that has sensors, acamera, triple-axis accelerometer with dual-axis
gyroscope, GPS receiver, and heart rate monitor. The
main advantage of our system is that the server in akindergarten
automaticallyrecognizes
eachkindergartner’s activity mode by using an acceleration
spectrogram and standard deviation of the heart rate.
The server detects a kindergartner’s potential
emergency situation by searching for a child with adifferent activity mode from the other kindergartners.
Experimental results show that our system enables therecognition of children’s activity modes with an Fmeasure of over 0.8.

A previous security camera for deterrent the crime is
only for record the movies and that is not enough toremove the parent’s fear of an accident of thekindergartner [1-3]. On the other hand, Child Watch
Systems [4-6] enable to detect the position of child byusing RFID. However, they [4-6] only detect the
information of the position and cannot detect a
potential emergency of the child.

This paper is organized as follows. We present theoverview of our system in Section 2, explain thedevelopment of the wearable device in Section 3, anddescribe the method for recognizing a child’s activity
mode in Section 4. We then explain the Event Checker inour system in Section 5 and present experimental results
and our conclusion in Sections 6 and 7 respectively.

To detect potential emergencies, it is important toacquire the kindergartner’s activity mode, such as
walking, running, sports training, playing outside,playing inside, or eating. For example, during sports
training, if only one kindergartner stops moving, we
can estimate that the kindergartner may have aproblem and that it may be an emergency.

2. System for Potential Emergencies
An overview of our system for detecting akindergartner’s potential emergency situation is shownin Fig. 1. The wearable device regularly sends activity
information and biological information from its
sensors to the server in the kindergarten. If the server
detects a potential emergency, the device starts to takeParent and child in home
Streaming activity and biologicalinformation regularly

Access information about
incidents at kindergarten

Request to send photographs
in potential emergencies
Emergency e‐mailSend photographs

Server inkindergarten

Child inkindergarten

Check situation
through photographs

Parent in office

Figure 1: Overview of our system for detecting potential emergencies kindergartners may experience.
photographs and sends them to the server. The serverthen sends an emergency e-mail to the parents and theparents can check the situation by downloading thephotograph from the server. The parents can also
check the situation after returning home with thekindergartner by using a viewer that we constructed
for graphically showing the incidents in a day.
3. Developing wearable device
In constructing a wearable device for a kindergartner,
we made a point of using a child-friendly design that akindergartner may delight in putting on by him or
herself. The device must also be unobtrusive in longtime wear because a kindergartner may not understandthe mechanism of and the reason for wearing thedevice. As a result of investigating children’s bodysizes and a questionnaire survey to parents, we
decided to construct a device that is below 100 g andhas a radius under 10 cm.
3.1 Child-friendly design
To construct a device having a child-friendly design,
we followed three requirements.
- The device will not obstruct children’s actions.
- The camera on the device can take accurate
photographs of what a kindergartner is seeing.
- The device will fit closely to the kindergartner’s
body to properly detect acceleration of thekindergartner’s body.
We sketched many designs and narrowed it down tothree design candidates. Figure 2(a) shows a design
called Banana. A cloth or rubber strap fits over thechild’s shoulder, and the body of the device (shaped likea banana) fits on the flank of the child. The Banana fits
the body well, but the position of the camera is a little
low, making it difficult to take photographs of situations
or of the faces of adults in front of the kindergartner
because a kindergartner is about 1 m tall. Figure 2(b)shows a design called Dolphin that improves on the
Banana by adding a strap that wraps around the child’s
body, enabling the camera to be positioned higher.
However, the Dolphin is difficult to adjust for different
body sizes. Figure 2(c) shows a design called Omusubi.
In Japanese, an omusubi is a triangular rice ball. The
Omusubi improves on the Dolphin as the lengths of its
straps are adjustable. Figure 3 is a snapshot of the
Omusubi device that we made being worn.

(a)

(b)

(c)

Figure 2: Child-friendly design.

Funnel I/O

VGA camera module

XBEE RF moduleGPS
Lithium
Polymer battery
Heart rate monitor intercace

Gyroscope withaccelerometer

Figure 5: Circuit in device.
Figure 3: Omusubi device.
3.2 Wireless Mesh Network
We use XBee RF modules [12-13] in wearable devices
for communicating with a server. In comparison with
Wi-Fi modules, XBee RF modules are lightweight (4g) and have low power consumption (40 mA). The
main advantage of using XBee RF modules is thatthey can organize a mesh network dynamically.
In addition, use of a Wi-Fi network is problematic
because Wi-Fi networks have blind spots nearbuilding walls, restricting the coverage. A wirelessmesh network does not have this problem.
Another advantage of XBee RF modules is that, by
use of their mesh network function, information froma wearable device located far from the server can becommunicated to the server by using a wearable
device near the server as a relay point (Figure 4).

Figure 4: Mesh network function of XBee.
3.3 Device circuit
We use the Funnel I/O board to integrate theinformation from sensors. The Funnel I/O board canrun on low voltage (3.3 V) and has a circuit for
charging a lithium polymer battery. The Funnel I/Oboard also has an XBee socket.
We connected the Funnel I/O board to a dual-axis
gyroscope with a triple-axis accelerometer (IMU 5
Degrees, SparkFun), heart rate monitor interface(SEN-08661, SparkFun), GPS (CXD2951GA, Sony),
VGA camera module (SEN-09334, SparkFun), XBee
RF module, and lithium polymer battery. All the
sensors work on the 3.3-V voltage supplied by theconstant voltage circuit on the Funnel I/O board(Figure 5). The device weighs 97 g.

The heart rate monitor interface converts the signal
from the Polar Heart Rate Monitor used, which is beltshaped and needs to be fastened very tightly to the chest.
Because the heart rate monitor may be too
uncomfortable for kindergartners, we reformed it toenable it to be attached to the skin like a sticking plaster.
4. Activity recognition
We differentiated four types of activity mode, walking,outdoor playing, eating, and indoor playing, by using
the data from the accelerometer and heart rate monitor
in the wearable device. We measured the accelerationand heart rate of five five-year-old children (three girls
and two boys). These measurements were conductedin a meeting room of a university, a park near theuniversity, and a walkway in between.
The four types of activity modes were as follows.
Walking: Children walk between the university andpark while holding their parent’s hand.
Outdoor playing: Children play in the park with a ballor playground equipment.
Eating: Children eat lunch while sitting on chairs in
the university meeting room.
Indoor playing: Children fold origami while sitting on
chairs in the university meeting room.
Our proposed algorithm for recognizing activities isshown in Figure 6. First, we differentiate walking andoutdoor playing from eating and indoor playing byusing the data from the accelerometer. Then we
differentiate between eating and indoor playing byusing the data from the heart rate monitor.
4.1 Recognition by acceleration
We use vertical acceleration to differentiate theactivity mode. Figure 7 shows fast Fourier transform(FFT) spectrograms [14] of walking, playing outside,
eating, and playing inside. The vertical axes indicate
the frequency, and the horizontal axes indicate thetime.
The spectrogram for walking (Figure 7(a)) has a peaknear 2 Hz. The spectrogram for outdoor playing(Figure 7(b)) has intermittent peaks near 4 Hz. The
spectrograms for eating (Figure 7(c)) and indoor
playing (Figure 7(d)) do not have peaks.

Here, Ma is an activity mode by acceleration. Ma=1
means playing outside, Ma=2 means walking, and
Ma=3 means eating or playing inside.

Start

4.2 Recognition by heart rate
We differentiate between eating and playing inside byusing the one-minute average of a heart rate Have andthe standard deviation of the heart rate Hstd. Heart rateis measured in beats per minute (BPM).

No
3Hz≧Fs>1Hz

Yes

No

5Hz>Fs>3HzYes

H ave  H ave or H std  H std

No

Yes
Outdoorplaying

Walking

EatingIndoorplaying

Fs: Strongest frequency of vertical acceleration.
Have: Average heart rate.
Have: Normal average heart rate while eating or
indoor playing.
Hstd: Standard deviation of heart rate.
Hstd: Normal standard deviation of heart rate whileeating or indoor playing.
Figure 6: Algorithm of activity recognition.
To acquire the strongest frequency Fs, we use theautocorrelation method [15]. This method estimates
the fundamental frequency (F0) from a speech signal.
It is robust for signals with noise.

kˆ  k X t  X t  k ,(1)argmaxwhere X is one-minute-long vertical acceleration data.
The optimized time lag k̂ is one cycle of the strongest
frequency Fs.
We differentiate among the activity modes of playing
outside, walking, eating, and playing inside with thefollowing expression.

1 5Hz  Fs  3HZM a  2 3Hz  Fs  1Hz
 3 Other frequency
「散歩」(a) Walking(2)「室外遊び」(b) Outdoor playing

Table 1 shows the average and standard deviation of
the heart rates of three children. All the subjects show
the same tendencies: the average heart rate whenplaying inside is higher than when eating, and thestandard deviation when playing inside is smaller thanwhen eating.
Table 1. Average and standard deviation of heart
rate
EatingChild AChild B
Child C

Have
100.9
100.8
97.0

<
<
<

Indoorplaying
Have
110.1
110.0
101.1

Eating
Hstd
5.699
5.710
8.863

Indoorplaying
Hstd> 4.781> 4.817> 5.910

To differentiate between the activity modes of eating
and playing inside, we use the following expression.

1
Mh  
2

H ave  H ave or H std  H std (3)H ave  H ave or H std  H std

Here, Mh is the activity mode by heart rate; Mh =1
means eating, and Mh =2 means playing inside. H ave isthe normal average heart rate in a day when Ma= 3.
H std is the normal standard deviation of heart rate in a
day when Ma= 3.
4.3 Recognition of group activity mode
The activity recognition as described in sections 4.1
and 4.2 does not always output correct results because
heart rate depends on the condition of the child.
To solve this problem, we introduce a group activity
mode. This assumes that children are in the samegroup activity mode if they are in the same classroom.
For example, if most of the kindergarteners in a
「食事」(c) Eating「室内遊び」(d) Indoor Playing(dB)

Figure 7: Spectrograms of acceleration for four activity modes.

classroom are in the eating activity mode, we canestimate that the remaining kindergarteners in theclassroom will also be in the eating activity mode. We
can then estimate that the group activity mode iseating.
However, the activity mode of a kindergarten class
may sometimes be divided almost equally into twomodes, making it difficult to determine the group
activity mode. For example, when eating lunch, somekindergarteners eat very quickly and finish before
their classmates, while others eat very slowly and arestill eating when their classmates have finished; theformer will be recognized as playing indoors, and thelatter will be recognized as eating.
Accordingly, we use changing mode as the group
activity mode when more than 40% of thekindergarteners are in a different activity mode.
5. Event Checker
If a parent receives an emergency e-mail, he or she
can check the situation in the kindergarten by using
the Event Checker in our system. The Event Checker
has two modes: viewing mode and configuring mode.
The viewing mode shows the events in a day
structured by activity mode.
5.1 Viewing mode
Figure 8 shows a screenshot of the viewing mode. In
the viewing mode, the vertical axis indicates time
divided by group activity modes. Each group activity
mode has more than one thumbnail photograph. Eachparent only sees the photographs from the device
attached to his or her own child for privacy reasons.
On the right of the thumbnail photograph is a circle
with stars indicating the distance and position of otherchildren when the photograph was taken. The distance
between children is measured by the radio field
intensities of the XBees in the devices. The maximum
communication distance between XBees is about 30 m.
Therefore, distances of 0–30 m can be detected.
However, if there is an obstacle such as a concrete
wall between two children with devices, the XBees
cannot communicate and the distance cannot bemeasured.
5.2 Configuring mode
In the configuring mode, parents can configure whichsituations should be detected as a potential emergency
and when it is necessary to send an emergency e-mail.
For example, a parent may configure the following: an
emergency e-mail will be sent if the child is more than
10 m from classmates while eating, and an emergency
e-mail will not be sent if the child is more than 10 mfrom classmates while playing outdoors. When achange of group activity mode occurs, the device
automatically takes a photograph and sends it to theserver. The parent can also configure when the camera
will take photographs and when photographs are sent
to the server.

Outdoor playing 11:15 ‐ 11：50 2 photograph(s)Changing

11:50 ‐ 12：00 1 photograph(s)Eating12:00 ‐ 12：50 1 photograph(s)Figure 8: Viewing mode of Event Checker.
6. Experimental Results
We evaluated the performance of our system inrecognizing four types of activity mode (walking,outdoor playing, eating, and indoor playing) using theF-measure, which is given by the weighted harmonicmean of precision P (proportion of selected activity
modes that are correct) and recall R (proportion ofcorrect activity modes that are identified). For theevaluation, we used 10 min of data for each type ofactivity mode from the measured data described in
section 4 and divided it into 1-min intervals. We
attempted to recognize the activity mode of each 1min interval by our method and evaluated theperformance.
Table 2 shows the results of our experiment with threesubjects. For each activity mode, the F-measure isover 0.80, indicating that our system is fairly accurate
in recognizing activity modes.
Table 2: F-measure for recognizing activity
modesChild AChild B
Child C
Average

Outdoorplaying

Walking

EatingIndoorplaying

1.00
0.90
1.00
0.97

1.00
0.90
1.00
0.97

0.80
0.90
0.90
0.87

0.80
0.90
0.90
0.87

7. Conclusion
We have constructed a system for detecting
kindergartners’ potential emergency situations andthat enables emergency e-mails to be sent from aserver in the kindergarten. This system includes a
device we constructed called Omusubi; it is under 100
g and fits well to a kindergartner’s body. If something
happens to a kindergartner wearing Omusubi, a parent
can check the situation by viewing photographs taken
by Omusubi. By using vertical acceleration data andheart rate data, we can recognize four types of activity
mode: outdoor playing, walking, eating, and indoor
playing. Experimental results showed that the Fmeasure of each activity mode of 3 subjects was morethan 0.8.
We plan to construct about 20 devices and test thissystem in an actual kindergarten.
Acknowledgment
This is a contract project with Strategy Information
and Communication R&D promotion programme(SCOPE) by Ministry of Internal Affairs andCommunications for 2009-2010.
References[1] Ismail Haritaoglu, David Harwood and Larry S.
Davis. "W4: real-time surveillance of people andtheir activities", IEEE Transactions on PatternAnalysis and Machine Intelligence, Vol. 22, No.
1, pp809-830, 2000.
[2] Ali Benkhalil, Stanley Ipson and William Booth.
"Real-time video surveillance system using afield programmable gate array", International
Journal of Imaging Systems and Technology,
Vol. 11, No. 2, PP.130-137, 2000.
[3] Bangjun Lei and Li-Qun Xu: "Real-time outdoor
video surveillance with robust foregroundextraction and object tracking via multi-statetransition management". Pattern Recognition
Letters, Vol. 27, Issue 15, pp. 1816-1825, 2006.
[4] Yukio Okamura, Toru Yamano, Hiroaki Kubota
and Takeshi Doumen."An Information System
for Children Tracking on the Way to and Backfrom School", IEEJ Transactions on Electronics,
Information and Systems, Vol. 129, Issue 7, pp.
1371-1376, 2009.
[5] Dirk Haehnel, Wolfram Burgard, Dieter Fox,
Ken Fishkin and Matthai. Philipose. "Mappingand Localization with RFID Technology". IEEE
International Conference on Robotics andAutomation (ICRA), pp. 1015-1020, 2004.

[6]

Ari Juels. "RFID Security and Privacy: A
Research Survey". Journal of Selected Areas inCommunication (J-SAC), Vol. 24, No. 2, pp381395, 2006.
[7] Hiroshi
Kanasugi
and
Ryosuke
Shibasaki."Measurement and Analysis of
Human
Behavior
Using
WearableSensors",Proceedings of the 25th Asian
Conference on Remote Sensing,Vol. 2, pp.12181223, 2004.
[8] Kazuya Kousaka, Kazuhiro Imaki, Kazuhiro
Imaki,
Hirohide
Haga
andShigeoKaneda."Classification of Children'S Group
Activity from acceleration data by usin Wavelet
tranformation", Information Processing Societyof Japan SIG Technical Notes, Vol. 2009, No.
13, pp. 1-8, 2009 (in Japanese).
[9] Kazuya Kousaka, Kazuhiro Imaki, Kazuhiro
Imaki,
Hirohide
Haga
andShigeoKaneda."Automatic segmentation of group
activity based on acceleration",Proceedingsof the 23rd Annual Conference of theJapanese
Societyfor
ArtifcialIntelligence,2C3-4, 2009.
[10] Makoto Sato, Chie Morita and Miwako
Doi,"Human
actionrecognitionusingphysiologicalandaccelerationdata",
Information Processing Society of Japan 72nd
National Convention, 3T5B-2, 2003 (inJapanese).
[11] Kazushige Ouchi, Takuji Suzuki, Miwako Doi,"LifeMinder : A Wearable Healthcare Support
System with Timely Instruction Based on the
User's Context", IEICE transactions on
information and systems, E87-D(6), pp.13611369, 2004.
[12] Digi internatinal. "Xbee Zigbee/Mesh RF
Modules",
http://www.digi.com/pdf/ds_xbeemeshmodules.
pdf[13] Digi internatinal. "White paper: Demystifying
802.15.4 and ZigBee”,
http://www.digi.com/register/index.jsp?mu=/pdf/wp_zigbee.pdf[14] Sean A. Fulop and Kelly R. Fitz."Algorithms forcomputing the time-corrected instantaneous
frequency (reassigned) spectrogram, withapplications", Acoustical Society of America,
2006.
[15] Bracewell, R. "The Autocorrelation Function."
The Fourier Transform and Its Applications.
New York: McGraw-Hill, pp. 40-45, 1965.