Infant Behavior Simulation:
Computational Approach to Infant Safety
Yoshifumi Nishida∗1,∗2 , Koji Kitamura∗3,∗4,∗1 , Yoichi Motomura∗1,∗2 , Altion Simo∗1,∗2 , Tatsuhiro Yamanaka∗5,∗1∗1 Digital Human Research Center,

National Institute of Advanced Industrial Science and Technology (AIST)
2-41-6 Aomi, Koto-ku, Tokyo 135-0064, Japan∗2 CREST Program, Japan Science and Technology Agency (JST)∗3 Tokyo University of Science
2641 Yamazaki, Noda-shi, Chiba 278-8510, Japan∗4 Research Fellow of the Japan Society for the Promotion of Science (JSPS)∗5 Ryokuen Children’s Clinic
2-1-6 Ryokuen, Izumi-ku, Yokohama-shi, Kanagawa 245-0002, Japan

Abstract— A computational approach must be taken in order
to comprehensively understand the behavior of infants andchildren. This approach is also required for the clarificationof the dynamics of a system that includes behavior-related
accidents. The present paper describes an infant behavior model
for simulating infant behavior in a virtual environment. The
determinant factors of infant behavior are classified into internal
factors, such as the physical and cognitive capabilities estimatedfrom the age of the infant, and external factors, such as visual
stimuli from surrounding objects. An infant behavior simulator
that models the causal connections among these internal andexternal factors is described herein. The present paper reports
the performance evaluation of the developed simulator anddescribes its perspective. This simulator will enable the analysis
of infant accidents at home from an engineering perspective
and will facilitate the design of rooms and houses that aresafer for infants. Furthermore this simulator can be used asa starting point for developing a robot behavior model for theliving environment and the care of humans.
Keywords— Computational theory of behavior, Infant behavior simulation, Infant behavior model, Unintentional injury,
Infant accident

I. I NTRODUCTION
A recent report [1] states that unintentional injury is theleading cause of death among children younger than 14
years of age, even though the rate of injury has declined
approximately 40% over the last ten years. In the UnitedStates, unintentional injury claims more than 5,600 lives
a year, or an average of 15 children each day. In Japan,
unintentional injury is also the leading cause of death amongchildren and accounts for 21.7% of child fatalities. The
present report clarifies the necessity for recognizing childinjury as a preventable public health issue, rather than as theresult of uncontrollable ”accidents”.
When we consider accidents of any type, all accidents arecomprised of three phases; 1) Before the accident 2) At thetime of the accident, and 3) After the accident. Before an
accident, we must focus on prevention. When an accident
has happened, we must focus on emergency treatment. Afteran accident, we must focus on healing and rehabilitation. The
most economical of these three actions is prevention.
In order to prevent the unintentional injury of children
and design safer environments for children, we must develop
techniques for understanding child behavior. However, despite
our familiarity with infant behavior in a living environment,
we possess a limited understanding of the dynamics ofchild behavior in a living environment and accident-related
behavior among children.
Research on infants1 has been carried out in the medical
field. Statistical data on infant accidents [2] and on the development of infant behavior are available; thus, many preventive
measures have been proposed. However, since no accurate
tool has been developed for observing the wide variety of
infant behaviors that occur in a living environment, we lackquantitative data on infant behavior in a living environment.
This results in difficulty in comprehensively understanding
infant behavior that occurs in a living environment.
Conversely, in the field of cognitive science, research on
infant behavior has been carried out. For example, detailed
investigation of infant vision, has clarified the following
facts. The use of stimulus information for stereoscopic depth
perception appears to develop between 3 and 4 months ofage [3], [4], while the use of stimulus information for pictorial
depth information appears to develop between 6 and 7 monthsof age [5]. To individuate objects around themselves, infants
can use shape information at 7 months, texture information
at 11 months, and color information at 12 months [6]. It isalso known that the distance to objects, the size and shapeof objects, and whether objects are moving or not have anaffect on the reaching behavior of infants [7]. Despite thisresearch, the infant behaviors or capabilities addressed in thisfield are very limited when compared with the diversity of
infant behaviors that occur in a living environment.
As a consequence, there is a considerable gap between
1 In this paper, the term ”infant” refers to children under three years of age

Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

1/12

II. C OMPUTATIONAL T HEORY OF H UMAN B EHAVIOR
Computational theory is a concept for explaining human activities from the information-processing perspective advancedby Marr [8]. In order to simulate human behavior in a virtuallycreated environment, it is necessary to develop a computational theory of human behavior not only for describing
human behavior but also for explaining and generating it.
Before explaining how this problem can now be addressed, we
will briefly describe the history of the computational theoryof human.
The history of the computational theory of human canbe described from the following three perspectives: 1) the

Recent Development ofComputational Theory
Based on Brain Reductionism

Brain Phenomena Sensing
by fMRI

Computational Theory of
Human Behavior in Laboratory

Physical Phenomena Sensing in Whole Spaceby Ubiqutous Technology
Computational Theory of

Human Behavior in Living SpaceMeso-scopic Computational Theory of
Human Behavior

Promising Development ofComputational Theory of
Human Behavior

the behavioral phenomena investigated in neuroscience andcognitive science and the statistical investigation of theoccurrence of accidents. To bridge this missing link, infant behavior and infant-behavior-related accidents must be
described ”mesoscopically,” namely, more macroscopicallythan is the case in neuroscience and cognitive science, andmore microscopically than is the case in the investigation of
statistical data on accidents.
The present research uses a computational approach inwhich a computer is used to generate an explanation of thephenomena from the viewpoint of information-processing andcomputational theory. This allows the mesoscopic investigation of infant behavior by integrating the existing knowledge
on infant behavior from various research fields and analyzing
infant behavior more comprehensively than is currently possible in research fields such as cognitive science or the developmental behavior of children. The present study attempts toestablish both a computational approach to comprehensively
and mesoscopically understanding child behavior and the
dynamics of a system that takes infant behavior into account.
This paper also emphasizes the necessity of using sensing
technology in order to describe infant behavior mesoscopically in accordance with the computational approach to infant
behavioral science.
This paper presents an infant behavior model for simulating
infant behavior in a virtual environment. This paper is organized as follows. In Section II, we describe the possibility ofdeveloping a new computational theory of infant behavior asa key component of the infant behavior simulator. Herein, we
broaden our perspective to include a computational theoryof human behavior and describe why we should now beable to undertake the difficult task of developing such atheory. In Section III, we analyze the factors related to infant
injury accidents in the home and clarify the key factors
to be considered in infant behavior simulation. Section IVshows the infant observation system for tracking not only
an infant but also the objects handled by the infant inorder to analyze infant behavior in a living environment.
Section V presents the developed infant behavior model for
infant behavior simulation. A performance evaluation of thedeveloped simulator is shown in Section VI. In Section VII,
we describe the perspective of the infant behavior simulation.
Our conclusions are described in Section VIII.

Social Phenomena Sensing in Whole Worldby Internet Technology

Macro-scopic Comuputational Theoryof Human Activity

Fig. 1.

Computational theory of human behavior

emergence of a new observation device, 2) the emergenceof a medium for representing human behavior, and 3) a
mesoscopic phenomena to be investigated.
The emergence of fMRI, which enables us to observe
brain activity in a living human, and computers enabling
us to model and represent brain activity, has lead to therecent development of the computational theory for explaining
mesoscopic phenomena between neurons and brain activity.
In addition, the emergence of robots, which enable us torepresent both brain activities and body activities, has leadto the computational theory for explaining the mesoscopic
phenomenon of physical embodiment. Today, it can be said
that ”brain reductionism”, which refers to the trend in researchof attempting to describe phenomena at the level of brain
activity, has become a strong paradigm in the computational
theory of human.
Recently, two new forms of observation technology haveemerged; ubiquitous sensing technology [9], [10] and World
Wide Web technology [11]. Ubiquitous sensing technology
enables us to observe physical phenomena in a whole space
using multiple sensors embedded in the space. It can be used
for observing human behavior in a living environment fromthe microscopic perspective2. In contrast, Web technology
2 Micro-, meso-, or micro- are relative concepts. In Section I, we referto behavior description by ubiquitous sensing as ”mesoscopic”. However,herein, we refer to it as ”microscopic” in comparison with the macroscopic
descriptions created from statistical data.

Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

2/12

enables us to observe social phenomena across the world
by collecting a large amount of data via the Web. It canbe used for observing human activities from a macroscopic
perspective. In addition, game technology has become available as a medium for representing human behavior. Today’s
game technology has rich capabilities for expressing humanbehavior as computer graphics.
The emergence of these technologies implies that we candevelop a new computational theory of human behavior ina living environment, which is a mesoscopic phenomenon
that exists between microscopically described human behavior
and macroscopically described human activities. The authors
believe that a computational theory of human behavior, comparable to that of human brain science, will be developed.
The present paper deals with infant behavior for the following reasons. Firstly, developing a computational theory of
infant behavior is not only a new aim in academic research but
is also an aim strongly requested by aging or aged societies.
As infants have the highest rate of unintentional-injury-related
death, aging or aged societies are deeply concerned about
infant accidents.
The second reason is that a computational theory of infant
behavior will be one of the most effective methods forscientifically approaching the problem of infant injuries or
accidents that occur in a living environment, as it is impossible to analyze accidents by creating ”artificial accidents.” In
addition, a number of accidents occur due to a combinationof multiple factors, such as infant abilities, environmental
factors, and parental behavior. Computational representation
enables us to comprehensively integrate these factors using
a computer and conduct an analysis-by-synthesis in order toinvestigate and clarify the dynamics of a system of accidents.
The final reason for developing a computational theory of
infant behavior is that we are likely to succeed in developing
such a theory. The data, knowledge, and sensing technologies
required for developing such a theory are available. For
example, the developmental behavior of infants has beenclarified [12], [13]. As mentioned above, infant injuries or
accidents that occur in a living environment are phenomenathat can be described microscopically and macroscopicallyusing available sensing technologies. Microscopic description
should be performed by observing infant behavior in a living
environment using ubiquitous sensing devices, while macroscopic description should be performed through the collectionof accident data from hospital databases. In addition, in theearliest stage of the science of infant-related accidents, overly
complex factors, such as the high level of cognitive abilities
of adults, are avoidable without deteriorating the usefulnessof the model. These complex factors can be integrated into
the computational theory incrementally following the development of the basic theory.
The present paper describes our research on the creation of
a computational theory of infant behavior as a key componentof infant behavior simulation.

III. I NFANT-B EHAVIOR -R ELATED ACCIDENTS
A. Classification of infant accidents
From the viewpoint of interaction with objects and otherpersons, infant accidents at home can be classified into threegroups.
• Accidents due to interaction with objects
This group includes accidents due to single objects (e.g.,
being burned by a stove), and accidents due to multiple
objects (e.g., an infant steps on a toy on the floor, andhits his or her head on a desk while falling.)• Accidents due to interaction with others
Examples of this group of accidents include being bitten,
or kicked by a brother or sister.
• Accidents due to a combination of interaction with
objects and others
An example of this type of accident results when an
infant is pushed by another young child and falls down
the stairs.
This paper focuses on accidents due to interaction with
objects since this type of accident occurs most frequently
according to statistical data. Moreover, these accidents are
addressed first because they involve the fewest factors.
B. Analysis of infant accidents due to interaction with objects
Let us consider a concrete example of an accident in orderto analyze key factors in infant accidents due to interaction
with objects.
”A child wanted to drink orange juice and found a cup
in the cabinet. He climbed up on a chair to get the cup, but
he couldn’t reach it. So he stood on tiptoe. Then he lost hisbalance and fell to the floor. He hit his forehead against thefloor.”
From this example, it can be seen that the accident is deeply
related to the objects around the infant and his capabilityof performing behaviors. In this case, objects such as thecup, the chair, the cabinet, and the floor were involved in theaccident. The boy grew until he could walk, climb, and standon his tiptoes. This fact suggests that we must model both
environmental factors and developmental behavior factors.
C. Key factors in infant accidents due to interaction with
objects
1) Environmental factors: In our research, we believe thatan object has three types of functions for modeling environmental factors in infant accidents: 1) psychological/cognitive
function, 2) physical function, and 3) physiological function.
Figure 2 shows our concept of object functions.
Object functions are related to the ”Cause of the infant’s
intention”, ”Cause of the physical realization”, and ”Causeof the accident”, respectively. The ”Cause of the infant’s
intention” means that an infant performs an action because
he or she is attracted to an object. For example, when an
infant finds a ball on which his favorite animated characteris printed, he or she will enthusiastically try to reach it. The”Cause of the physical realization” means that infant uses an

Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

3/12

object for realizing his or her intention. For example, if theball is put on a shelf, he or she will try to reach it using achair. The ”Cause of the accident” means that the infant isinjured by an object. For example, an infant can be injuredby the corner of a shelf if he or she hits his or her head
when falling from a chair. Thus, object functions are related
to infant’s behaviors and accidents.
This view of objects in the environment is based on theconcept of affordance [14], which refers to the environmental
property that makes a person do or not do something. The
authors believe that this perspective is one of the available
perspectives for modeling the environment. Moreover, thisperspective is useful for application in the present researchbecause we are able to concretely define such functions using
accident data collected by hospitals.
Of the three object functions, this paper deals with thepsychological/cognitive function of objects as the first stepin modeling all functions. In Section IV, we analyze therelationship between an infant’s interest in an object and the
distance to the object.

This paper utilizes the knowledge of the development of
infant behavior that is known in the medical field. In Section
V, we describe the details of how we model developmental
behavior factors.
AgeFirst (%)

Second (%)

Third (%)

Fourth (%)

Fifth (%)

Respiratory
disturbance /Angiopathy(16.1)

Sudden death
syndrome

Accident

0

Birth defect /Chromosomal
aberration(36.3)

(8.1)

(8.1)

Fetal
homorrhagic
disorder(4.1)

Accident

Birth defect /Chromosomal
aberration(17.6)

Malformation
neoplasmaHeart
disease

(7.5)

(6.0)

(5.0)

Malformation
neoplasmaHeart
diseaseMurder

1-4(24.8)
Accident(35.0)(17.2)

Birth defect /Chromosomal
aberration
(7.9)

(5.5)

(3.8)

Accident

Malformation
neoplasmaHeart
disease

Suicide(22.0)(21.2)

(9.5)

(9.2)

Birth defect /Chromosomal
aberration
(6.5)

4-9

10 - 14

Fig. 3.

Pneumonia

Resent trend in the cause of death among children in Japan

4,500
4,000
3,500

Frequency

Physiological Function

Causeinfant’s intention

The cornerof shelf hurts
an infant head
Functionsof
ObjectThe ball makes
infant want tothrow.
Psycological / Cognitive
Function

3,000
2,500
2,000
1,500
1,000
500
0
0

1

2

3

4

5

6

7

8

9

10 11

12 13 14

Age in years

Cause
accident

Causephysical realization

Fig. 4. Relationship between the frequency of injury and age in Victoria,
Australia from June 2002 to July 2003 (This data was provided by theMonash University Accident Research Center (MUARC))

Physical Function

The chair is used
for taking a ballplaced on shelf.

IV. O BSERVATION AND A NALYSIS OF I NFANT B EHAVIOR
IN A L IVING E NVIRONMENT
A. Development of an infant behavior observation room

Fig. 2.

Object functions

2) Developmental behavior factors: As indicated above,
infant accidents are related to the developmental level of
infant behavior. Statistical data also shows this fact. Figure 3
shows the recent trend in the cause of death among children.
The figure indicates that the leading cause of death amongchildren is not disease but unintentional accidents. Figure 4
indicates more clearly which age group of children is more
likely to be involved in accidents.
The behavioral ability of infants develops more rapidly thanthat of both older children and adults. Therefore, even though
their behavior is much simpler than that of adults, it is difficultfor parents and even infants themselves to predict infants’behavior and its results.

Observing infant-object interaction is important in order tocreate a model of how objects affect infant behavior. Therefore, we constructed a behavior observation room in orderto study how infants interact with objects and furniture. Thissection briefly introduces the developed observation hardware
and analysis software. This section also presents the obtained
data on infant behavior transition and the relationship between
an infant’s interest and his or her distance from an object as
typical examples of the results of infant behavior analysis.
These findings were utilized in order to create a computationalmodel of infant behavior. The details of this model aredescribed in Section V.
Figure 5 shows an experimental behavior observation room,in which ultrasonic location sensors [15] [16] and a camera
are installed. The ultrasonic location system consists of an

Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

4/12

ultrasonic receiving system, an ultrasonic transmission system, a time-of-flight measurement system, a network, and apersonal computer. The ultrasonic receiving system receives
ultrasonic pulses emitted from the ultrasonic transmitters andamplifies the received signal. The time-of-flight measurement
system records the travel time of the signal from transmission
to reception. The network synchronizes the ultrasonic location
system and collects time-of-flight data from the ultrasonicreceiving system. The positions of objects are calculated
based on more than three time-of-flight results. The sampling
frequency of the proposed ultrasonic location system is 50 Hz.
The proposed system can detect the positions of objects andinfants within 30 mm for all objects or infants to which
ultrasonic transmitters are attached. The transmitter is shownin Fig. 6.

camera equipped with a fish-eye lens and for collecting synchronous infant and object position data from the ultrasoniclocation system. The software also has a function that helps
the user to label behavior while replaying the captured images
and trajectories. The labels indicate semantic informationsuch as ”Sit”, ”Stand”, ”Run”, ”Interest in Object”, and”Interest in Mother”. These labels were selected in order toreduce difficulty in differentiating labels among analyzers.
Specifically, these labels were selected because they showedcorrelation coefficients over 0.7 in both intra-observer andinter-observer reliability evaluations [17]. The labels arerecorded by associating the video frame number with thetime. Figure 8 shows examples of the infant activities capturedby the fish-eye camera. Figure 9 shows an example of themeasured trajectories of an infant, a mother, and objects.
The lower image shows video data captured from a camera
installed on the ceiling. The upper left image is zoomed inon the infant using information from a sensor that is attachedto the back of the infant. The upper right image shows theinfant’s trajectory.
Zoom around infant

Infant trajectory

Fig. 5. Behavior observation room equipped with ultrasonic location sensors
and a camera

Cognitive/Behavior/Object label

30mm

Fig. 7.

27mm
Fig. 6.

Ultrasonic location sensor developed by the authors

B. Analysis of infant behavior data
1) Data analysis software: Figure 7 shows the softwarethat we developed in order to analyze infant behavior. The
software has functions for capturing video images from a

Software for capturing and analyzing infant behavior

2) Results of analysis: Figure 10 shows an example of astate transition diagram obtained from the observation of a2-year-old infant.
Figure 11 shows the time rate of infant’s interest. It indicates that the infant maintained an almost constant interest in
objects. In this paper, we indicated that the infant is interested
in an object if he or she looks at the object, although we can
not be certain whether the infant is actually interested in theobject. Figure 12 shows the interest induction of an object inrelation to the infant’s distance from the object. The y-axis iscalculated using the following equation to eliminate the effectof the arrangement of the objects.
P (Interest|Distance) =P (Interest ∩ Distance)
P (Distance)

The conditional probability P (Interest|Distance) is utilized

Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

5/12

Sit

0.80
0.25

0.15

Stand

0.01

0.21
1.00

0.64

0.68

Lean

0.95

0.75

0.02

Walk

0.04
0.05

0.01
0.14

0.05

0.25

Run

Climb up
0.75

0.25

Lie down
1.00
Climb
down

Fig. 10.

State transition diagram of experimental data gathered during 2 hours of monitoring a 2-years-old subject

in order to create the computational model of infant behavior
in Section V.
Figure 12 confirms that infants are more likely to beinterested in nearer objects while maintaining interest in themother independent of the distance to the mother. The lower
part of Fig. 13 shows an example of the visualization of an
infant’s interest distribution in a case in which the infant, themother, and the objects are arranged as shown in the upper
part of the figure.
Uninterest 0.3%

Fig. 8.

Interest to mother 19.2%

Examples of infant activities

4000
3500

InfantMotherObject1
Object2
Object3
Object4
Object5
Object6
Object7
Object8
Object9Object10
Object11
Object12
Object13
Object14
Object15
Object16
Object17
Object18
Object19
Object20
Object21
Object22

Chair

3000

[mm]

2500
2000

Table
1500
1000
500

Chair

Chair

0
-500
-500

0

500

1000

1500

2000

2500

3000

3500

4000

[mm]
Fig. 9. Example of the measured trajectory for a 1.5-year-old child andhandled objects

Interest to object 80.5%

Fig. 11.

Infant’s interest in objects and mother

V. D EVELOPMENT OF AN I NFANT B EHAVIOR S IMULATOR
A. Probabilistic model of infant behavior
This section describes the developed model of infant behavior. The developed model was created by integrating theenvironmental model and the developmental behavior model.
Figure 14 indicates the causal connection that we assumed in
the creation of the model. The model is expressed as a probabilistic model by Eq. (1). The equation expresses the joint

Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

6/12

0.20

Environmental
Facotrs

Probability of interest

0.18

MotherObject0.16

Developmental Behavior
Factors

Distance toobjects

0.14

Behavior t-1

A

0.12

P(Interesting Object |Distance)

0.10
0.08

Age

C

P(Developmental Behavior |Age,Behavior t-1)

Interesting
Object

0.06

B

0.04

P(Induced Behavior |Interesting Object)

0.02
0.00
300

800

1300

1800

2300

2800

3300

3800

4300

Developmental
Behavior

InducedBehavior

4800

Distance from object[mm]Fig. 12. Analysis of the interest induction of an object in relation to theinfant’s distance from the object

D
Behavior t

Fig. 14.



*KIJ
RTQDCDKNKV[

2EMHFW

0RWKHU

>PP@






.QY
RTQDCDKNKV[


















>PP@

Fig. 13.

for infants to predict infant behavior and infant injuries
stemming from infant behavior. Such an application would
allow parameters such as age, kinds of objects and furniture,
and arrangement of the objects and furniture to be changed
in a virtually created environment.
In this paper, based on the concept of affordance [14],which refers to the environmental property that makes aperson do or not do something, we assume that an infant
performs a behavior if he or she has the capability for
performing the behavior induced by the affordance propertyof the environment. To model infant behavior from thisviewpoint, we assume that environmental and developmental
behavior factors are independent of each other. Therefore, Eq.
(1) can be rewritten as follows.
P (Behaviort , V ariablesenv , V ariablesdev ) =P (Behaviort , V ariablesenv ) ×
P (Behaviort , V ariablesdev )Causal connection of infant behavior

Visualization of an interest distribution in a living environment(2)

Below, we explain the details of the environmental factors
and the developmental behavior factors that this paper isfocused on and show a concrete calculation of Eq. (2).
B. Modeling environmental factors

probability of a behavior according to time t (Behaviort ),environmental factors (V ariablesenv )，and developmental
behavior factors (V ariablesdev ).
P (Behaviort , V ariablesenv , V ariablesdev )(1)

A presumable application of the developed model is one
that allows a user who wants to design a safe environment

This paper is concerned with the distance of an infant to an object (Distance) and the object that the
infant is interested in (InterestingObject) as environmental factors (V ariablesenv ). Using Distance andInterestingObject, we created a model for explaining the
behavior that the infant can perform in a certain environment. Specifically, we created a model expressed as
P (Behaviort , Distance, InterestingObject).

Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

7/12

P (Behaviort , Distance, InterestingObject) =P (Behaviort |InterestingObject) ×
P (InterestingObject|Distance)P (Distance)(3)

C. Modeling developmental behavior factors(C in Fig. 14)
Here we model the behavior that the infant whose
age is Age can perform at a certain time t. We focus on the previous behavior of the infant (Behaviort−1 )and the infant’s age (Age) as the developmental factors (V ariablesdev ), and create the model expressedby P (Behaviort , Age, Behaviort−1 ). If we assume that
Behaviort−1 and Age are independent of each other, we
can rewrite P (Behaviort , Age, Behaviort−1 ) as follows.
P (Behaviort , Age, Behaviort−1 )P (Behaviort , Age) × P (Behaviort , Behaviortt−1 )P (Behaviort |Age)P (Age)×
P (Behaviort |Behaviortt−1 )P (Behaviortt−1 )

=
=
(4)

1) Utilization of knowledge of behavior development:
We based constraints stemming from behavior development
according to the DENVER II [12], [13], which was originallyused for evaluating the developmental level of infant behavior.
Figure 15 shows a section of the DENVER II developmental
evaluation sheet. Some examples of rough-and-large behaviorof the entire body by infants indicated in the DENVER II
are ”lie on one’s front”, ”roll over”, ”lie face up”, ”crawl”,”sit”, ”stand”, ”fall”, ”walk”, ”run”, ”jump”, ”pull oneselfup”, ”walk backward”, ”climb up”, ”kick”, ”throw”, ”standon one foot”, and ”pick up”. Some examples of rough-andlarge behavior of the hands are ”place hands together”, ”reachfor an object using the hands”, ”rake with the hand”, ”holdan object in both hands”, ”change hands”, ”pinch with the
thumb”, and ”move only the thumb”. This knowledge isutilized in modeling developmental behavior factors. Below,

we describe the details of each conditional probability in Eq.
(4).
walk

Kinds of Behavior

1) Modeling interest induced by objects (A in Fig. 14):
The interest that the infant has in a certain object canbe explained by various factors such as shape, color, size,
weight, and whether the object is moving or not. In thispaper, as the first step to modeling the interest induced by
objects, we focused on the distance between the infant andthe object since this can be measured by a sensor and is amuch more significant factor when compared with the weight
and the size of the object. Therefore, we can model theinterest induced by objects using the conditional probability
P (InterestingObject|Distance) mentioned in Section ??.
2) Modeling behavior induced by objects (B in Fig. 14):
Based on the concept of affordance, we created the modelfor explaining behavior induced by objects. We assumed thatbehavior is induced only by interesting objects. Hence, we
can model behavior induced by objects using the conditional
probability P (InducedBehavior|InterestingObject).
Using the above assumptions, we can calculate
P (Behaviort , Distance, InterestingObject) as follows.

stand upusing objects
sit formore than 5[s]roll overhold up bodyby both legs
hold up head

2

3

Fig. 15.

4

5 6 7 8 9Age [month-old]

12 15 18

Example from the DENVER II

2) Constraints stemming from behavior development:
When the DENVER II was created, many factors such asage, race, parental income, and geometry were considered.
Age was found to be the most significant factor [12]. Based on
this fact, in the present research, we assumed that the possible
behavior at a certain age Age depends only upon age andcreated the conditional probability P (Behavior|Age) using
the DENVER II.
3) Physical constraints in behavior transition: The behavior at time t is constrained by the previous behavior at times(s < t). For example, a person cannot ”run” directly fromthe ”sit” state without going through the ”stand” state. Here,
we assume that behavior transition has the Markov property,
namely, that future behavior at time t depends only upon
the present behavior at time t − 1. Therefore, the following
equation can be derived,
P (Behaviortt|

Behaviorts , s < t) =P (Behaviortt|Behaviortt−1 ).

(5)

We defined P (Behaviortt |Behaviortt−1 ) using the DENVER II. In our current model, each value of the conditional
probabilities was defined as 0 or 1. An example of the statetransition is shown in Fig. 16. The state transition changes
largely depend on age, as shown in Fig. 17. In this figure,the left-hand side of each of the six figures indicates thestate transition related to rough-and-large behavior of thehands, and the right-hand side of these figures shows the statetransition related to rough-and-large behavior of the entire
body. The figure indicates that the transition becomes complex
as the infant becomes older.
D. Probabilistic integration of environmental factors anddevelopmental behavior factors
Here, we describe the concrete calculation of Eq. (2). The
following are the factors that this paper focuses on.
V ariablesenv
V ariablesdev=={Distance, InterestingObject}{Age, Behaviortt−1 }Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

(6)
(7)

8/12

P (Behaviort |InterestingObject)
P (InterestingObject|Distance)
P (Behaviort |Age)
P (Behaviort |Behaviortt−1 )P (Distance)P (Behaviortt−1 )P (Age)

×
×
×
×
(9)

Using Eq. (9), for example, we can calculate the conditional probability of behavior (Behaviort ) with respect toAge, Behaviort−1 and Distance. Since we assume Age,
Behaviort−1 and Distance are independent of each other,
we can derive the equation of conditional probability as
follows.

Fig. 16.

P (Behaviort , InterestingObject|Distance, Age, Behaviortt−1 )P (Behaviort , Distance, InterestingObject,
Age, Behaviortt−1 )(P (Distance)P (Behaviortt−1 )P (Age))P (Behaviort |InterestingObject)
P (InterestingObject|Distance)
P (Behaviort |Age)
P (Behaviort |Behaviortt−1 )State transition for the case of a 1.5-year-old infant

0 month

=
/
=
×
×
×
(10)

VI. P ERFORMANCE E VALUATION OF D EVELOPEDS IMULATOR

3 months

A. Simulation method

6 months

9 months

12 months

15 months

Fig. 17.
age

Change in the state transition of behavior with respect to infant

To evaluate the effectiveness of the developed infant behavior model, we simulated infant behavior using the developedmodel and compared the output of the simulation with datameasured in the infant observation room mentioned in Section
IV. We assume that the measured data are correct.
The simulation procedure was comprised of the following.
We substituted the positions of an infant, a mother, furniture,and objects into Eq. (10) and calculated the conditional
probability of each behavior at the next moment. For example,
we calculated the conditional probability P (Behaviort =”Stand”, InterestingObject = ”Cup” | Distance =”1024mm”, Age = ”14month − old”, Behaviortt−1 =”W alk”), and then sorted the output behaviors in descendingorder and selected some of the most likely behaviors as the
behavior candidates. In this evaluation, we used the measured
data from two infants aged 14 months and 26 months.
B. Evaluation method

Therefore, we can rewrite Eq. (2) as follows.
P (Behaviort , Distance, InterestingObject,
Age, Behaviortt−1 )P (Behaviort , Distance, InterestingObject) ×
P (Behaviort , Age, Behaviortt−1 )Recall and precision index are widely used for performance
analysis in the field of information retrieval [11] as the indexes
are similar to type 1 and type 2 errors in statistics. Recall andprecision are defined as follows.

=(8)

By substituting Eqs. (3)(4) into Eq. (8), we can derive thefollowing equation.
P (Behaviort , Distance, InterestingObject,
Age, Behaviortt−1 )Recall

≡P recision ≡=The num. of relevant ones of the candidates
The num. of the whole relevant things(11)
The num. of relevant ones of the candidates
The num. of output candidates(12)

Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

9/12

Recall is used for evaluating how well a system can find aselected item, while precision is used for evaluating how wella system can reduce the number of unnecessary items. Bothindexes are a trade-off.
Fundamentally, this paper utilizes recall and precision inorder to evaluate the performance of the developed model.
While the precision index was expanded to correspond with
our purpose, recall was used without modification. Our system outputs behavior with corresponding probability values.
Therefore, we must evaluate not only the correctness of the
behavior output but also the correctness of the probability ofthe behavior output. K. Järvelin and J. Kekäläinen proposed
Discounted Cumulative Gain (DCG) [18] as a new precisionindex which can be used for such a purpose. Based on DCG,
we defined and used ”ranking precision” (RP) as a new
precision index. The use of recall and ranking precision inthis paper are described as follows.
The n-th recall is expressed by

4th condition
Simulation using the integrated model without the behavior transition model• 5th condition
Simulation using the integrated model without the interest induction model• 6th condition
Simulation using the integrated model without the behavior induction model•

C. Results and discussion

1) Results of evaluation: Figures 18 and 19 show theresults of the recall and ranking precision of the simulation
for an infant aged 14 months. Figures 20 and 21 show theresults for an infant aged 26 months. The x-axis of each graphindicates the number of behavior candidates.
Naturally, recall was 100% under the 1st condition, whichindicates that all possible behaviors were output as behavior
candidates. Under the other conditions, recall rose gradually
The num. of relevant behavior
.
(13)
Rn =
and finally reached 100% as the number of candidates inThe num. of the whole relevant behavior
creased. The reason why recall rose steeply at some points
The n-th ranking precision is expressed by
was that our simulator often output the same probabilities
Probability of relevant behavior
among several behaviors; thus, in such cases, we could notRPn =. (14)
increase the number of candidates one by one.
The sum. of probabilities of behavior candidates
Ranking precision was 5% under the 1st condition. Under
The average recall is expressed by
the other conditions, ranking precision decreased graduallyM
as the number of candidates increased. However, ranking
1 
R̄ =
Rn ,
(15)
precision under the 2nd to 5th conditions was 1.7 to 3.7
M n=1
times as high as that under the 1st condition. This means
where M denotes the number of evaluations.
that each of the developed models contributed to explaining
The average ranking precision is expressed byinfant behavior. In the case of the infant aged 14 months,
the developmental behavior model was found to be the mostM
1 
influential. In the case of the infant aged 26 months, after
RP =
Pn .
(16)
M n=1
averaging ranking precision, the behavior transition model
was found to be the most influential.
Additionally, in this paper, we used F-value [11] for comFigure 22 compares F-values among the simulations underprehensive evaluation. F-value is defined as follows.
different conditions. The results of this comparison show that1
the F-value of the integrated model (the 2nd condition) was
F =
,
1
1
more than two times as high as that of the random model (the+ (1 − α)αRPR
1st condition).
where α is a weighting value. We used 0.5 as α, which means
VII. P ERSPECTIVE OF THE I NFANT B EHAVIOR
that we consider the importance of both index R and index
S IMULATION
RP to be the same.
This section describes the perspective of the proposed
Furthermore, we compared the performance among simuinfant behavior simulation. The relatively short-term goals oflations under the following conditions.
the simulator are the following.
• 1st condition• To be helpful in educating parents for the purpose ofSimulation in which all behaviors are output with thepreventing accidents
same probability
Knowledge applicable to individual homes can be ob• 2nd condition
tained in order to help prevent injury accidents. Effective
Simulation using the integrated model, which consists
education is made possible by combining the simulationof the developmental behavior model, the behavior tranwith realistic graphics.
sition model, interest induction model, and behavior• To be helpful in designing an environment that is safe
induction model
for infants• 3rd condtion
In designing ordinary homes, nursing homes, daycare
Simulation using the integrated model without the decenters for children, and kindergartens, for example, thevelopmental behavior model
Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

10/12

100

100

98

Recall (%)

Recall (%)

95

90

85

RandomWhole Model
Except Developental Behavior Model
Except Behavior Transition Model
Except Interest Induction Model
Except Behavior Induction Model

80

75

1

2

3

4

5

6

7

8

9

96
94
92
RandomWhole Model
Except Developental Behavior Model
Except Behavior Transition Model
Except Interest Induction Model
Except Behavior Induction Model

90
88
86

10 11 12 13 14 15 16 17 18 19 20

1

2

3

4

5

6

The number of behavior candidates

Fig. 18.

Evaluation of recall in the case of an infant aged 14 months

Fig. 20.

188

9

10 11 12 13 14 15 16 17 18 19 20

Evaluation of recall in the case of an infant aged 26 months

18

RandomWhole Model
Except Developental Behavior Model
Except Behavior Transition Model
Except Interest Induction Model
Except Behavior Induction Model

20

7

The number of behavior candidates

RandomWhole Model
Except Developental Behavior Model
Except Behavior Transition Model
Except Interest Induction Model
Except Behavior Induction Model

16

14

12

14

Precision (%)

Precision (%)

16

12
10
8

10

8

6

6
4

4

2

2
0
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

0

20

1

2

3

Evaluation of precision in the case of an infant aged 14 months

safety of such environments can be evaluated in advance.
The proposed simulator will also make it possible toimprove existing homes in order to reduce the risk of
injury accidents involving infants.
To realize the above two applications, we are developing afunction for visualizing infant behavior and infant accidents
using game technology [19]. Examples of visualized results
are shown in Fig. 23. Visualization is realized using VirtoolsTM[20], which is originally a rapid prototyping software forcreating video games that can run on MicrosoftTM Internet
Explorer. VirtoolsTM features are as follows: 1) It is easy to
add new functions using the C++ language. 2) It is possible to
distribute the developed software with no additional royalties.
3) A free downloadable player is available as a plug-in forMicrosoftTM Internet Explorer. These features will be very
useful for efficiently disseminating the simulator as digitalcontent on the Web allowing the public to evaluate theeffectiveness of the developed simulator for parents.
The longer-term goals for the simulator are the following.

Fig. 21.

•

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Evaluation of precision in the case of an infant aged 26 months

Real-time monitoring of infants for the purpose of preventing accidents
Real-time monitoring of infants in order to prevent
accidents will be possible by integrating the proposed
simulator into a sensing system using the recently developed inexpensive sensors, sensor networking technology,
RandomWhole Model
Except Developental Behavior Model

F-value

Fig. 19.

4

The number of behavior candidates

The number of behavior candidates

0.300

Except Behavior Transition Model

0.250

Except Behavior Induction Model

Except Interest Induction Model

0.200
0.150
0.100
0.050
0.000

Fig. 22.

14 month-old

26 month-old

Evaluation of F-values among different models

Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

11/12

Developmental
Behavior Model

Behavior Visualization

ACKNOWLEDGMENTS
The authors wish to thank Drs. Erin Cassell and Karen
Ashby at the Monash University Accident Research Center(MUARC) in Australia for providing injury accident statistical
data. In addition, the authors would like to thank Dr. Jun
Kohyama at Tokyo Kita Shakai Hoken Hospital in Japanfor providing valuable information on infant physiology. This
study was funded in part by a Grant-in-Aid from the Secom
Science and Technology Foundation and by a Sasakawa
Scientific Research Grant from the Japan Science Society(No. 16-322).

8 month-old

14 month-old

R EFERENCES
24 month-old

Fig. 23.

•

Example of visualization of an infant-behavior-related accident

and ubiquitous computing technology.
Development of infant informatics
The proposed simulator contributes to the developmentof a new research area, which can be referred to as
infant informatics or child informatics. Infant informatics
attempts to comprehensively understand infant behavior
by considering infant behavior from an informatics perspective.
VIII. C ONCLUSIONSThe present paper described the necessity of a computational theory of infant behavior for scientific study of infantrelated injury accidents. In addition, this research highlighted
the significance of the computational approach as a method
for addressing the problem of child safety in the home.
As one possible approach, the authors proposed an infant
behavior simulation system that enables the simulation of
infant behavior in a virtual environment. To create a computational model of infant behavior for the simulator, theauthors classified the determinant factors of infant behaviors
as either internal or external factors. Internal factors weremodeled using a developmental behavior model. This model
describes behaviors displayed by an infant in relation to
the infant’s age. Behavioral capabilities are modeled using
knowledge of the developmental behavior of infants. External
factors were modeled using an environmental model that
included the objects around the infant, as well as his or her
position. The environmental model contains information on
the behavior induced by the objects surrounding the infant.
The infant behavior induced by the objects was selected using
the data collected in an experimental behavior observation
room developed by the authors. The performance of thedeveloped simulator was evaluated by comparing the outputof the simulator and the data measured in the infant observation room. Finally, the perspective of the infant behavior
simulation was described.

[1] A.L. Wallis, B.E. Cody, A.D. Mickalide, ”Report to the Nation: Trends in
Unintentional Childhood Injury Mortality, 1987-2000,” National SAFEKIDS Campaign, 2003 (http://www.safekids.org/content documents/
nskw03 report.pdf)[2] Victorian Injury Surveillance and Applied Research System
http://www.monash.edu.au/muarc/VISAR/[3] E.E. Birch, J. Gwiazda, R. Held, ”Stereoacuity Development for Crossed
and Uncrossed Disparities in Human Infants,” Vision Research, Vol. 22,
pp. 507-513, 1982
[4] R. Fox, R.N. Aslin, S.L. Shea, S.T. Dumais, ”Stereopsis in Human
Infants,” Science, Vol. 207, pp. 323-324, 1980
[5] A. Yonus, C.E. Granrud, ”Infants Distance Perception from Linear
Perspective and Texture Gradients,” Infant Behavior & Development,
Vol. 9, pp. 247-256, 1986
[6] T. Wilcox, ”Object Individuation: Infants’ Use of Shape, Size, Pattern,
and Color,” Cognition, Vol. 72, pp. 125-166, 1999[7] J. Kaufman, D. Mareschal, M.H. Johnson, ”Graspability and Object
Processing in Infants,” Infant Behavior & Development, Vol. 26, pp.
516-528, 2003
[8] D. Marr, Vision, W. H. Freeman and Company, 1982
[9] U. Hansmann, L. Merk, M.S. Nicklous, T. Stober, Pervasive Computing
Handbook. The Mobile World, Springer-Verlag Telos, 2001[10] J. Fraden, Handbook of Modern Sensors: Physics, Designs, and Applications, AIP Press, 2003[11] R. Baeza-Yates, Modern Information Retrieval, Addison Wesley, 1999[12] W.K. Frankenburg, J. Dodds, P. Archer, et al., The DENVER II Training
Manual, Denver, CO: Denver Developmental Materials, Inc., 1992
[13] The Japanese Society of Child Health, Japanese Edition DENVER II,
The Japanese Child Medical Press, 2003[14] J. J. Gibson, The Senses Considered as Perceptual Systems, Greenwood
Press, 1966[15] Y. Nishida, H. Aizawa, T. Hori, N.H. Hoffman, T. Kanade, M.
Kakikura, ”3D Ultrasonic Tagging System for Observing Human Activity, ” Proceedings of IEEE International Conference on Intelligent
Robots and Systems (IROS2003), pp.785-791, October 2003[16] Y. Nishida, K. Kitamura, T. Hori, A. Nishitani, T. Kanade, H. Mizoguchi, ”Quick Realization of Function for Detecting Human Activity
Events by Ultrasonic 3D Tag and Stereo Vision,” Proc. of 2nd IEEE
International Conference on Pervasive Computing and Communications(PerCom 2004), pp. 43-54, 2004[17] P. Martin, P. Bateson, Measuring Behavior, Cambridge University
Press, 1990
[18] K. Järvelin, J. Kekäläinen, ”IR evaluation methods for retrieving highly
relevant documents,” Proceedings of the 23rd Annual International
ACM SIGIR Conference on Research and Development in Information
Retrieval, pp.41-48, New York, 2000．
[19] A. Simo, K. Kitamura, Y. Nishida, ”Children Accidents’ Simulation:
Planning the Emergent Situations,” Proc. of 4th International Conferenceon Interaction Design and Children,” June 2005 (in press)[20] http://www.virtools.com/

Proc. of the 4th IARP/IEEE-RAS/EURON Workshop on Technical Challenges for Dependable Robots in Human
Environments, T16-01, 2005

12/12