(c) 2012 IEEE. Personal use of this material is permitted. However,
permission to reprint/republish this material for advertising or
promotional purposes or for creating new collective works for resale orfor redistribution to serves or lists, or to reuse any copyrighted
component of this work in other works must be obtained from the IEEE.

2012 IEEE RO-MAN: The 21st IEEE International Symposium on
Robot and Human Interactive Communication.
September 9-13, 2012. Paris, France.

Extrapolation Simulation for Estimating Human Avoidability
in Human-Robot Coexistence Systems
Koji Sunada, Yoji Yamada, Takamasa Hattori, Shogo Okamoto, and Susumu Hara
towards the subject’s face. Again, however, the effects ofhuman actions to avoid harm were not studied.
The second author of this paper previously proposed anddeveloped a next-generation cell production system, in which
a human and a robot work cooperatively [7], [8]. There isa strong desire to remove the light curtains from production
sites because they cause a reduction in available space thattends to decrease productivity. The light curtains can beremoved if the coexistent robot is basically designed as being
inherently safe, yet there is always the fundamental problem
that a power-limited robot grasping a sharp-edged object
could cause more harm to a human eye than to another partof the body (e.g., a shoulder).
We conducted psychological experiments to observe human avoidance actions in a particular situation where human eyes were under threat in a human-robot coexistencesystem [9].1 In our experiments, the robot’s motions were
controlled so as not to touch the participants to ensure theirsafety.
Using the data from the psychological experiment, we
propose to simulate collisions between the human eye androbot’s end effectors, taking human avoidance actions into
account. We termed this an “extrapolation simulation.” Human avoidance actions have not been taken into considerationin previous robot safety simulations. Collision probability
involving avoidability can be estimated by carrying out extrapolation simulations under various conditions. This paper
presents a method to estimate safety conditions under our
experimental setup. We can identify the difference between
safety conditions when avoidance actions are taken and when
they are not. This study contributes to the development ofa tool for conducting practical risk assessments in order torealize a safe human-robot coexistence system.

Abstract— In the design of human-robot coexistence environments, human avoidance actions are not taken into consideration. We conduct a psychological experiment to observe
human avoidance actions where a mechanical harm approached
his or her eyes. Using the observed motions, we carry out“extrapolation simulation” of human avoidance actions in collision situations. As simulation results, first, we express collision
probability using the motion range of the end effector of arobot as a variable. Second, we identify the motion conditions,which ensure the safety, of the end effector in our setup. Finally,
we quantitatively compare the safety conditions between taking
and not taking avoidance actions into consideration. This study
contributes to providing a tool for conducting a practical risk
assessment aimed at realizing a safe human-robot coexistencesystem.

I. I NTRODUCTION
Humans, by nature, take harm-avoidance actions when
they perceive a threat from an approaching object. Hence, thepossibility of avoiding or limiting harm (hereafter referred toas “avoidability”), which is one element of risk [1], should
be taken into consideration when designing human-robot coexistence systems. Nonetheless, avoidability currently tends
to be either neglected or discretely estimated in two or
three levels by assessor’s intuition because human avoidance
action is an unexplored human factor.
Previous research into the safety of human-robot interactions has focused mainly on the end result of a harmfulinteraction. Oberer and Schraft [2] investigated injury indicesusing finite element models to simulate the collision of arobot with the head, chest, and pelvis of a dummy, andHaddadin et al. [3] followed this up with head and chest
collision experiments. Park et al. [4] proposed a model forthe collision between a human head and a robot, and showedthat the optimal elastic modulus and thickness of the robot’s
covering could be determined so as to prevent skin injuries.
However, harm-avoidance action, which humans are most
likely to engage in, was not taken into consideration in thesestudies.
Human reactions have been considered in some studies.
Ikeura et al. [5] measured the galvanic skin reflex of a subject
when a robot was projected straight toward the subject’s face
to investigate which robot motions the human regarded as
threatening. In order to identify robot motion conditions that
aroused human fear, Yamada et al. [6] measured the pupillary
diameter of a subject when a robot end-tip accelerated

II. P SYCHOLOGICAL E XPERIMENT
A. Experimental Design
We consider a general situation in which a gripper oran object grasped by a next-generation robot, which should
coexist with humans, becomes a hazard. In reference to apractical study [7], we assume a situation where a sharp endeffector tip of a production site robot is suddenly projected
towards the eyes of a worker sitting opposite.
In the assumed situation, the worker naturally takes someavoidance actions based on visual and auditory information.
Nevertheless, workers may not necessarily hear the motors

This work was supported by KAKENHI 21246041.
K. Sunada, Y. Yamada, T. Hattori, S. Okamoto, and S.
Hara are with the Graduate School of Engineering, Nagoya
University,
Furo-cho,Chikusa-ku,
Nagoya464-8603,
Japan.

1 The present studies were approved by the local ethical committee.

http://www.mech.nagoya-u.ac.jp/asi/en/
978-1-4673-4605-4/12/$31.00 ©2012 IEEE

785

330

installed in the robots while in an actual working environment. For this reason, we focus on the more hazardous
situation in which visual information alone is provided to
the worker.
An upper-body humanoid robot (HIRO, Kawada Industries, Inc.), designed to operate collaboratively with a human [7], [8], was used in the experiments. The participant
wore protective glasses and was separated from the robot by
a working table. To minimize any hazard to the participant,the original end effectors for picking up and placing mechanical parts were replaced with pyramid-shaped flexible
polyurethane foams.

50

170
Bearing

y

100

140

180

250

x
O
Near
Middle

230

Far
End effector
Participant

Top view

z

280

155

200

1) Participants: Nine people, five males and four females
between the ages of 18 and 28, participated in the experiment. Every participant was healthy with good eyesight; noone reported suffering from belonephobia.2
2) Experimental Setup: Each participant sat on a stool
in front of the robot. The participants wore noise-cancelingearphones (NW-A845, Sony Corp.) to block any external auditory information; they instead listened to sounds
recorded in a factory. They were exposed to the working
area of the robot and performed the task of inserting two
mechanical parts, rollers and retainers, between bearing rings
using tweezers. While they were performing the task, one
end effector at the tip of the robot arms would suddenly
approach their eyes. The end-effector tip arrived at a point
approximately 50 mm forward of the participant’s initial eye
position at the end of the robot motion. It was because we
ensured the participant’s safety. The contact between the end
effector and the participant might cause some damage to theparticipant.
The probability distribution of the foreperiod (i.e., theperiod of the task before the robot arm is projected towards
the participant) is often modeled using an exponential distribution [10]. In this manner, statistically random foreperiods
were taken as the sum of 10 s and exponential random values
with a mean of 15 s, excluding those longer than 60 s.
Fig. 1 shows a schematic diagram of the relevant locations
and distances in the experimental setup. The location of the
bearing rings is indicated by the cross. Three initial positions(far, middle, and near) were set for the end effectors. The
initial distance between the participant’s eyes and an endeffector tip was set to be approximately 470, 370, and
270 mm for far, middle, and near patterns, respectively. For
all patterns, each participant was asked to confirm that theend-effector tips, located at a viewing angle of approximately
30◦ , were in their peripheral vision when the task position
was in the center of the visual field.
Fig. 2 shows the speed patterns of the end-effector tip.The
maximum speed and initial acceleration of the tip were based

110

B. Experimental Conditions

x

O

Side viewFig. 1. Schematic diagram of the human-robot interactive locations in
psychological experiment (unit: mm)

Speed [mm/s]

1500
1200
900

Far
Middle

600

Near

300
0
0

0.2

0.4

0.6

0.8

Time [s]Fig. 2. Speed patterns (far, middle, near) of the end effector in psychological experimenton the runaway assumption.3
C. Experimental Procedure
Each participant performed 60 trials. Either the right orleft end effector approaching humans and its initial position
were randomly chosen for each trial.
3 We set the robot to output the same highest possible speed percentage
for each pattern. The maximum speed and initial acceleration inevitably
decreased in the order pattern far, middle, near due to differences in thetravel distance.

2 For safety reasons, initial experiments had to be conducted with young
people in good health who seemed to have reasonably good reactions.

786

End effectors

Fig. 3.

the psychological experimental results which suggested thata shorter initial distance between the eyes and the endeffector tip is associated with a shorter avoidance reaction
time. The initial distance between the eyes and the end
effector significantly influence the avoidance reaction time.
Therefore, the initial distance should be remained as it wasin the psychological experiment.
The second rule is not to modify the end-effector motion
during the short time after the end effector started approaching. The end effectors were in participant’s peripheral vision
before one of them started moving. We considered that theparticipants reacted to the end effector approaching when it
first started moving towards them. Hence, it is consideredthat the end-effector motion immediately after it started
moving particularly related to the human avoidance actions.
Therefore, the end-effector motion during the short timeafter it started moving should be remained as it was in thepsychological experiment.
On the basis of these rules, we have to modify the endeffector motion a short time after it starts moving andsimulate a collision between the eye and the end effector.
When the end effector approached closer to the participants,
they might take some additional avoidance actions. These
actions may possibly increase the avoidability. However,
we could not observe such actions in the psychologicalexperiment owing to our setup to ensure participant’s safety
as was described in II-B.2. Under the current experimental
condition, it is not possible to take into account the additional
human avoidance actions. We estimate the avoidability basedon the observed data in the psychological experiment, while
the additional avoidance actions are not taken into account.

Human

Observed marker positions with motion capture system

D. Observed Data
The movements of the participants, as well as those of
the robot, were captured by motion capture cameras. Eachparticipant wore a cap and a suit with motion capture markers
attached. Fig. 3 shows the observed data of the markers.
Four markers were attached to the participant’s head and
15 markers were attached to the upper body. Three markers
were attached to each end effector, and the positions of theend-effector tip were estimated.
III. E XTRAPOLATION S IMULATION
A. Method
In the extrapolation simulations, we made an end effector
collide with the eye of a human model on a computer,while the human model performed according to the motion
observed in the psychological experiment. Fig. 4 shows threepictures of the extrapolation simulation. In the left picture,
the end effectors are still not moving, and the human model
was performing the task. In the middle picture, the end
effector is approaching the human model, and the humanmodel is avoiding from it. In the right picture, the end
effector collides with the eye of the human model.
The system of extrapolation simulations was constructedby the motion capture system and a human modeling software (Jack, Siemens AG). The motion capture system andthe human modeling software shared marker-position data,
and the human-motion data was generated from the series ofmarker-position data from the participant’s body. The humanmodel was created tailoring to each participant’s physicalsize.

C. Simulation Condition of End-effector Motion
We set the end-effector motion in the extrapolation simulation according to the above simulation rules. The end
effector starts moving and accelerates with the same timing
and motion as in the psychological experiment. After theend effector reaches the pre-set maximum approach speed,
the end-effector motion is modified from the experimental
motion to a uniform linear motion. The end effector maintains the maximum approach speed and the direction until itcollides with a human eye. The maximum approach speed
is assumed to be the limiting speed of an end effector in
actual production site robots. The linear trajectory of the end
effector is adjusted to collide with the human model’s eye
at near-side eye. That is, a right end effector collides with aleft eye and a left end effector collides with a right eye.
We set five levels for the maximum approach speed foreach initial-position condition, based on our experimental
setup. Fig. 5 shows the speed for velocity levels I–V under
the far condition. In velocity level I, the maximum approach
speed is set to the approximate maximum speeds observed in
the psychological experiment, which were 1300 mm/s under
the far condition, 1200 mm/s under the middle condition,and 1000 mm/s under the near condition. The maximum
approach speeds for velocity levels II, III, IV, and V were

B. Simulation Rules
It has not been discussed how a collision involves human
avoidance action in the pre-impact phase [2], [3]. In theextrapolation simulations, we used the motion data observedin the psychological experiment as the human model’s motion, while we modified the end-effector motion to be morehazardous. We established two rules for the modificationof the end-effector motion to ensure that the extrapolation
simulations were reasonable.
The first rule is not to modify the initial relative location of
the end effector to the participants. Hattori et al. [9] reported
787

Human modelRobot

End effectors

Collision point

Initial position

Time
Fig. 4.

A series of screen images in extrapolation simulation

motion range of the end effector is longer than the collision
distance, the end effector may collide with a human eye.
The collision distance becomes an important criterion for
estimating the safe motion range.
2) Collision Probability Curve: In the field of mechanicalsafety, risk analysis often involves the construction of a risk
curve. The risk curve indicates the relationship between thefrequency (probability) and the effect (severity) of a risk. We
would like to draw a risk curve using our simulation results,
but we cannot estimate the severity of harm in this study. Asa preliminary stage for drawing of the risk curve, we present
the collision probability under the variable conditions of theend-effector motion.
An end effector collides with a human eye when thecollision distance is shorter than the motion range of the end
effector. Hence, once the motion range has been defined, alltrials can be divided into collision trials and non-collision
trials. We expressed the collision probability as the rate ofcollision trials in each simulation pattern, using the motion
range as a variable.
Fig. 6 shows the collision probability curve for three
initial-position conditions when the maximum approach
speed was set at 1000 mm/s. For example, when the initial
distance between the eyes and the end effector is middle,
approximately 370 mm apart, the collision probability isshown by yellow squares. The probability is zero when the
motion ranges of the end effector is smaller than 400 mm.
The probability then increases with the motion range. It
reaches 1.0 when the motion range is 515 mm.
As these curves show, our simulation method is a practical
tool to provide the collision probability curves involving
human avoidance actions. However, we should note that in
the simulations, the end effector goes directly towards thehuman model and collides with the eye; but this scenario
is rare in real situations. Therefore, our collision probability
tends to be higher than the real probability of an end effector
colliding with a human eye.
3) Minimum Collision Distance as Safe Motion Range:
We estimate the safe motion range of the end effector fromthe collision probability curves. Fig. 7 shows the conceptual
diagram of a collision probability curve and a safe motion

Speed [mm/s]

1500
1200

Velocity level I
Velocity level II

900

Velocity level III
600

Velocity level IV
Velocity level V

300
Experimental data
0
0

0.2

0.4

0.6

0.8

Time [s]Fig. 5.

Velocity levels I–V under far condition

TABLE IM AXIMUM A PPROACH S PEED IN E XTRAPOLATION S IMULATION [mm/s]

Far
Middle
Near

I
1300
1200
1000

Velocity level
II
III
IV
1200 1100
1000
1100 1000
900
900
800
700

V
900
800
600

each set 100 mm/s slower than in the previous velocity level(see Table I).
D. Simulation Procedure
For the simulations, we picked out 60 trials from one participant, who did not behave peculiarly in the psychologicalexperiment. This gave a total of 300 trials, owing to the fivesimulation patterns.
E. Simulation Results
1) Definition of Collision Distance: We measured the
distance of the end-effector tip from its initial position until
collision for each trial, as shown in the right picture of
Fig. 4. We termed the distance as “collision distance.” Weregarded the movable distance of the end effector from its
initial position as the motion range of the end effector. If the788

Minimum Collision Distance[mm]

Collision probability

1
0.75
Far

0.5

Middle
Near

0.25
0
270

370

470

570

570

470
Far
Middle
370

Near

270

670

500

800

1100

1400

Motion range [mm]Maximum approach speed [mm/s]

Fig. 6. Collision probability curve for three conditions (far, middle, near)with maximum approach speed set to 1000 mm/s

Fig. 8. Relationship between minimum collision distance and maximum
approach speed for three conditions (far, middle, near)

Safe

Unsafe25

Difference of minimum
collision distance [mm]Collision probability

Avoidable Uncertain Unavoidable

1

20
15

Far
Middle

10

Near
5
0
500

0
Motion range
Fig. 7.

800

1100

1400

Maximum approach speed [mm/s]
Fig. 9. Effect of avoidance actions: Differences of the minimum collision
distances on taking and not taking human avoidance into consideration

Safety condition according to collision probability curve

range. It is sure that the participant is able to avoid collisionin the motion range where the collision probability is zero.
Such a motion range is regarded as a safe motion range.
On the other hand, in the motion range where the collision
probability is above zero, it is unclear whether the participant
is able to avoid collision. Therefore, a minimum collision
distance is a critical value between the safe and unsafe
motion range. We regard the minimum collision distance as
the maximum safe motion range.
Fig. 8 shows the minimum collision distances. They depend on the maximum approach speed of the end effector andthe initial distances between the eyes and the end effector.
In general, the minimum collision distances decrease as
the maximum approach speed of the end effector increases.
This is simply because it becomes more difficult to avoid
a more quickly approaching harm for humans. We are able
to estimate the safety conditions according to Fig. 8. For
example, when the initial distance between the eyes andthe end effector is middle, approximately 370 mm apart,and the maximum approach speed is 800 mm/s, the motion
range of the end effector should be limited shorter than theminimum collision distance of 415 mm. We should notethat if human avoidance actions are not concerned thenthe minimum collision distances are flat unlike those in
Fig. 8. Hence, the extrapolation simulations provided us withestimated parameters important for risk analysis.

4) Effects of Human Avoidance Actions: In order to quantify the effects of human avoidance actions on the minimum
collision distances that is the safe motion range of the end
effector, we calculated their differences on taking and nottaking avoidance actions into consideration. If a human does
not take avoidance actions, minimum collision distances areequal to the initial distances between the human and the end
effector.
Fig. 9 shows the differences of the minimum collision
distances between taking and not taking human avoidance
actions into consideration. Again, in general, the effects ofhuman avoidance actions vary by the maximum approach
speed of the end effector and the initial-position conditions.
The effects are most significant when the maximum approach
speed is slow. For all of the three initial positions, the effects
reach approximately 15 mm when the maximum approach
speed is slowest. This means that the participants could avoid
the harm by approximately 15 mm in the setup. The effectsof avoidance actions tend to decrease as the end effector
becomes quicker and are least significant when the end
effector is quick; they are around 5 mm or less.
These data quantitatively indicate the effects of human
avoidance actions. These effects for avoidability have neverbeen considered thus far in risk assessment.
789

when human actions were involved. This paper contributes toour ability to take human avoidance actions into reasonableconsideration when designing industrial environments.
We will focus on situations in which human avoidance
actions affect most efficient. We consider a situation in whicha robot attacks from side of a human. Because the trajectoryat which the human avoids collision would be different tothat at which the end effector approaches. We plan to conduct
a new psychological experiment.

IV. D ISCUSSION
A. Effects of Human Avoidance Actions
We quantified the effects of human avoidance actions on
the extension of the safe motion range in our setup. These
effects were at most 20 mm. Some may consider that they
are not important in risk assessment. However, we consider
that they affect significantly to the mitigation of the severityof harm as well as the extension of safe motion range. We
assumed harm to an eye of which the severity becomes
particularly high. Ito et al. [11] reported the experimental
results which suggested that an eyelid contributed to themitigation of the severity of harm to the eye from a mechanical hazard approaching. In the psychological experiment, weobserved that some participants closed their eyelid when an
end effector approached to the eye. In Fig. 9, for example, at
600 mm/s under near condition, the difference of minimum
collision distances was around 15 mm. The 15 mm extended
distance to collision makes the 25 ms extension of time tocollision. The experimental results by Evinger et al. [12]suggested that the time from applying stimulus to closing
eyelid was around 90 ms. The extended time by considerationof the human avoidance actions corresponds to time between
one-fourth to one-third of time required for closing eyelid.
The human avoidance actions could make closing eyelid
in time and mitigate the severity of harm. Therefore, we
consider that the quantified effects of avoidance actions areimportant in practical risk assessment.

VI. ACKNOWLEDGMENTS
We are indebted to Professor Shuji Mori of Kyushu University and Professor Kazunori Hase of Tokyo Metropolitan
University for their insightful advice. We are also grateful tograduate student Soichiro Ito at Nagoya University for hiscooperation.
R EFERENCES[1] Safety of Machinery — General Principles for Design — Risk Assessment and Risk Reduction, ISO 12100:2010, 2010.
[2] S. Oberer and R. D. Schraft, “Robot-dummy crash tests for robot
safety assessment,” in Proc. 2007 IEEE International Conference on
Robotics and Automation, Roma, Italy, Apr. 2007, pp. 2934–2939.
[3] S. Haddadin, A. Albu-Schäffer, M. Frommberger, J. Rossmann, and
G. Hirzinger, “The “DLR crash report”: Towards a standard crashtesting protocol for robot safety - Part I: Results,” in Proc. 2009 IEEE
International Conference on Robotics and Automation, Kobe, Japan,
May 2009, pp. 272–279.
[4] J.-J. Park, S. Haddadin, J.-B. Song, and A. Albu-Schäffer, “Designing
optimally safe robot surface properties for minimizing the stresscharacteristics of human-robot collisions,” in Proc. 2011 IEEE International Conference on Robotics and Automation, Shanghai, China,
May 2011, pp. 5413–5420.
[5] R. Ikeura, H. Otsuka, and H. Inooka, “Study on emotional evaluationof robot motions based on galvanic skin reflex,” The Japanese Journalof Ergonomics, vol. 31, no. 5, pp. 355–358, Oct. 1995, (in Japanese).
[6] Y. Yamada, Y. Umetani, and Y. Hirasawa, “Proposal of a psychophysiological experiment system applying the reaction of humanpupillary dilation to frightening robot motions,” in Proc. 1999 IEEE
International Conference on Systems, Man, and Cybernetics, vol. 2,
Tokyo, Japan, Oct. 1999, pp. 1052–1057.
[7] New Energy and Industrial Technology Development Organization,“Project for strategic development of advanced robotics elemental
technologies, Human-robot cooperative cell production and assembly
systems (Field of next-generation industrial robots), Safe upper-bodyhumanoid equipped with compact handling systems, Accomplishment
Report in Fiscal 2006–2008,” May 2009, (in Japanese).
[8] T. Ogure, Y. Nakabo, S. Jeong, and Y. Yamada, “Hazard analysis of anindustrial upper-body humanoid,” Industrial Robot: An International
Journal, vol. 36, no. 5, pp. 469–476, Sep. 2009.
[9] T. Hattori, Y. Yamada, S. Mori, S. Okamoto, and S. Hara, “Psychological experiments on avoidance action characteristics for estimating
avoidability of harm to eyes from robots,” in Proc. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura,
Portugal, Oct. 2012, (accepted).
[10] R. D. Luce, Response Times: Their Role in Inferring Elementary
Mental Organization. New York, NY: Oxford University Press, 1986.
[11] S. Ito, Y. Yamada, T. Hattori, S. Okamoto, and S. Hara, “Investigation
into the severity of eyes attacked by a sharp object in collision experiments taking human avoidance/mitigation actions into consideration,”in Proc. 29th Annual Conference of the RSJ, Tokyo, Japan, Sep. 2011,
pp. 1N1-7, (in Japanese).
[12] C. Evinger, K. A. Manning, and P. A. Sibony, “Eyelid movements
mechanisms and normal data,” Investigative Ophthalmology & Visual
Science, vol. 32, no. 2, pp. 387–399, Feb. 1991.

B. Remark about Extrapolation Simulation
We proposed the extrapolation simulation while we did
not consider the additional avoidance actions which could
have been caused by closer approaches of the end effector.
We consider that it is more accountable to estimate theavoidability lower in the practical risk assessment, whichis reflected by choosing subjects who likely to take slower
avoidance actions for establishing the standard data.
V. C ONCLUSIONS AND F UTURE W ORKThis study aimed to take human avoidance actions into
consideration when designing a human-robot coexistence environment. We have conducted a psychological experiment toobtain data on human avoidance actions. Using the obtained
data, we carried out extrapolation simulations of human
avoidance actions in collision situations. As the simulation
results, first, we expressed the collision probability as therate of collision trials in each simulation pattern, using the
motion range as a variable. Second, we estimated some safety
conditions for the end-effector motions in our simulation
setup. Finally, we quantified the effects of the avoidance
actions as the difference between taking and not taking
avoidance actions. The motion ranges of the end effector, that
ensures safety, increased by approximately 20 mm at most790