The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

Noname manuscript No.
(will be inserted by the editor)

Field Trial of a Networked Robot at a Train Station
Masahiro Shiomi, Daisuke Sakamoto, Takayuki Kanda, Carlos Toshinori
Ishi, Hiroshi Ishiguro, and Norihiro Hagita,

Received: date / Accepted: date

Abstract We developed a networked robot system inwhich ubiquitous sensors support robot sensing and ahuman operator processes the robot’s decisions during interaction. To achieve semi-autonomous operation
for a communication robot functioning in real environments, we developed an operator-requesting mechanismthat enables the robot to detect situations that it cannot handle autonomously. Therefore, a human operator helps by assuming control with minimum eﬀort.
The robot system consists of a humanoid robot, ﬂoor
sensors, cameras, and a sound-level meter. For helping people in real environments, we implemented suchbasic communicative behaviors as greetings and route
guidance in the robot and conducted a ﬁeld trial ata train station to investigate the robot system’s eﬀectiveness. The results attest to the high acceptability of
the robot system in a public space and also show thatthe operator-requesting mechanism correctly requested
help in 84.7% of the necessary situations; the operator only had to control 25% of the experiment time inthe semi-autonomous mode with a robot system thatsuccessfully guided 68% of the visitors.
Keywords Networked robot, Semi-autonomous
communication robot, Field trial

Masahiro Shiomi, Takayuki Kanda, Carlos Toshinori Ishi, Hiroshi
Ishiguro, and Norihiro Hagita
ATR-IRC, Kyoto, 619-0288, Japan
Tel.: +81-774-95-1432
Fax: +81-774-95-1408
E-mail: m-shiomi@atr.jp
Daisuke Sakamoto
Japan Science and Technology Agency, ERATO Igarashi Design UI Project, 1-28-1-7F Koishikawa, Bunkyo, Tokyo, 112-0002,
Japan

1 Introduction
One of our goals is to develop a communication robotthat is capable of natural human-robot interaction andcan support human activities in real environments. For
example, in the future, a communication robot at a
train station might provide information about departure platforms, transfers, and nearby shops by eﬀectively using both verbal and nonverbal expressions (Fig.
1). Since the target audience of a communication robot
is the general public (people without specialized computing and engineering knowledge), a conversational interface that uses both verbal and nonverbal expressions
is important. Past studies in robotics have emphasized
the merits of robot embodiments that show the eﬀectiveness of facial expressions [1], eye-gaze [2], and gestures [3].
However, it remains diﬃcult to achieve robust verbal communication with communication robots. One
major diﬃculty is the speech recognition of colloquialutterances in noisy environments. Such disturbances
increase the dependence on the distance between the
robot’s microphone and the speaker. Recent speech recognition technology can only recognize formal utterancesin noiseless environments. Although research continues
in robot audition [4], the diﬃculties in real environments remain beyond the grasp of current technology.
Therefore, robots continue to have diﬃculty handling
natural language conversations as deftly as humans.
Another basic diﬃculty exists in the development
process for communication robots in real ﬁelds. They
must be placed in real situations; otherwise, we cannot reproduce similar situations in laboratories or elsewhere. Human behavior in real environments is toocomplicated to predict and becomes even more complex
with a large number of various people, such as children,

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

2adults, and senior citizens, or if the environment is more
intricate.
We must improve the sensing abilities of robots insuch real environments so that they can work robustly.
Some researchers are studying stand-alone robots thathave complete sensing, decision making, and acting capabilities. On the other hand, if current technologies
are used, such approaches have limitations to increase
the abilities of robots in real environments.
To solve these problems, we have chosen a strategy known as a “network robot system”[5] that combines robots, ubiquitous sensors, and humans. In thisstrategy, a human operator supports the decisions of arobot during interaction. In other words, a robot behaves semi-autonomously [6]. Semi-autonomous communication robots can achieve useful tasks in real ﬁelds
supported by a human operator.
Inspired by Norman’s human model (p. 28 in [7]),we developed a model of a semi-autonomous communication robot that consists of three layers: visceral, behavioral, and reﬂective. The visceral layer corresponds
to involuntary actions that can be done by such simple creatures as lizards. The behavioral layer corresponds to the subconscious behavior of mammals obtained through repeated training. The behavior fromthese two layers is unconsciously absorbed. Thus, humans can consider (reﬂective layer) how to behave evenwhen taking actions with these two layers, such as walking, driving, and so forth.
The reﬂective layer is operated by humans (bothoperators and developers) in our model. In the beginning, most of the other parts will also be performed byoperators, the reactive layer can be prepared from the
beginning, and the behavioral parts will be graduallyreplaced with software modules by developers. Therefore, most language communication will be continually
managed by human operators, and new behaviors will
be continually supplied by human developers.
This paper reports a semi-autonomous robot systemthat makes two major contributions. First, it demonstrates how ubiquitous sensors and an operator con-Fig. 1 Route guidance at a train station

tribute to achieve the semi-autonomy of a robot. In
other words, if we permit human operators, vast potential exists for robotics technology to further contributetoward human-like communication services. Second, it
demonstrates how people interact with guide robots ina train station, indicates how eﬀectively communication
robots support human activities in a real environment,
and provides some design implications for such communication robots.

2 Related Works
2.1 Autonomous Approach
Many past works focused on a robot that acts in everyday environments frequented by ordinary people [9–14]. For instance, Burgard et al. developed a tour guiderobot [9] with robust navigational skills that behaved asa museum tool. Siegwart et al. developed a robot thatguided people in large-scale environments [10]. Baueret al. realized a robust navigating robot under an unknown urban environment using GPS data, cameras,laser range ﬁnders, and interactions with people [12].
Some researchers developed robots to support people indaily environments such as shops [13, 14]. These cases
indicate that autonomous robots are already feasiblefor delivering pre-deﬁned messages associated with locations, particularly at places with many people whohave a special interest in robots, such as museums andworld expos.
However, the inputs for these robots are limited; although buttons and tactile sensors were robustly used,
these robots did not exploit natural language, whichlargely limited their capabilities.

2.2 Semi-Autonomous Approach
A human operator is often used to simulate the missing components of a system under development and toobserve people’s reactions to such a nascent system.
This is known as the Wizard of Oz (WOZ) method [15,16] in human-computer interfaces. Some studies haveused WOZ techniques for human-robot interaction, although these prototypes demonstrated little autonomy.
For example, Woods et al. used a tele-operated robotthat approached people to observe their reactions [17].
Green et al. also used a tele-operated robot in a livingroom setting to determine what services people needed[18].
However, it is diﬃcult for a human operator to becompletely responsible for a robot’s functionality. Our
semi-autonomous approach’s thrust resembles the WOZThe final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

3method: a human operator with a prototype systemgathers realistic data from users. On the other hand,
an important diﬀerence is that we are trying to separate the two major components, the reﬂective layer
and the behavioral/visceral layer, and to automate thelatter as much as possible. Since we do not intend to
immediately make our system autonomous, we assume
that the system can request help from human operators. At the same time, we are trying to minimize operator support by focusing on autonomy in the nonlanguage part, which will probably result in situations
where a few operators can control hundreds of robots.
In robotics, many studies have investigated the teleoperation of mobile robots, arm robots, pet-type robots[19], and even humanoids [20]. In particular, space exploration and similar domains require distant teleoperation that causes communication delay. Thus, several
studies have utilized partial autonomy in teleoperation,such as obstacle avoidance and goal-directed locomotion [21–23].
Unfortunately, these studies did not focus on natural human-robot interaction, which apparently requires
more complicated collaboration between an automatedsystem and human operators.

Fig. 2 System overview

3 System Configuration
Figure 2 shows a system overview that consists of ubiquitous sensors and three layers: reﬂective, behavioral,
and reactive. The basic software design follows the architecture of a communication robot [8](reactive andbehavioral layers). The system also has an operatorrequesting mechanism that autonomously requests help
from the human operator.
Environmental sensor data are sent to the robot by
a 802.11 b/g wireless network. The robot uses this sensor information to move and interact with people. The
robot sensor data and the environmental sensor data
are sent to the operator to support the robot by thesame network; basically the operator uses image datafrom the environmental cameras and sound informationfrom the robot. The details of each system are describedas follows.

Fig. 3 Robovie

height and weight are 120 cm and 40 kg. The robot
has the following degrees of freedom (DOFs): two for
its wheels, three for its neck, and four for each arm.
Its lower mobile base is a Pioneer 3-DX (ActiveMedia).
We used corpus-based speech synthesis [24] for generating speech. Robovie can work one hour without being
recharged. To communicate with other sensors and an
operator, it uses a 802.11b/g wireless network.
3.1.2 Floor Sensors

3.1 Sensor and Actuator
3.1.1 Robovie
Figure 3 shows “Robovie,” our interactive humanoid
robot that is characterized by its human-like physicalexpressions and its various sensors [8]. It has a head,
two arms, a body, and a wheeled-type mobile base. Its

To detect the positions of people, we used an external
remote PC and ﬂoor sensors because they can collecthigh-resolution data and are robust to occlusion. We
installed 128 ﬂoor sensors units, VS-SS-F (Vstone Corporation, Osaka), around the robot that covered a 4 x
8 m area. Each sensor unit is 500 [mm2 ] with a resolution of 100 [mm2 ]. Their output is 1 or 0; the ﬂoor

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

4

Fig. 4 Multiple people on ﬂoor sensors

sensor is either detecting pressure or it is not. Therefore, 25 binary data were acquired by one ﬂoor sensor.
Floor sensors are connected with each other throughan RS-232C interface at a sampling frequency of 5 Hz.
Fig. 4 shows an interaction scene between the robot andpeople (left) and an example of outputs from the ﬂoor
sensors (right). A black point indicates that a sensor
detected pressure from a robot or a person.
We used a Markov Chain Monte Carlo method anda bipedal model to estimate people’s positions on the
ﬂoor sensors [25] since it is one robust sensing method
for positions. It is free from occlusion, and the average
position error is less than 21 cm. Thus, it is useful whena person interacts closely with the robot.
The ﬂoor sensors enable us to achieve three crucial
functions for robot autonomy. The ﬁrst is an approaching function with which the robot can approach a detected person to start an interaction. For this purpose,
the ﬂoor sensor system estimates and sends the x-y coordinate position information of people who exist on thesensors using ﬂoor sensor data to the robot by network.
The second is a pointing gesture. The interacting
person’s position is used to generate a pointing gesturefor route-guidance. The interacting person’s distanceis also used to point at the destination by the robot.
For this purpose, the ﬂoor sensor system calculates the
distance information between the robot and the nearest
person using its position information.
The third function is the robot’s position compensation. Although it has an odometer to estimate its position, since this devise is aﬀected by wheel slip, it is
not very accurate. The system continues to track the
robot’s position, and such information enables the robot
system to maintain position accuracy. For this purpose,
the robot sends speed and odometry information to theﬂoor sensor system, which estimates the robot’s position with this information and sends the estimated position to the robot by network.
3.1.3 Speech Recognizer
For the speech recognition function, we prepared an external remote PC to which the robot sends audio input
from its directional microphone to achieve fast speech

recognition using a software application for automatic
speech recognition robust to noisy environments andspeaker variability (age and gender) [26]. In the frontend block, environmental noise is ﬁrst reduced by afeature-space noise suppression method using clean speech
Gaussian mixture models and Wiener ﬁlters. The speech
recognition block is comprised of two parallel decoders
that correspond to adult and child speech, and each decoder uses two phoneme acoustic models for male andfemale speech. Four diﬀerent levels of signal-to-noise
ratios were also implemented in the acoustic models toimprove robustness against noise. For the speech recognition block’s language model, we implemented a dictionary of about 100 words, including place names andgreetings, and a simple grammar that imposed rules forconnecting words in the dictionary. Finally, the speech
recognition results were sent to the robot by network.

3.1.4 Sound-Level Meter
We installed a sound-level meter with an external PC
to measure the environment’s noise level and to send
it to the robot to autonomously change its volume. For
example, the robot increases its volume when the noiselevel exceeds 70 [db] and decreases it when the noiselevel is less than 65 [db]. These thresholds were decided
beforehand based on the environment’s noise level.

3.1.5 Tele-operation Interface
We developed an operator interface for controlling therobot’s behaviors (Fig. 5) that consists of two windows:
controlling robot and informing the operator. The left
window is an interface for choosing all the needed situated modules of the robot, such as greetings, oﬀering route guidance, explaining several destinations, andsaying goodbye. The right window uses a sphere image
to inform the operator that he/she needs to assume
control. This interface changes to red when the robot
requests the operator’s help. With the software the operator can also control such low-level functions as wheel
speed, the position of each joint, and utterances; but
due to the time delay caused by excessive input, theoperator rarely uses them.
The operator uses vision information from six cameras through cables and the auditory information fromthe robot’s microphone transmitted through the wireless network. This information is available in real time
without delay.

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

5

Fig. 5 Control software for human operator

3.2 Reactive Layer
The conceptual purpose of the reactive layer is to achievesafe interaction with lifelike behavior. For lifelike behaviors, the robot controls eye movements based on output
from its touch sensors to exhibit lifelikeness. For safety,
the robot’s locomotion and motors stop when an object contacts its bumper or overload of any motor is
detected. These reactive behaviors were prepared for ageneral environment. Therefore, we only implemented
simple mechanisms in the reactive layer that work correctly and do not require software updating.

3.3 Behavioral Layer
The conceptual purpose of the behavioral layer is toachieve task/environment dependent behaviors. The behavioral layer consists of situated modules, a situatedmodule controller, and episode rules. The situated modules allow the robot’s interactive behavior with situation dependent sensory data processing to recognize reactions from humans. Because each module works ina particular situation, developers can easily implement
situated modules by only considering the particular limited situation. A situated module is implemented by directly coupling the communicative sensory-motor units
with others to supplement such sensory-motor units asutterances and gestures.
Episode rules describe the state transition rules among
the situated modules. The robot can autonomously interact with people with the behavioral layer. We implemented two types of situated modules: guidance behavior and greeting and free-play behaviors.
3.3.1 Situated Modules
In the beginning, the robot approaches a person detected by the ﬂoor sensors and initiates interaction by
greeting and oﬀering a handshake. If the person requests directions, the robot immediately starts guidance behaviors if it correctly recognized the utterance.
Adults often seek such information. The robot is also

capable of free-play behavior that is popular with children. The robot sometimes triggers tactile interaction
with a child by saying, “Let’s play a game called touch.”
After the child reacts, it continues performing free-play
behaviors as long as the child responds and initiates
brief small talk, such as “Where are you from?” It also
oﬀers children a hug. The robot also oﬀers such information around the station as, “There is a new shopping
center close to the station.” After it exhibits several
free-play behaviors, it initiates guiding behavior. At theend of the interaction, the robot exhibits goodbye behavior.
For the situated modules for guiding, our robot canguide visitors to 38 nearby places by asking, “Where
are you going?” If an interacting person responds, therobot starts to provide directions. For example, whenguiding a visitor to the bus stop, the robot points toward the exit and says, “Please go out this exit and turnright. You’ll immediately see the bus stop.” When the
robot explains the route, it utilizes a pointing gesture aswell as such reference terms as “this” and “that.” Since
Japanese has three types of reference terms associatedwith positional relationships between two interacting
persons and the object being discussed, we installed a“three-layer attention-drawing model” for these reference terms and pointing gestures [3]. Thus, the robot
autonomously generates behavior to guide visitors tothese destinations using appropriately chosen referenceterms and gestures. It also has a map for these locations. If the interacting person cannot directly see thedestination, such as a place outside the station, therobot points to a visible place, such as the exit, andverbally supplements the remaining directions.

3.3.2 Episode Rules
The relationships among behaviors are implemented as
rule-governing execution orders called episode rules tomaintain a consistent context for communication. Theirbasic structure consists of previous behaviors (e.g., successfully ﬁnished greeting behaviors) and subsequent
behaviors (e.g., oﬀering route-guidance behavior). The
situated module controller selects a situated module
based on the 1311 implemented episode rules. As described above, episode rules are designed to achieve thefollowing six kinds of behaviors: approaching a visitor, extending a greeting, oﬀering small talk, providing information around the station and free-play, oﬀering route guidance, and saying goodbye. Moreover, an
event-driven transition was described so that when apassenger initiates a route-guidance conversation, therobot begins to oﬀer route guidance.

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

6

Fig. 6 Illustration of interaction scenes with operator

Figure 6 shows one example of the episode rules
and interaction scenes between a robot and a person.
After the ﬂoor sensors detected the person’s position,
the robot approached the person with the approachingsituated module. This caused a reactive transition governed by episode rule 1. The robot greets the personby executing the situated module Greeting. After the
Greeting, the robot provides route guidance by executing the Oﬀer Guide. In this example, because the visitor
does not respond or the speech recognizer fails to detect what the visitor said, the Oﬀer Guide results in a
No answer, and the robot asks the visitor again usingOﬀer Again. At the same time, the operator-requestingmechanism (described below in 3.4) ﬁres so that theoperator is asked to assume control. The visitor mightanswer the robot with such a response as, I’d like to
go to the bus stop, which is heard by the operator. As
a result of the operator’s control, Guide to the bus stop
is ﬁnally selected.

3.4 Reﬂective Layer
The conceptual purpose of the reﬂective layer is to integrate an autonomous robot system with humans so thatthe system as a whole can process natural language,
think deeply, and improve with human support. The
robot can autonomously operate without the reﬂective
layer. In addition, using the reﬂective layer, the system
requests help from the human operator and starts towork autonomously when an interaction between the
robot and the target visitor is ﬁnished. In this system,
two types of information are used for the mechanism:
the sub-system’s report and behavior transition.

3.4.1 Reports from Sensors and Actuators
Each sensor and actuator can individually report problems to request operator assistance. For example, in
the reactive layer, the robot stops its body movements
and locomotion when an actuator detects a motor overload. The robot also stops its body movements and locomotion when a tactile sensor is continuously reacting
more than ﬁve seconds. Another mechanism detects interaction level errors. The speech-recognition module
monitors several negative phrases, such as “I don’t understand,” “That’s not right,” and “That isn’t what I
asked.” Such statements indicate a problem at the levelof human-robot interaction.
3.4.2 Behavior Transition
The second mechanism detects interaction-level problems. For example, if the interacting people ﬁnd thatthe robot does not answer correctly, most of the timethey simply leave without complaining. Such interactionlevel problems reﬂect the service contents. Therefore,
in this study, we focused on situations in which a robot
guides people to develop this mechanism.
Episode rules also monitor behavior transitions, andthe robot system requests the operator’s help when atransition pattern matches pre-deﬁned situations whose
details for trapping apparent errors are described below.
a) When the robot cannot hear anything in particular from a visitor after twice oﬀering route guidanceThis episode rule refers to the following situations:(a) an interacting person does not speak because he/she
is not interested in route guidance, which is outside
the range of the implemented interaction model, or (b)The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

7

the speech recognition module fails. If the route-guiding
module twice gets a no-answer, the system requests theoperator’s help.
b) When the robot continues the interactive behavior more than ten times without performing the routeguiding module
The robot usually exhibits the route-guiding module if people follow the ordinary interaction ﬂows. On
the other hand, people can cause a diﬀerent interaction
ﬂow, for example, by continuously ignoring the robot’s
handshake request and initiating interaction by touching its shoulder.
c) When the robot continuously oﬀers route guidance three or more times
The robot exhibits route-guidance behavior when
it recognizes such a spoken request as “please give mesome directions.” This episode rule refers to the following situations: (a) an interacting person is greatlyinterested in route-guidance behavior or (b) the robot’sroute guidance continuously fails, which results in continuous requests for route guidance.

4 Field Trial at a Train Station
4.1 Environment
The six-day experiment was conducted at a terminal
station of a railway line that connects residential districts with the city center with four to seven trains perhour. Fig. 7 shows the experiment’s environment. Most
users descend the stairs from the platform after exiting
trains. We set the robot and the sensors in front of theright stairway (Fig. 7) and informed the visitors thatthe robot can provide directions. As shown in Fig. 7, weplaced ﬂoor sensors in the center of a 4 * 8 [m] ﬂoor areaand six cameras on the ceiling. The sound-level meterwith an external PC was installed to the right undercamera ’F’. The robot moved on the ﬂoor sensors.
We recorded all sensory data including the datafrom the ﬂoor sensors, video images from the ceilingcameras and the robot’s camera, and the auditory information from the robot’s microphone. We used thesedata to investigate the eﬀectiveness of our robot system. We also received permission to record these video
and auditory data from the train station authorities
and placed posters in the station to inform visitors.

Fig. 7 Station map

robot. We asked those who interacted with the robot
to complete questionnaires after their interaction. Children were asked to ﬁll out questionnaires if they understood their purpose.

4.3 Conditions
We prepared an autonomous condition to reveal how
much the robot can do without human help to investigate how completely the autonomous robot can support
people in a real environment. We did not prepare a fulloperated condition because this study’s main purpose
is to investigate how the developed system supports
autonomous robots by minimizing the operator’s load.
In the experiments, we prepared several time slots andcounterbalanced their order.
Autonomous condition: The robot was completely
autonomous and did not use the functions in the reﬂective layer; the operator never assumed control.
Semi-autonomous condition: The robot used all
the implemented layers: reactive, behavioral, and reﬂective. It usually operated autonomously when there wereno visitors, so the operator just monitored the situation
without taking control of the robot.
As described in Section 3.5.1, the operator only controlled the robot’s behavior and worked with the speech
recognition function when the operator-requesting mechanism detected a need for operator help. The operator
did not monitor interactions unless the robot asked forhelp so that we could observe fairly well whether thissemi-autonomous mechanism works. The operator ﬁnished its control, and the system started to work autonomously when interaction between the robot andthe target visitor ﬁnished. The operator received morethan two hours of training for controlling the robot.

4.2 Participants
5 Results

The station visitors were mainly commuters and students, and on weekends families visited the station to
see the robot. The visitors could freely interact with ourThis section reports the results from two major viewpoints: technical achievements and attitudes toward theThe final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

8

robot. The former consists of system performance, thesuccess rate of the route guidance, the operating time,
the performance of the operator-requesting mechanism,and the success rate of speech recognition. The latter
consists of how visitors interacted with the robot andquestionnaires of subjective impressions.

5.1 Technical Achievements
5.1.1 System Performance
The robot system worked quite well under both conditions (Fig. 8). Based on the position information fromthe ﬂoor sensors, it autonomously approached and interacted with 168 people during the trials: 77 people in
the autonomous condition and 91 people in the semiautonomous condition. When the robot provided route
guidance, it correctly pointed with gestures calculatedby the position relationships between the visitor anditself.
Note that the sensors robustly estimated the positions of people and the robots in crowded situations;for example, even though many occlusions were causedby the presence of over 15 people, no collisions occurred(Fig. 9). The robots never caused a dangerous situation(e.g., with small children or senior citizens).
In the semi-autonomous condition, the operator also
worked well as a speech recognition function for the
robot in the trials. For example, during crowded andnoisy situations, the robot smoothly talked with thevisitors with operator support.
5.1.2 Task Performance
For each visitor who responded to the robot’s oﬀer ofroute-guidance or asked about a route, we assume thatthe robot successfully gave directions if it correctly offered one or more route-guidance directions. Thus, evenwhen a visitor asked for routes to more than one place,
the guidance was judged a success.
In the autonomous condition, the success rate was
29.9% among 77 visitors. The main cause of failure
was speech recognition error. In the semi-autonomous
condition, the success rate was 68.1% among 91 visitors. 31.9% failure remained, mainly due to visitors
who stopped interacting with the robot before the operator assumed control. In addition, speech recognition
failure sometimes caused a breakdown of the operatorrequest actions. If the speech recognizer simply failed
to detect the speech or the recognition result was rejected because it did not match the pre-assumed model(this often happened), the operator was successfully requested. The problem was when the speech recognizer

picked up a false positive result, which resulted in mistaken guidance even though the system had not detected the situation as problematic.
5.1.3 Operating TimeThe experimental time was 45,900 seconds, the overall interaction time was 20,078 seconds, and the overall
idling time was 25,822 seconds. The robot autonomouslyinteracted with people for 8,551 seconds; the operating time (when the operator controlled the robot) was
11,527 seconds. Thus, the operator controlled the robot
25% (11,527/45,900) of the experimental time in thesemi-autonomous condition.
A tradeoﬀ exists between task performance and operating time. That is, higher operating time results inbetter task performance, but it also requires more elaborate operator control. In our case, we designed thesystem to minimize operator time; if the operator controlled the robot from the beginning of the interaction,task performance would increase.
5.1.4 Performance of Operator-requesting Mechanism“Operator needed” situationsFirst, we evaluated the performance of the operatorrequesting mechanism when the operator is needed,deﬁned as a situation where a person silently looked at
the robot for 10 seconds after it talked to the person
or where a person asked the robot twice for a routeguidance. We measured such cases: the operator-requestingmechanism called the operator within 10 seconds, 20
seconds, or the end of interaction when a operator isneeded situation happened.
The reason for the deﬁnition of an operator isneeded situation depends on the observed interactions
between the robot and visitors, as described in Section
5.2.1. We often observed people repeatedly for routeguidance in the autonomous condition; they observed
the robot and talked to it for more than 20 seconds,
even though the robot did not react to them. In addition, some children silently looked at the robot morethan 10 seconds after the robot addressed them.
“Operator is needed” situations happened 85 times
within interactions with 91 subjects. The operator-requestingmechanism called the operator 32 times (37.6%) within
10 seconds, 61 times (72.9%) within 20 seconds, and 72
times (84.7%) before the end of the interaction. Therefore, the operator-requesting mechanism detected almost all situations that the robot could not handle byitself. We believe that the mechanism improved the success rate of the route guidance.
“Operator is not needed” situations

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

9

We also evaluated the performance of the operatorrequesting mechanism in “operator is not needed” situations, deﬁned as situations where an “operator isneeded” did not happen in an interaction and where
nobody was on the ﬂoor sensors more than three seconds.
“Operator is not needed” situations happened 490
times in the semi-autonomous condition. The operatorrequesting mechanism mistakenly called the operator
23 times (4.7%). The cause of the mistakes is based on
the noise information of the ﬂoor or the tactile sensors
and speech recognition errors. Therefore, the operatorrequesting mechanism detected almost all of the “operator is not needed” situations. We believe that themechanism reduced the operating times.

Fig. 8 Interaction between robot and visitors

5.1.5 Success Rate of Speech Recognition
We evaluated the speech recognizer’s performance, whichis critically relevant to task performance in the autonomous condition, by calculating the correct answers
per speech utterances recognized by the robot. If therobot system failed to detect speech, such as a voicethat is too low, it was not counted. A correct answer isdeﬁned as speech where the speech recognizer outputs
a recognized word whose meaning matches the visitor’s
speech.
The system detected 1,571 sentences during the ﬁeld
trials, 334 of which were correctly recognized. This is a
success rate of only 21.3%, although in the laboratory
we achieved word accuracies that exceeded 90% with
70 dBA of background noise [26]. This contradiction
indicates the ineﬃciency of current technologies in realworld situations.
Speech recognition failed for several reasons: mismatching with the prepared language model, inadequate
vocabulary, excessively low speech volume, excessively
loud voices (mainly from children), and non-constantenvironmental noise. In the station, the noise level usually ranged between 65 and 70 dBA, which is not quiet
but is still a possible level for the speech recognizer.

5.2 Attitudes toward the robot
5.2.1 Visitor Interactions
Visitors freely interacted with the robot, especially many
who seemed curious about such interaction. When arobot oﬀered route guidance, some people repeatedly
asked for such simple destinations as a vending machineor the toilet. They seemed fascinated by the robot andcontinued to interact with it even though it failed toreact to their speech due to speech recognition errors.

Fig. 9 Many people interacting with robot

Moreover, bystanders often observed conversations between the robot and other visitors, particularly parentsof children who were interacting with the robot.
Perhaps, more interesting interactions reﬂect thesmaller numbers of people who actually used the robots.
Some visitors asked for information about a place thatthey really seemed to want to ﬁnd, such as the nearest
bus stop or shopping mall. These people appeared satisﬁed with the information from the robot, said thankyou, and left after getting route guidance (Fig. 8).
Moreover, interactions between the robot and families were also interesting. Some parents encouraged theirchildren to interact with the robot when it addressed
them. Most children showed some anxiety because therobot was completely new to them; they often did not
say anything for more than 10 seconds (Fig. 10-(a)).
In such situations in the semi-autonomous condition,
the operator-requesting mechanism sometimes called
the operator because the robot could not hear anything
for a long time or because interactive behaviors wererepeated over ten times. Then the operator supported
the robot by selecting such behaviors as oﬀering route
guidance again (Fig. 10-(b)).
Other interesting scenes involved multiple groups ofpeople around the robot, as shown in Fig. 9. In suchsituations, diﬀerent groups simultaneously interactedwith the robot. They did not interact with each otherdirectly, but they interacted with the robot in turn.
Moreover, some parents ordered to their children to
take turns with other children.

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

10

(a)

derstanding, and Naturalness (p<.01). For these impressions, the subjects evaluated the semi-autonomous
robot more highly than the autonomous robot. In otherwords, these results indicate higher acceptability of thesemi-autonomous robot than the autonomous robot; wenote that all items on the autonomous condition arearound the middle. We think the results also indicatethat visitor basically accepted the autonomous robot.
Furthermore, there was no signiﬁcant diﬀerence in the
Safety impression.

6 Discussion
6.1 Design Implications(b)
Fig. 10 Family interacting with robot in semi-autonomous condition

This study also oﬀers design implications. In this section, we describe them based on observations of theinteractions between the robot and ordinary people in
the train station.
6.1.1 Bystanders who overheard interaction

Fig. 11 Subjective impressions

5.2.2 Subjective Impressions
We asked all the visitors who interacted with the robot
to answer a questionnaire in which they rated items on ascale of 1 to 7, where 7 is the most positive. We gathered
77 questionnaires in the autonomous condition and 91
in the semi-autonomous condition. The following items
were used:- Explanation: degree to which you understood therobot’s explanations- Understanding: degree to which the robot understood you- Naturalness: degree of naturalness of robot’s behavior- Safety: degree to which you think the robot is safe
Figure 11 shows the questionnaire results. We veriﬁed the diﬀerences between the semi-autonomous andautonomous conditions with an Analysis of Variance(ANOVA) that revealed signiﬁcant diﬀerences between
the conditions for the impressions of Explaining, Un-

In the trials, we often observed bystanders who were
just looking at the conversations between the robot
and other visitors. Unfortunately, most bystanders werestanding beyond the ﬂoor sensors, so the robot could
not approach them. If the environmental system covered a wider area in the station, the robot could interact
with more people.
One of the possibilities for covering a wider area is
to use laser range ﬁnders. Recently, Dylan et al. developed a robust human-tracking system with multiple
laser range ﬁnders [27] that might enable a robot tomove by using the robust position information of therobot and visitors. Satake et al. also proposed a method
for a mobile robot to approach visitors more naturally
by considering their trajectories [28]. From another perspective, Shiomi et al. investigated how human-robot
interaction changes when the robot moves forward or
backward to encourage people to listen to a guide robot[11]. These approaches are also useful to increase thenumber of interacting people.
6.1.2 Response timing of the robot in conversations
The robot sometimes could not respond to visitors because it was waiting for the speech recognition result
from the speech recognizer or the operator when the
robot was talking with visitors. Long waiting times might
give negative impressions to the interacting visitors; infact, some visitors reported such feelings in their freeanswer comments.

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

11

We believe that a conversational ﬁller behavior isuseful to reduce such negative feelings [29]. In Japanese“etto” is used to buy time and resembles “well...” or“uh...” in English. When visitors ask the robot something, it should use such words to buy time while itis waiting for the speech recognition results from the
speech recognizer or the operator.

6.2 Perspective for Development Methodology

6.1.3 Interaction with multiple groups of people

6.2.1 Finding Interaction FlowsMultiple groups of people often simultaneously interacted with the robot, as shown in Fig. 9. At ﬁrst, we
assumed that each group of visitors such as a family
might prefer to interact with the robot. Thus, the interaction ways in real environments were quite diﬀerent
from our estimation.
To more naturally interact with multiple groups ofpeople, the robot should consider their purposes toward
the robot. If they conﬂict, the situations will be chaotic.
To prevent such situations, the robot should control social situations and explicitly indicate the contexts tounify everyone’s purposes toward the robot. [30]. To interact with a group simultaneously, the robot should estimate whether a group’s state is suitable for the robot’s
intended task [31].

In this study, people’s main interest was testing therobot’s capability, not receiving information. Even after receiving the information, some lingered around therobot and continued to interact with it. In fact, somepeople asked the robot about route guidance more than
three times.
At ﬁrst, we assumed that visitors might prefer to ﬁnish the interaction after receiving the route guidance.
Thus, the interaction ﬂows in real environments werequite diﬀerent from our estimation. Therefore, in autonomous condition, the robot sometimes failed to interact with people well.
On the other hands, in the semi-autonomous condition, the operator was able to supply a ﬂexible interaction ﬂow in such situatios. We expect that analysis of
the operation logs will enable us to improve the interaction ﬂow by retrieving a typical interaction ﬂow madeby the operator.

6.1.4 Eﬀects of operator’s existence
In the semi-autonomous conditions, we did not explicitly admit the existence of the operator. If we had confessed the operator’s existence, would the interactions
change? Yamaoka et al. investigated how people feel
when they are interacting with the robot itself or a human behind it [32].
They reported that two-thirds of the participantsof the experiments felt that they were interacting withthe robot itself even if they were informed about theoperator’s existence. Their enjoyment was unaﬀectedby the knowledge of whether the robot was controlled
by a program or a human, although their impressionof robot intelligence indicated that they distinguishedbetween these conditions. Therefore, we think that theoperator’s existence would not change interactions dramatically.
On the other hand, visitors must be informed of theoperator’s existence if the robot’s services are closely
related to such privacy issues as using personal information. Our study was conducted as an academic trial
to investigate the eﬀectiveness of a semi-autonomous
robot system in a real environment with a route-guidanceservice; no such problems happened in our study. We
note that a semi-autonomous approach is a powerful
way to realize an actual working robot in a real environment, but ethical issues must be considered carefully.

In this section, we report the prospects of completely
exploiting the gathered realistic user data as a reﬂective layer. These data enable us to implement greater
autonomy in the system, even though we only improveda few of the robot’s functions based on this approach.

6.2.2 Incremental Developing Behaviors
One of the diﬃculties in developing such a real ﬁeld system is that predicting all the behaviors of people is very
diﬃcult. In the experiment, we prepared several destinations for guidance about facilities without including
such simple destinations that visitors could see because
they were quite near the robot.
However, the main interest of visitors was testing
the robot’s capability; visitors often asked for such places
as the vending machines in the station, even though
they were visible just a few meters away. Thus, we implemented such guidance behavior during the experiment. Because the provided service was very simple,
we did not have another chance to incrementally implement behaviors. We believe that the need for incremental development will increase if the robot’s task becomes more complex.
6.2.3 Decreasing operator load
One important future work in semi-autonomous robot
systems is decreasing the operator load. This can beachieved in two ways: increasing the robot’s autonomy
and developing more useful interfaces for operations.

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

12

The former is related to developing a recognition
system about the environment around the robot andthe interacting people. For example, robust position estimation and localization within a wide area are important functions to increase the robot’s autonomy [27,33]. Another perspective for the former is to learn theoperator’s decision or interaction logs with sensor information because such log information provides powerfullearning data for robot behavior [34,35].
Related to the latter, some researches enable one operator to control multiple robots by reducing the operator load [6, 36]. One problem in teleoperation with multiple conversational robots is the conﬂict of using theoperator’s resources. The operator can only deal withspeech recognition for one robot at a time, even thoughmultiple robots simultaneously need the resource. Conversational interactions tend to follow patterns thatsometimes make it possible to anticipate the need forthe operator. Dylan et al. scheduled behaviors to avoid
conﬂicts about operator resources [6]. These approaches
will decrease the load of operators who control communication robots.

6.2.4 Improving Speech Recognition Performance
In our experiments, even though the prepared speech
recognition system achieved 92.5% word accuracy in anindoor, 75 dBA noise environment, it only resulted in
21.3 accuracy in the real environment.
Currently, the most critical failure is caused by speech
recognition, which mainly reﬂects utterances that do
not ﬁt the implemented language model. For example,
even though Would you tell me the route to Kyoto?
is included in the model, it has diﬃculty recognizing the
keywords in such similar utterances as “Tell me how to
go to Kyoto” (grammar mismatch) and “Would you tellme the route to Osaka?” (vocabulary mismatch). Thisis critical because daily conversation has various waysof expressing ideas.
We analyzed how many utterances spoken by visitors matched the implemented language model, which
resulted in a rate of 51.3%. Speech recognition often
fails when mismatches exist between utterances and themodel. One of the diﬃculties of improving the performance of speech recognition in a real environment is the
producibility of the situations; however, such real data
must be gathered and used to increase system performance and to develop a new system that works in realenvironments. Therefore, we plan to improve the performance for the remaining 48.7% of the utterances byadding mismatched utterances to the language model.

6.3 Limitations
Since we only conducted tests with a particular robot
and sensors, an operator, and in the speciﬁc environment of a train station, the generality of our ﬁndings islimited from the viewpoint of reproducibility. However,such a situation is diﬃcult to avoid in human-robot interaction because using two or more diﬀerent robots
is too expensive; generalizing ﬁndings by preparing alarge number of operators with knowledge of robotics
and an understanding of the system in diﬀerent environments is also diﬃcult. Yet we believe such ﬁeld trials
are critical to investigate how the current technologies
work in real situations and how to improve them by realistic data. We believe that our ﬁndings are applicableto other robots with similar appearance and interaction
complexity.

7 Conclusion
We implemented a networked robot system that consists of a semi-autonomous communication robot, ﬂoor
sensors, cameras, and a sound-level meter. Moreover,
we implemented an operator-requesting mechanism that
autonomously detects situations that the robot cannot
handle by itself and requests that a human operator assume control. This mechanism is an important function
for semi-autonomous robots. Such basic communicative
behaviors as greetings and route guidance are implemented for the robot, which autonomously approaches
visitors and interacts based on the position information
estimated by the ﬂoor sensors. The robot autonomously
controls the interaction ﬂows based on sensory information.
We conﬁrmed that the robot system worked correctly in a real environment through a ﬁeld trial at a
train station where the robot was given a route-guidancetask. The results suggest the promising potential of therobot system for serving people. The mechanism correctly requested operator’s help in 84.7% of the necessary situations. The operator only controlled 25% of
the experiment time and mainly operated such highlevel functionality as the transition of behaviors. The
robot system successfully guided 68% of the visitors
whose subjective impressions were also good and indicated high acceptability of our robot in the public
space. Observed interaction scenes between the robot
and visitors also provided design implications. Important future work includes establishing a methodologythat utilizes the gathered data to improve the robot’s
performance.

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

13
Acknowledgements We wish to thank the staﬀ of the Kinki
Nippon Railway Co., Ltd. for their cooperation. We also thankthe following ATR members for their helpful suggestions and cooperation: Satoshi Koizumi, Toshihiko Shibata, Osamu Sugiyama,
Kenta Nohara, and Anton Zolotkov. This research was supportedby the Ministry of Internal Aﬀairs and Communications of Japan.

References
1. C. Breazeal and B. Scassellati: “Infant-like social interactions
between a robot and a human caretaker,” Adaptive Behavior,
8(1), 2000.
2. Bilge Mutlu, Fumitaka Yamaoka, Takayuki Kanda, Hiroshi
Ishiguro, and Norihiro Hagita, Nonverbal Leakage in Robots:
Communication of Intentions through Seemingly Unintentional
Behavior, 4th ACM/IEEE International Conference on HumanRobot Interaction (HRI2009), pp. 69-76, 2009.
3. Hideaki Kuzuoka et al. “Reconﬁguring spatial formation arrangement by robot body orientation,” 5th ACM/IEEE International Conference on Human-Robot Interaction(HRI2010),
pp. 285-292, 2010.
4. Toru Takahashi, Kazuhiro Nakadai, Kazunori Komatani, Tetsuya Ogata, and Hiroshi G. Okuno: Missing-Feature-Theorybased Robust Simultaneous Speech Recognition System with
Non-clean Speech Acoustic Model. Proceedings of IEEE/RSJ
International Conference on Intelligent Robots and Systems(IROS-2009)
5. A. Sanfeliu, N. Hagita, and A. Saﬃotti, “Network Robot Systems,” Special Issue: Network Robot Systems, Robotics andAutonomous Systems, 2008.
6. Dylan F. Glas et al.: “Field Trial for Simultaneous Teleoperation of Mobile Social Robots,” HRI2009, to appear.
7. D. A. Norman: Emotional design, Basic Books, 2003.
8. T. Kanda, H. Ishiguro, M. Imai, and T. Ono: “Development
and Evaluation of Interactive Humanoid Robots,” Proceedingsof the IEEE, Vol. 92, No. 11, pp. 1839-1850, 2004.
9. W. Burgard et al.: “The interactive museum tour-guiderobot,” National Conference on Artiﬁcial Intelligence, pp. 1118, 1998.
10. R. Siegwart et al.: “Robox at Expo.02: A Large Scale Installation of Personal Robots.” Robotics and Autonomous Systems,
42, 203-222, 2003.
11. M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita: “A Larger
Audience, Please! - Encouraging people to listen to a guiderobot -,” 5th ACM/IEEE International Conference on HumanRobot Interaction (HRI2010), 2010.
12. Andrea Bauer et al., “The Autonomous City Explorer: Towards Natural Human-Robot Interaction in Urban Environments,” Int J Social Robotics, vol. 1, no. 2, Springer, 2009.
13. M.K. Lee, J. Forlizzi, P. E. Rybski, F. Crabbe, W. Chung, J.
Finkle, E. Glaser, and S. Kiesler, “The Snackbot: Documenting
the design of a robot for long-term human-robot interaction,”
Proceedings of HRI’09, 7-14.
14. H.-M. Gross, H.-J. Böhme, C. Schröter, S. Müller, A. König,
C. Martin, M. Merten, and A. Bley, “ShopBot: Progress in
Developing an Interactive. Mobile Shopping Assistant for Everyday Use,” in International Conference on Systems, Man andCybernetics, 2008
15. D. Dahlback, A. Jonsson, and L. Ahrenberg: “Wizard of Oz
studies - why and how, Knowledge-based systems,” Vol. 6, No.
4, pp. 258-266, 1993.
16. S. Dow, et al.: “Wizard of Oz Support throughout an Iterative Design Process,” Pervasive computing, Vol. 4, No. 4, 2005.

17. S. Woods et al.: “Comparing Human Robot Interaction Scenarios Using Live and Video Based Methods,” Towards a Novel
Methodological Approach, Int. Workshop on Advanced Motion
Control, 2006.
18. A. Green et al.: “Applying the Wizard-of-Oz Framework to
Cooperative Service Discovery and Conﬁguration,” Proc. IEEE
Int. Workshop on Robot and Human Interactive Communication, 2004.
19. Lee, Jun Ki, Stiehl, Walter Dan, Toscano, Robert Lopez,
and Breazeal, Cynthia. “Semi-Autonomous Robot Avatar as a
Medium for Family Communication and Education,” Advanced
Robotics, vol. 23, no. 14, pp. 1925-1949(25), Brill, 2009
20. N. E. Sian et al.: “Whole Body Teleoperation of a Humanoid
Robot Integrating Operator’s Intention and Robot’s Autonomy,” IROS2003, pp. 1651-1656, 2003.
21. B. P. Sellner et al.: “Attaining Situational Awareness for Sliding Autonomy,” HRI2006, pp. 80-87, 2006.
22. Andrew Correa, Matthew R. Walter, Luke Fletcher, Jim
Glass, Seth Teller, and Randall Davis, Multimodal Interaction
with an Autonomous Forklift, in 5th ACM/IEEE International
Conference on Human-Robot Interaction (HRI) 2010, Osaka
Japan, pp. 243-250, March 2010.
23. Tiﬀany L. Chen and Charles C. Kemp, Lead Me by the
Hand: Evaluation of a Direct Physical Interface for Nursing
Assistant Robots, in 5th ACM/IEEE International Conferenceon Human-Robot Interaction (HRI) 2010, pp. 243-250, March
2010.
24. H. Kawai, T. Toda, J. Ni, M. Tsuzaki, and K. Tokuda,
XIMERA: “A New TTS from ATR Based on Corpus-Based
Technologies,” Proc. of Fifth ISCA Workshop on Speech Synthesis (SSW5), pp. 179-184, 2004
25. T. Murakita et al.: “Human Tracking using Floor Sensors
based on the Markov Chain Monte Carlo Method,” Proc. Int.
Conf. Pattern Recognition (ICPR04), pp. 917-920 (2004).
26. C. T. Ishi et al.: “Robust speech recognition system for communication robots in real environments,” International Conference on Humanoid Robots, 2006.
27. D. Glas et al., “Laser Tracking of Human Body Motion Using Adaptive Shape Modeling,” In Proc. Int. Conf. Intelligent
Robots and Systems, pp. 602-608. 2007.
28. S. Satake, T. Kanda, D. F. Glas, M. Imai, H. Ishiguro, and
N. Hagita, “How to Approach Humans?-Strategies for Social
Robots to Initiate Interaction,” HRI2009, pp. 109-116, 2009.
29. Toshiyuki Shiwa, Takayuki Kanda, Michita Imai, Hiroshi
Ishiguro, and Norihiro Hagita, How Quickly Should a Communication Robot Respond?, International Journal of Social
Robotics, 1(2), pp. 141-155, 2009.
30. M. Shiomi, T. Kanda, S. Koizumi, H. Ishiguro, and
N. Hagita, “Group Attention Control for Communication
Robots,” International Journal of Humanoid Robotics (IJHR),
2008.
31. M. Shiomi, K. Nohara, T. Kanda, H. Ishiguro, and N. Hagita,“Estimating Group States for Interactive Humanoid Robots,”
IEEE International Conference on Humanoids, Dec. 2007.
32. Fumitaka Yamaoka, Takayuki Kanda, Hiroshi Ishiguro, andNorihiro Hagita, Interacting with a Human or a Humanoid
Robot?, IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS2007), pp. 2685-2691, 2007.
33. D. F. Glas, T. Kanda, H. Ishiguro, and N. Hagita, Simultaneous People Tracking and Localization for Social Robots Using
External Laser Range Finders, in Proc. 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
2009), pp. 846-853, 2009.
34. M. Shiomi, T. Kanda, K. Nohara, H. Ishiguro, and N. Hagita,“Adaptive supervisory controls of a communication robot thatapproaches visitors,” International symposium on Distributed
Autonomous Robotic Systems (DARS) 2008.

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

14
35. S. Ikemoto, H. B. Amor, T. Minato, H. Ishiguro, and B. Jung.
Physical Interaction Learning: Behavior Adaptation in Cooperative Human-Robot Tasks Involving Physical Contact. 18th
IEEE International Symposium on Robot and Human Interactive Communication, 2009.
36. Raj M. Ratwan, J. Malcolm McCurry, and J. Gregory
Trafton, “Single Operator, Multiple Robots: An Eye Movement
Based Theoretic Model of Operator Situation Awareness,” in
5th ACM/IEEE International Conference on Human-Robot Interaction (HRI) 2010, pp. 243-250, March 2010.

Masahiro Shiomi received M. Eng and Ph.D. degrees
in engineering from Osaka University, Osaka, Japan in
2004 and 2007, respectively. From 2004 to 2007, he wasan Intern Researcher at the Intelligent Robotics andCommunication Laboratories (IRC) and is currently a
Researcher at IRC at the Advanced Telecommunications Research Institute International (ATR) in Kyoto,
Japan. His research interests include human-robot interaction, interactive humanoid robots, networked robots,
and ﬁeld trials.
Daisuke Sakamoto received his B.A.Media Architecture, M.Systems Information Science, and Ph.D. in
Systems Information Science from Future UniversityHakodate in 2004, 2006, and 2008, respectively. From2006 to 2008, he worked for ATR Intelligent Robotics
and Communication Labs. (IRC) as Intern Researcher.
From 2008 to 2010, he worked at The University ofTokyo as a Research Fellow of the Japan Society for
the Promotion of Science, and continues to be involvedin the JST ERATO Igarashi design interface project asa Collaborator, and ATR IRC as a Cooperate (Visiting) Researcher. He is now a Researcher of JST ERATO Igarashi Design Interface Project. His researchinterests include Human-Robot Interactions, HumanComputer Interaction, and Interaction Design of Interactive Robots.
Takayuki Kanda received his B. Eng, M. Eng,and Ph. D. degrees in computer science from Kyoto
University, Kyoto, Japan, in 1998, 2000, and 2003, respectively. From 2000 to 2003, he was an Intern Researcher at the ATR Media Information Science Laboratories and is currently a Senior Researcher at theATR Intelligent Robotics and Communication Laboratories, Kyoto, Japan. His current research interests include intelligent robotics, human-robot interaction, andvision-based mobile robots. Dr. Kanda is a member of
ACM, the Robotics Society of Japan, and the Information Processing Society of Japan.
Carlos Toshinori Ishi received the B.E. and M.E.
degrees in electronic engineering from the Instituto Tecnologico de Aeronautica (Brazil) in 1996 and 1998, re-spectively. He received the PhD degree in engineering from The University of Tokyo (Japan) in 2001. Heworked at the JST/CREST Expressive Speech Processing Project from 2002 to 2004 at ATR Human Information Science Laboratories. He is currently with ATRIntelligent Robotics and Communication Laboratories,
since Jan. 2005. His major interests are on prosodic andvoice quality information extraction from speech signals, linguistic and paralinguistic information processing, auditory processing, sound source localization andnoise suppression processing, and application for communication robots. Dr. Ishi is a member of the Acoustic
Society of Japan, and the Robotics Society of Japan.
Hiroshi Ishiguro received his D. Eng. degree from
Osaka University, Japan in 1991. In 1991, he started
working as a Research Assistant in the Department
of Electrical Engineering and Computer Science, Yamanashi University, Japan. He moved to the Department of Systems Engineering, Osaka University, Japan,
as a Research Assistant in 1992. In 1994, he becamean Associate Professor of the Department of Information Science, Kyoto University, Japan, and started research on distributed vision using omnidirectional cameras. From 1998 to 1999, he worked in the Department
of Electrical and Computer Engineering, University ofCalifornia, San Diego, USA, as a visiting scholar. He
has been a Visiting Researcher at the ATR Media Information Science Laboratories since 1999 and developed Robovie, an interactive humanoid robot. In 2000,
he moved to the Department of Computer and Communication Sciences, Wakayama University, Japan, as
an Associate Professor and became a Professor in 2001.
He is now a Professor of the Department of Adaptive
Machine Systems, Osaka University, Japan, and a groupleader at ATR Intelligent Robotics and Communication
Laboratories.
Norihiro Hagita received B.S., M.S., and Ph.D.
degrees in electrical engineering from Keio University,
Tsuruoka City, Japan in 1976, 1978, and 1986, respectively. From 1978 to 2001, he was with the Nippon Telegraph and Telephone Corporation (NTT). He joined
the Advanced Telecommunications Research InstituteInternational (ATR) to establish the ATR Media Information Science Laboratories and the ATR Intelligent Robotics and Communication Laboratories in 2001
and 2002, respectively. He is now a director at ATR
ICT Environment Research Laboratories Group and
ATR Intelligent Robotics and Communication Laboratories. His current research interests include communication robots, network robot systems, interaction media, and pattern recognition.He is a Fellow of the Institute of Electronics, Information, and Communication
Engineers, Japan and the ATR. He is also a member

The final publication is available
http://link.springer.com/article/10.1007/s12369-010-0077-4

15of the Robotics Society of Japan, the Information Processing Society of Japan, and The Japanese Society for
Artiﬁcial Intelligence. He is a Co-Chair for the IEEE
Technical Committee on Networked Robots.