Development of an Android Robot for Studying
Human-Robot Interaction
Takashi Minato1, Michihiro Shimada1, Hiroshi Ishiguro1, Shoji Itakura2
1Graduate School of Engineering, Osaka University,

2-1 Yamada-oka, Suita, Osaka, 565-0871, Japan{minato,ishiguro}@ams.eng.osaka-u.ac.jp
shimada@ed.ams.eng.osaka-u.ac.jp
2Graduate School of Letters, Kyoto University,
Yoshida Honmachi, Sakyo-ku, Kyoto 606-8501, Japan
itakura@psy.bun.kyoto-u.ac.jp

Abstract. Behavior or Appearance? This is fundamental problem in robot development. Namely, not only the behavior but also the appearance of a robot
influences human-robot interaction. There is, however, no research approach to
tackling this problem. In order to state the problem, we have developed an android robot that has similar appearance as humans and several actuators generating micro behaviors. This paper proposes a new research direction based on
the android robot.

1 Introduction
In recent years, there has been much research and development of intelligent partner robots that can interact with humans in daily life, such as Sony AIBO and Honda
ASIMO. In this research, communication between the robots and humans is emphasized in contrast to industrial robots performing specialized tasks. Meanwhile, theintelligence of a robot is a subjective phenomenon that emerges during human-robot
interaction. It is, therefore, indispensable to reveal a principle of human-robot andhuman-human communication, that is, a principle of interaction for developing apartner robot and realizing its intelligence.
Some researchers have tackled this problem. For example, Kanda et al. [1] andScheeff et al. [2] evaluated how the behavior of their robots affects human-robot
interaction by observing their interaction. These works have gradually revealed theeffects of robot behavior on human-robot interaction. There is, however, a possibility
that robotic appearance distorts our interpretation of its behavior. The appearance ofthe robot is essentially one of its functions; therefore, the effect of appearance mustbe evaluated independently. It is generally difficult to isolate the effects of a robot’s
behavior from those of the robot’s appearance which is dissimilar from humans. One
way to discriminate is developing a robot whose appearance is the same as humans.
This paper proposes a new research direction to tackle a fundamental problem of arobot behavior and appearance using a robot called an android which has similar
appearance to humans. To state the problem, we form a fundamental hypothesis aboutthe effect of a robot’s behavior and appearance using existing knowledge gained fromresearch or practical experience. Moreover, we design experiments using the developed android. This research is currently in progress and only preliminary experimental results have been obtained. In this paper we describe the hypotheses and the developed android, and finally present brief results of preliminary experiments.
The rest of paper is organized as follows. Section 2 and 3 describe our research objective and hypotheses. Section 4 introduces the android robot developed for thisresearch. Section 5 shows brief results of the experiments to observe behavior of
subjects while interacting with the android. Finally, Section 6 presents conclusions.

2 Purpose and Approach
Our goal is to create a design methodology of robot behavior and appearance torealize natural communication between robots and humans. Our approach is to form ahypothesis about the effects of behavior and appearance on interaction and examine
the hypothesis in a psychological experiment.
Kanda et al. [1] investigated the effects of interactive behavior using a humanoid
robot named “Robovie” [3]. It is, however, possible that the results depend on theappearance of Robovie because its robotic appearance influences the interaction. In
the psychological field, Johnson et al. [4] reported that infants followed the gaze of anovel object that had facial features and contingent interactivity and did not follow an
object that did not have facial features or contingent interactivity. According to thisevidence, it is clear that the effect of a robot’s appearance cannot be ignored.
There is a bottom-up approach to tackle the “behavior versus appearance problem,” in which the interaction is evaluated while incrementally enhancing the behavior or appearance of the robot. However, there is also a top-down approach, in which
we initially build a robot which has the same motion and appearance as humans andevaluate the interaction while removing some aspect of behavior or appearance.
To employ the later approach, we introduce an android robot that has a similar appearance as humans. McBreen and Jack [5] evaluated some human-like agents whichwere created from human photorealistic images in an e-retail application (a homefurnishings service). The results show that the conversation with the video agent isthought to be more natural than the conversation with the other agent (e.g., a 3-Dtalking head, a still image with facial expressions, and a still image). This work suggests that the close resemblance to humans removes the effect of the robot‘s dissimilar appearance and enables an investigation purely of the effect of behavior. Comparing the results with the android and other humanoid robots, the effects of behavior
and appearance are extracted independently.
In traditional robot research, the design of a robot appearance has been entrusted toan artistic designer and not had an engineering meaning. However, the robot’s appearance can be designed based on the engineering methodology from our result.
In this research, it is necessary to evaluate human behavior in the interaction withthe android. There are qualitative and quantitative methods to evaluate. Kanda et al.
[6] employed a qualitative method by measuring the psychological attitudes of people

Fig. 1. uncanny valley
using the semantic differential method (SD). However, it is difficult to prepare opposite pairs of adjectives in a questionnaire to obtain a result that is explicable.
Some researchers quantitatively evaluate human behaviors. For example, Matsuda
et al. [7] investigated the brain activities of people who were playing a video gameusing Near-Infrared Spectroscopy (NIR). Kanda et al. [8] quantitatively evaluatedbehaviors of people who were in communication with Robovie using the motion
capture system and eye mark recorder. According to these studies, we employ thequantitative method using a motion capture system and eye mark recorder.

3 Hypotheses about Appearance and Behavior
Mori [9] mentioned the relationship between familiarity and similarity of robot appearance and motion to humans. Familiarity of a robot increases with its similarity of
appearance and motion until a certain point, when a subtle imperfection of the appearance and motion becomes repulsive (Fig. 1). This sudden drop is called an “uncanny valley.” In the figure, appearance and motion are evaluated on the same axis. It
is, however, not always the case that they are evaluated in the same manner.
We hypothesize that robot’s appearance and behavior independently influence
human-robot interaction. Namely, an identical behavior can differently influence if
the appearances are different. With respect to robot’s appearance, our hypothesis is
the following:− The evaluation of interaction increases with similarity of robot’s appearance. At
the point of closely resemblance to humans, there is a valley like “uncanny valley”
as shown in Fig. 2 (a). The depth of the valley decreases with complexity of robot’s behavior.

Fig. 2. There are two factors that influence interaction: behavior and appearance. (a) An
evaluation of interaction plotted against similarity of appearance. There is uncanny valley.
(b) An evaluation plotted against complexity of behavior. There is a peak means synergy
effect of appearance and behavior. (c) Synthesized evaluation. There are two features: “uncanny valley” and “synergy hill.” Actually, each variable cannot be represented in one axis.

Goetz et al. [10] proposed the “matching hypothesis” that the appearance and social behavior of a robot should match the seriousness of the task and situation andexamined it in a psychological experiment with the Nursebot robot, “Pearl.” The
result suggests that human-like behavior does not always make a good impression
and that the robot’s appearance determines what behavior is appropriate. We hypothesize that there is a synergy effect of a robot’s appearance and behavior.
− The evaluation increases with the complexity of the robot’s behavior. At the pointof matching robot’s appearance, there is a synergy effect of appearance and behavior shown as a peak in Fig. 2 (b).
Synthesizing two hypotheses, the evaluation of interaction is qualitatively represented as Fig. 2 (c). The robot’s uncanny appearance is mitigated by its behavior if
the behavior closely resembles that of humans.

Fig. 3. Hypothesis about complexity of behavior. Subtle emotional behaviors including facial
expressions are human-like behaviors.

Fig. 4. Hypothesis about person’s age.

Many factors influence the complexity of behavior. One of them is emotion. In
general, one feels frustration in communicating with a person who keeps a straight
face. It seems uncontroversial to assume that emotional behavior including facial
expressions is human-like behavior. Fig. 3 shows the hypothesis about the complexityof behavior. A simple motion empty of meaning is less complexity, and a subtle emotional gesture and facial expression have high complexity and similar to human behavior.
In the above, we focus on the attributes of a robot. The attributes of a person (e.g.,age and gender) interacting with a robot influence the interaction. To compare a subject’s reaction at different ages, a couple of infants less than 13-months old and preschool children from three to five years old directed toward the developed android.
As a result, infants seemed to be attracted by the android. However, children wereafraid of the android at a glance and unwilling to face it. The behavior of children is
explained in terms of Mori’s “uncanny valley.” The result suggests that the uncanny
valley seems to change owing to person’s age. With respect to person’s age, we hypothesize as follows:− The uncanny valley becomes the deepest in early childhood and shallower in adulthood.
− A synergy hill (see Fig. 2) becomes the steepest at younger children and smootherat adults.
Fig. 4 illustrates the hypothesis. We will next form hypothesis about other attributes.

Fig. 5. The developed android robot named “Repliee R1.” Left: External appearance. Upper
Right: Head appearance. Lower right: Head appearance with eyes closed.

Fig. 6. The skeleton of the android.

4 The Developed Android Robot
Fig. 5 shows the android robot named “Repliee R1” that is developed as a prototype. To make the appearance closely resemble humans, we made a mold of a girl,and we carefully chose a kind of silicon that would make the skin feel human-like.
The appearance is a five-year-old Japanese girl. The prototype has nine DOFs in thehead (five for the eyes, one for the mouth and three for the neck) and many free joints
to make a posture. The actuators (motors) are all embedded inside the body.
The touch sensor used in the android is a strain rate force sensor. The mechanism
is similar to human touch insofar as it detects touch strength while the skin is deforming. The android has four touch sensors under the skin of the left arm (Fig. 7). Only
four sensors can measure the touch strength all over the surface of the left arm. Thesetactile sensors enable various touch communications.
The android shown above is developed as a prototype. In the future, we will implement an android with the same number of joints as humans, tactile sensors covering the whole body, vision sensors, and auditory sensors.

Fig. 7. Skin, inside body, and tactile sensors. Space between skin and skeleton is filled withurethane foam, which can be replaced with other mechanism.

5 Preliminary Experiment
5.1 Study of Gaze Behavior
For a quantitative evaluation of interaction, we investigated eye motion of people
during a conversation with the android. An evaluation of semi-unconscious behavior
such as eye motion can reveal facts that do not appear in qualitative evaluations, suchas a questionnaire test. We predicted that gaze behavior would vary owing to thesimilarity of a robot’s appearance and the complexity of its behavior during communication. To test the prediction, three types of actors (interlocutors) were prepared:(A1) a human girl, (A2) the android with eye, mouth, and neck motions, (A3) the still
android. The girl was a five-year-old Japanese girl and was not shy with strangers.
Subjects were 18 Japanese undergraduate and graduate students. There were 10 males
and 8 females. A subject had a brief conversation with each actor in random orderwith replacements except excluding. To control the conversation, we designed thefollowing script.
Conversation Script (an English translation)
Actor: Hi, I’m [name].
Subject: [answers]
Actor: Let’s play together! I’ll give you a quiz. Are you ready?
Actor: What is a word starting with [any alphabetic character]?
Subject: [answers]
Actor: That’s right! Well, what is a word starting with [any alphabetic character]?
Subject: [answers]
Actor: No! Well, then, what is a word starting with [any alphabetic character]?
Subject: [answers]
Actor: That’s right! That was fun! Bye-bye!
This is only a sample script. The order of the robot’s positive and negative responses may differ. The conversation was held in a small room partitioned by a curtain (Fig. 8.) The experimenter behind the curtain controlled reactions of the android.
A speaker produced the prerecorded voice of the android. A2 moved its mouth whileFig. 8. Experimental room. A subject mounting an eye mark recorder has a brief conversation
with the android (Left) and the girl (Right.)

talking and sometimes blinked and moved its neck, but A3 was stationary even when
it was talking.
An eye mark recorder (NAC EMR-8) measured the eye motion with the rate of 30
Hz. We defined a gaze fixation as a gaze fixed for more than four frames (133 msec)
and counted the frequency that the subject fixated on the actor’s eyes (including glabella), nose and mouth in each conversation. At the end of the experiment, the subject
answered an open questionnaire about his or her impression of the actor.
5.2 Results
Fifty-four conversations (18 subjects × 3 actors) in total were held. Ten data wereomitted owing to much detection error. Table 1 shows the mean frequencies of thesubjects’ fixation falling on the actors’ eyes, nose, and mouth. A one way ANOVA
showed that there was a significant difference (F =3.32, p < 0.05) between actors
with respect to the frequency of fixation falling on the eyes and no significant difference with respect to the nose and mouth. Furthermore, a t-test showed that there weresignificant differences between A1 and A2 (t = 3.10, p < 0.005) and A1 and A3 (t =2.45, p < 0.05) with respect to eyes. Fig. 9 shows the distributions of fixation points
that fell on the face of the android and girl. Brighter points indicate high frequency offixation.
The result shows that subjects look at the android’s eyes more frequently thangirl’s, although Japanese people tend to avoid eye contact owing to cultural reasons[11]. The subject’s mental state may explain differences in gaze behavior [12, 13].
One possibility is to assume that the subjects tried to achieve mutual understanding.
Table 1. Mean frequencies of fixation per second (standard deviations in parentheses).

Eyes
Nose
Mouth

A1
0.30 (0.059)
0.085 (0.013)
0.0014 (3.2 × 10-6)

A2
0.92 (0.57)
0.15 (0.016)
0.0029 (1.3 × 10-5)

A3
0.82 (0.52)
0.13 (0.016)
0.0017 (4.7 × 10-6)

Fig. 9. Distribution of fixation point fell on the girl (Left), android A2 (Middle) and android
A3 (Right). Brighter point means high frequency of fixation.

Many subjects felt artificiality of the android’s eye movement rather than mouth
movement. It was found from the result that how subject’s gaze at the android, especially at its eyes, differs from that at humans. This result is important, because it ispossible that the difference in the effect of the robot’s appearance and behavior onhuman-robot communication is evaluated by measuring the participant’s gaze point aswell as the subjects’ gaze point in human-human communication. A human gaze is asemi-unconscious behavior and reflects a hidden factor which cannot be self-reported.
It is expected that measuring gaze behavior would find an effect of a robot’s appearance and behavior that would not appear in the answers to a questionnaire.
5.3 Discussion
We predicted that there was a difference in the subjects’ gaze behavior between A2
and A3. Contrary to our prediction, there was no significant difference. The result
showed that the random eye lids and neck motion and mouth motion synchronizedwith voice did not influence the conversation with the android. It is considered thatthe experiment was lacking in some assumptions. This section discusses further hypotheses about the android’s appearance and behavior from the observations of gazebehavior and answers to the questionnaire.
Uncanny valley
Many subjects mentioned that artificiality of the android’s appearance, behavior
and imbalance between appearance and behavior on the questionnaire. The artificiality of eye motion in particular may cause an increase in the number of fixations on the
android’s eyes. Furthermore, the high frequency of fixation could represent the uncanny valley shown in Fig. 2. To examine this prediction, it is necessary to ascertain
whether subjects provide fewer fixations on a robot that has robotic appearance, such
as ASIMO. We hypothesize that the frequency of fixation represents the evaluation ofcommunication, and the evaluation varies inversely with the frequency.
Eye contact
Some subjects mentioned that they could not make eye contact with the android. It
is considered that the lack of eye contact causes the uncanniness. Some psychologicalresearchers show that eye contact can serve a variety of functions (e.g., [12, 13]) inhuman-human communication. It is estimated that eye contact and the android’s appearance work synergistically to enhance communication. To ascertain this, we willcompare with a robot that has a robotic appearance and no eye contact behavior.
Contingent motion
One subject answered that the android with motion (A2) was more uncanny than
the still android (A3) because the motion was not contingent. Another subject mentioned that repeating same behavior of the android was unnatural. It is possible thatthe lack of the contingent android’s motion (A2) made no difference between A2 andA3 in the result. As described in section 2, a contingent motion of nonhuman object
varies an infant’s attitude [4]. It is estimated that a contingent motion of the android
provides an effect that works in synergy with its human-like appearance.
Involuntary waving motion
One subject mentioned that it was uncanny that the android (A2) was moving only
the head though human interlocutor (A1) was always moving the whole body slightly.
Miyashita and Ishiguro [14] showed that the slight involuntary waving motion of ahumanoid robot makes its behavior more natural. It is quite likely that a slight involuntary waving motion of the whole body seems animate living. To state that the involuntary motion provides a synergy effect, however, it is necessary to compare theandroid and other robots.
Habituation effect
All the subjects in the experiment were only those who saw the android for the
first time. In other words, they were not familiar with the android yet; therefore, thehabituation effect cannot be ignored. Some subject answered that they were surprisedat the android in the first conversation but familiar with it in the second conversation.
All of their gaze behavior showed that the frequency with which fixation fell on the
android’s eyes in the second conversation decreased from that of the first conversation. Habituation to the android seems to change the interaction. In section 3, we
hypothesized that person’s age changes the human-robot interaction. We must, however, investigate the short-term (order of minutes or hours) change of interaction.

6 Conclusion
This paper has proposed a new research direction based on the android robot to reveal a principle of human-robot interaction. An evaluation of this interaction is impeded by the difficulty of isolating the effect of behavior from that of appearance.
The appearance of the android, however, may decrease the effect of robot appearance.
Furthermore, this research gives a methodology for robot design, which had previously been entrusted to an artistic designer.
This paper has shown the fundamental hypotheses about the effects of robot behavior and appearance on human-robot interaction and the preliminary experiments toobserve human reactions to the android. We are still in the progress of forming moredetailed hypotheses and designing experiments.

References
1. Kanda, T., Ishiguro, H., Ono, T., Imai, M., Mase, K.: Development and Evaluation of anInteractive Robot “Robovie”, IEEE International Conference on Robotics and Automation(2002) 1848-1855
2. Scheeff, M., Pinto, J., Rahardja, K., Snibbe, S., Tow, R.: Experiences with Sparky, a Social
Robot. Workshop on Interactive Robot Entertainment (2000)
3. Ishiguro, H., Ono, T., Imai, M., Kanda, T., Nakatsu, R.: Robovie: an Interactive Humanoid
Robot. International Journal of Industrial Robot, Vol. 28, No. 6 (2001) 498-503
4. Johnson, S.C., Slaughter, V., Carey, S.: Whose Gaze Will Infants Follow? Features that
Elicit Gaze Following in 12-month-olds, Developmental Science, Vol. 1 (1998) 233-238
5. McBreen, H.M., Jack, M.A.: Evaluating Humanoid Synthetic Agents in E-Retail Applications, IEEE Transactions on Systems, Man and Cybernetics-Part A: Systems and Humans,
Vol. 31, No. 5 (2001) 394-405
6. Kanda, T., Ishiguro, Ishida, T.: Psychological Analysis on Human-Robot Interaction, IEEE
International Conference on Robotics and Automation (2001) 4166-4173
7. Matsuda, G., Hiraki, K.: Frontal Deactivation in Video Game Players, Annual Conference of International Simulation and Gaming Association (2003) 799-808
8. Kanda, T., Ishiguro, H., Imai, M., Ono, T.: Body Movement Analysis of Human-Robot
Interaction, International Joint Conference on Artificial Intelligence (2003) 177-182
9. Mori, M.: The Buddha in the Robot, Charles E. Tuttle Co., Japan (1982)
10. Goetz, J., Kiesler, S., Powers, A.: Matching Robot Appearance and Behavior to Tasks to
Improve Human-Robot Cooperation, IEEE Workshop on Robot and Human Interactive
Communication (2003)
11. Axtell, R.E. (Ed.): Do’s and Taboos Around the World, John Wiley & Sons, New York(1993)
12. Argyle, M.: The Psychology of Interpersonal Behaviour, Penguin Books, London (1967)
13. Argyle, M., Cook, M.: Gaze and Mutual Gaze, Cambridge University Press, London (1976)
14. Miyashita, T., Ishiguro, H.: Natural Behavior Generation for Humanoid Robots, IEEE
International Conference on Humanoid Robots (2003)