Geminoid- Tele-operated Android of an Existent Person Hiroshi ISHIGURO1, 21) ATR Intelligent Robotics and Communications Laboratories
2) Department of Adaptive Machine Systems, Osaka University
E-mail: ishiguro @ams.eng.osaka-u.ac.jp

Abstract
Androids are testbets for studying human-robot interaction.
They enable us to compare the developed robot technologies
and humans in a direct manner. Especially, they can be usedfor verifying cognitive and psychological hypotheses. And
then the verified hypotheses are applied for improving theandroids. This tight connection between robotics andcognitive science is called android science.
The developed androids so far, however, are not enoughfor various cognitive tests. Especially, they lack abilities forlong-term conversation. In order to compensate this problem,
we need to consider high-bride systems in which a human andthe android are integrated. The integrated system is calledgeminoid. Geminoid is a new category of robot. It is a teleoperated android of an existent person. By using the geminod,
we can tackle to the unsolved problems and extend theframework of android science.

Appearance and behavior of interactive robots
Why are we attracted in humanoids and androids? The
answer is simple. It is because of our tendency toanthropomorphize non-human things. We, humans, always
anthropomorphize targets of communication and interaction.
Therefore, we expect much with humanoids. In other words,
we find a human itself in the humanoid. Recently, researchers’ interests in robotics are shifting from traditional
studies on navigation and manipulation to interaction withthe robots.
The human-robot interaction study has been neglecting anissue. That is “appearance v.s. behavior problem.” Theinteractive robots that have been developed thus far are nonandroid types. Evidently, the appearance of the robot
influences the impressions of the subjects, and it is a very
important factor in the evaluation of the interaction (SeeFigure 1). There are many technical reports that compare
robots with different behaviors; however, thus far, theappearance of the robots has not been focused upon.
Although there are many empirical discussions on very
simplified static robots such as dolls, the design of a robot’sappearance, particularly to make it appear a humanoid, hasalways been a role of industrial designers. This is a serious
problem for developing and evaluating interactive robots.
The appearance and the behavior are tightly coupled, andthe results of the evaluation change with the appearance.

Figures 1: Humanoid and Android. The left picture shows ahumanoid Eveliee P1 in Osaka University based on
WAKAMARU that was developed by Mitsubishi Heavy Industry
Co. Ltd.; and the right one shows an android Repliee Q2developed by cooperation with KOKORO Co. Ltd. in OsakaUniversity.

Android Science that bridges science andengineering
One of the methods to tackle to the neglected issue is todevelop a very humanlike robot, i.e., an android, and use itfor studying human-robot interaction. The right figure ofFigures 1 shows the developed android. The android has 42
air actuators for the upper torso, excluding fingers. In thedevelopment, we have determined the positions of theactuators by analyzing the movements of a real human byusing a precise 3D motion capture system. The actuators canrepresent the unconscious movements of the chest due tobreathing in addition to conscious large movements of thehead and arms. Furthermore, the android has a function forgenerating facial expression that is important forinteractions with humans. Place acknowledgments(including funding information) in a section at the end of thepaper.
The development of the androids requires contributions
from both Robotics and Cognitive Science. In order torealize a more humanlike android, knowledge from humanscience is necessary. This new framework is called android
science [1, 2]. Thus, android science is an interdisciplinary
framework between engineering and cognitive science.
Robotics attempts to build very humanlike robots based on
the knowledge from cognitive science. Cognitive scienceFigure 3: Tele-operated android
Figure 2: Android science

employs the robot for verifying hypotheses forunderstanding humans (See Figure 2).
In the past, robotics research used knowledge from
cognitive science, while research in cognitive scienceutilized robots. However, the contribution from robotics tocognitive science has not been adequate; since appearance
and behavior could not be separately handled, non-android
type robots were not sufficient as tools of cognitive science.
We expect that this problem can be solved by using an
android that has an appearance very close to a human. On
the other hand, robotics research based on the cues fromknowledge in cognitive science faces a similar problem
since it is difficult to clearly recognize whether the cues
pertain solely to robot behaviors, isolated from theirappearance, or the combination of its appearance andbehaviors. In the framework of android science, androids
enable us to directly share knowledge between thedevelopment of androids in engineering and theunderstanding of humans in cognitive science.
Let us summarize major research issues in android
science here. The issues in Robotics are as follows• Development of the very humanlike appearance withsilicon• Development of the humanlike movements• Development of the humanlike perception byintegrating with ubiquitous sensor systems.
On the other hand, the issue in Cognitive Science is“conscious and unconscious recognition.” The goal of
android science is to realize a humanlike robot and find theessential factors for representing human likeness. How can
we define human likeness? Further, how do we perceive
human likeness? It is well known that a human has
conscious and unconscious recognition. When we observe
objects, various modules are activated in our brain. Each ofthem matches the input sensory data with the humanmodels; and then they affect on reactions. A typical example
is that even if we recognize a robot as an android, we react
to it as a human. This issue is fundamental both for theengineering and scientific approaches. It will be anevaluation criterion in the development of the android, andit provides us cues for understanding the human brain
mechanism of recognition.

Bottleneck of robotics
The developed android shown in Figure 1 enables us
various cognitive tests, however it is still limited. The
bottleneck is long-term interaction by talking. Unfortunately,

the AI technology for developing humanlike brains still
limited and we cannot expect humanlike conversation withrobots.
A method for solving this problem is to develop a hybrid
system using tele-operation techniques. If the operator talks
through the Internet, the android can have long-term
conversation with people.
When we see humanoids, we usually expect to havehumanlike conversation with them. However, thetechnology is very behind of the expectation. The
development of the technology takes time and it is rather
our final goal in robotics. In order to arrive at the final goal,
we need to use possible technologies and understand deeply
on what a human is. Our solution for this problem is tointegrate the android and tele-operation technologies asshown in Figure 3.
This approach also develops practical application ofandroids. If we access to the android locating in a distantplace, we do not need to go their. By using it, we can exist
at the distant place.

Studies on existence of the person
For compensating the bottleneck, we have developed ageminoid. Geminoid is a new category of robot. It is a teleoperated android of an existent person. Figures 3 show
myself and the geminoid.
After developing this prototype system, we haveencountered several interesting phenomena as follows.
• When I saw the static geminoid, it was like a mirror.
However, when it naturally moved, it was not myself
although we have copied my movement. This means we
do not objectively recognize our unconscious
movement by ourselves.
• While I operate the geminoid with the interface system,
I unconsciously adapt my movements to the geminoid’smovements.
• Both of me and the visitor can quickly, less than 5minutes, adapt to the conversation through the geminoid.
The visitor recognizes the geminoid as me while talking
each other. This is string entrainment, especially forpeople who know me well.
• When the visitor tough to the Geminod, I get a feelingto be toughed. The system does not provide sensor
feedbacks. But, by looking at monitor and interactingwith the visitor, I get the feeling.
By using the gemioid, we can have long-term
conversation for explaining these phenomena and extend theframework of android science. In the previous works inandroid science, we focused on human likeness. In addition• What is invariant of my existence between them?
These questions are related to a simpler question
whether my conscious and my body can be separate. By
using the geminoid, we can tackle to these issues and deeply
and systematically understand humans.
Our current challenges are as follows:
Tele-operation technologies for complex humanlike
robots: It is necessary to study the method to tele-operate
the geminoid in order to convey existence/presence, whichis quite different with traditional tele-operation for mobile
robots and industrial robots in the complexity. We are
studying a method to autonomously control an android bytransferring motions of the operator measured by a motion
capturing system. We are, also, developing methods toautonomously control eye-gaze and humanlike small andlarge movements.
Synchronization between speech utterances sent by
the tele-operation system and body movements: The
most important technology for the tele-operation system issynchronization between speech utterances and lip
movements. We are investigating how to produce natural
behaviors during speech utterances. This problem isextended to other modalities, such as head and arm
movements. Further, we are studying effects on non-verbal
communication by investigating not only synchronization of
speech and lip movements, but also facial expressions, headand even the whole body movements.
Psychological test for human existence/presence, so
called Sonzai-Kan in Japanese: We are studying the effectof transmitting Sonzai-Kan from a remote place, such as
participating to a meeting instead of the person himself.
Moreover, our interest is to study what existence/presence isthrough cognitive and psychological experiments. For
example, we are studying whether the android can represent
authority of the person himself by comparing the person
himself and the android.

Conclusion
Our purpose to develop the geminoid is to study on SonzaiKan by extending the framework of android science. The
scientific aspect is to answer the questions how humans
recognize human existence/presence. The technologicalaspect is to realize a tele-operated android that works onbehalf of the person accessing from a distant place. This
will be one of the practical networked robots that is realizedby integrating the Internet and robots.

References

Figure 3: Myself, the geminoid, and the tele-operation system.
The geminoid has been developed in ATR Intelligent Robotics
and Communications Laboratries.

to that, we can study on existence of the person. Thequestions are• What is the difference between my existence recognized
by myself and my existence recognized by others?

[1] Hiroshi Ishiguro, Interactive Humanoids and Androids
as Ideal Interfaces for Humans, Proc. Int. Conf. Intelligent
User Interfaces, 2006. See also www.ed.ams.eng.osakau.ac.jp/research/Android/android_eng.htm.
[2] Tim Hornyak, Android Science, Scientific American,May 2006.