Haptic Telexistence
Katsunari Sato*

Kouta Minamizawa†

Naoki Kawakami‡

Susumu Tachi*†

The University of Tokyo
1

Executive Summary“Haptic Telexistence” provides highly realistic haptic interaction
among humans and objects located in remote places. We havedeveloped innovative devices and constructed a master-slave
system to realize “Haptic Telexistence”. Human interaction will
be dramatically improved by this concept that perceives us theproperties of an object.

2

Project Overview and Vision

Nowadays, we can interact with humans or objects even if they
are located in remote places or in virtual environments. In theseinteractions, we can watch, listen, touch, and move objects.
However, the properties of an object are not present inconventional systems. When we communicate or perform a task, alack of haptic sensation reduces the realism and interactivity.
Therefore, there is increasing requirement for haptic technology
presently.
“Haptic Telexistence” aims at achieving highly realistic haptic
interaction among humans and objects located in remote places.
In order to realize this, a slave hand with sufficient mobility and amaster hand that can operate the slave hand easily are required. Inaddition, haptic sensors and displays are needed to perceive thehaptic information correctly.
We developed innovative devices and constructed a robot handmaster-slave system (Figure 1). Using conventional systems, wecan only perceive the stiffness of an object. However, we can also
perceive the exact shape of an object by using our system. Further,more natural and dexterous manipulations of an object become
possible. This means that complex tasks employing various tools
will be simplified. In addition, we can use the master hand as aninterface to the virtual world. Therefore, applications such as
telesurgery systems and 3D-modeling systems can be realized.
Our ultimate goal is to present all the haptic sensations. Further,our system is capable of presenting more object properties such as
texture and temperature. After the ultimate system is realized, we
expect a dramatic improvement in the life of human’s. For
example, not only will we be able to shake hands with people atremote locations but we will be able to feel the warmth of theirhands. In the case of internet shopping, we will be able to checkthe texture of an article before purchase.
We hope that the participants understand the enormous
capabilities of haptic technologies from their experience with oursystem for “Haptic Telexistence”.

Figure 1: Master-slave system for “Haptic Telexistence.”

3

Technical Innovations

Our system consists of four innovative devices, namely, a
dexterous slave hand, finger-shaped haptic sensor for the slave
hand, exoskeleton encounter-type master hand, and electrotactile
display (Figure 2). Each of these devices have more advantages
than the corresponding conventional ones. In addition, theintegrating them in order to realize “Haptic Telexistence” is also atechnical innovation.

Slave System

Master System

Dexterous
Slave Hand

Exoskeleton
Encounter-type
Slave hand

RemoteObject

Operator
Finger-shaped
Haptic Sensor: Operational Information

Electrotactile
Display: Haptic Information

Figure 2: System configuration.

3.1 Dexterous Slave Hand*

email: Katsunari_Sato@ipc.i.u-tokyo.ac.jp
email: Kouta_Minamizawa@ipc.i.u-tokyo.ac.jp‡
email: kawakami@star.t.u-tokyo.ac.jp*†
email: tachi@star.t.u-tokyo.ac.jp†Natural tele-interaction with remote places is possible only when
the slave hand has high mobility and acquires haptic information
correctly. Therefore, it is desirable that the salve hand with ahaptic sensor has a human-like structure.

Our multi-fingered slave hand has following futures [Hoshino andKawabuchi 2005]. This hand has a total of 15 degrees of freedom(DOF) — five DOF for the thumb, one for abduction of otherfingers, three for the index finger, and two for the remaining
fingers. Each fingertip has independent DOF and the index finger
and the thumb can be countered. Therefore, a pinching operation
by a fingertip is possible. For example, this slave hand can pinch a
card (Figure 3).
(a)3.3 Exoskeleton Encounter-type Master Hand
To operate the slave hand and to present force feedback, weconstructed a multi-fingered master hand (Figure 5). This hand
has two features [Nakagawara et al. 2005]. One is a compact
exoskeleton mechanism called “circuitous joint,” which covers awide workspace of an operator’s finger. Another is theencounter-type force feedback. This is realized using the photoreflector and force sensor. They measure the distance between tipof the master finger and that of the operator and the force between
them. Using this information, the fingers of this hand is controlled
to contact operator’s finger only when the slave hand touches anobject. This avoids unnecessary contact sensation and enables
unconstrained motion of the operator’s fingers.

Non-contact
(b)

(c)

photo
reflector &reflecting plate

Contact
Figure 3: (a) Appearance of the dexterous slave hand.
(b, c) Pinching a card by the fingertips.

force sensor

3.2 Finger-shaped Haptic Sensor
We developed a finger-shaped haptic sensor using GelForcetechnology. GelForce is a haptic sensor that measures thedistribution of both the magnitude and direction of force[Kamiyama et al. 2004]. This sensor comprises a transparent
elastic body, two layers of blue and red markers, and a CCD
camera (Figure 4). Because of this simple structure, we were able
to minimize the size and realize a high resolution. Now, thefinger-shaped GelForce sensor achieves a space resolution of 4.5mm and a time resolution of 14 ms. These performances can befurther improved by developing the camera.

LED lighting ofmarkers
imaging substrate

3.4 Electrotactile Display
Haptic sensation must be presented on the basis of thecharacteristics of human perception. It can be divided into twotypes, namely, a kinesthetic (force) sensation and tactile sensation.
Therefore, we mounted an electrotactile display [Kajimoto et al.
2004] on the master hand to present both the haptic sensations(Figure 6).

17 mm

master hand

16 mmwide-angle lensFigure 5: (Left) Exoskeleton encounter-type master hand.
(Right) Encounter-type force feedback.

elasticbady

12 mmelectrotactile display

20 mm
Figure 4: (Left) Structure of the finger-shaped GelForce.
(Right) Finger-shaped GelForce mounted on the fingertip of thedexterous slave hand shown in Figure 3.

Figure 6: (Left) Electrodes of the electrotactile display.
(Right) Master hand equipped with the electrotactile display.

The electrotactile display is a tactile device that directly activates
nerve fibers within the skin surface by an electrical current fromthe surface electrodes. It can selectively stimulate each type ofreceptor and produce vibratory and pressure sensations with anarbitrary frequency.

3.5 Realization of “Haptic Telexistence”
By using the above mentioned devices, we constructed amaster-slave system for “Haptic Telexistence”.
The master-slave manipulation is realized by bilateral impedancecontrol of the dexterous slave hand and the encounter-type masterhand. This control is done from position of the master and slave
fingers and force applied to them. The position is calculated fromthe angles of each finger joints. The force is measured by thefinger-shaped GelForce on the slave hand and the force sensor on
the master hand. The refresh rate of the control is 1 kHz.
Therefore, we can operate it smoothly and perceive sufficient
kinesthetic sensation.
When the slave hand touches an object, the finger-shaped
GelForce mounted on the slave hand acquires haptic informationsuch as the distribution of the magnitude and direction of force.
Then, this information is transmitted to the master system. The
electrotactile display presents us with tactile sensation based on
this information. Using the force distribution, the electrode stimuli
present information on the location. Subsequently, the force
magnitude information at each position is presented by thestrength of electrostimulus. As a result, we can feel a field, edge,
peak, and movement of an object.

4Context

The necessity of tele-operation technologies originated due tonuclear systems. In the 1940s, the Algona National Laboratory
constructed the first master-slave system, which was called“Master-Slave Manipulation.” This technology spread across theworld because of a desire to extend a person’s sensing andmanipulation capability to remote places.
In the 1980s, an advanced concept of tele-operation originated;
telexistence was proposed by Tachi [Tachi and Abe 1982] andtelepresence by Minsky [Minsky 1980]. These concepts arenamed after the technology that enables operators controlling theapplication to have a real-time sensation of being at a place otherthan their actual location.
To realize these concepts, a number of types of haptic and robotic
devices have been developed [Benali-Khoudja et al. 2002; Bouzitet al. 2002; Lee and Nichols 1999]. Using these devices, thetele-operation systems have been realized [Kwon et al. 1999;
Methil et al. 2006]. However, these systems are intended to beapplied to only specific tasks and can only present limited hapticinformation.
We researched both haptic and robotic devices in order to realize“Haptic Telexistence.” For realizing a slave system, weconstructed a slave hand that has multi-freedom five fingers. We
have also constructed a haptic sensor for the slave hand. First, wedeveloped GelForce that measure the force vector distribution on
the surface. We showed it at SIGGRAPH2004 [Kamiyama et al.
2004] (Figure 8. b). By improving this sensor, we realized thefinger-shaped haptic sensor for the slave hand.

By integrating these kinesthetic and tactile sensations, we canperceive the exact shape and stiffness of an object (Figure 7). This
enables highly realistic interactions with objects in remote places.
(a) SmartTouch(SIGGRAPH2003)(b) GelForce(SIGGRAPH2004)(c) Forhead Retina System(SIGGRAPH2006)

Figure 7: Sensing and presentation of haptic information. Whenthe slave hand touches an edge of a dice, the operator feels astrong force feedback (red arrows) through the master hand.
Simultaneously, operator feels linear electrical tactile sensation
based on the force distribution (yellow arrows) measured by thefinger-shaped GelForce.

Figure 8: (a, c) Applications using the electrotactile display.
(b) GelForceSimultaneously, we developed a master system. We constructed
the master hand that provides an operator with natural contact
sensation and operational feeling. In addition, we developed anelectrotactile display to present the natural tactile sensation. As
the result of the improvement in the electrotactile display, we
showed its applications in the following conferences; SmartTouchin SIGGRAPH2003 [Kajimoto et al. 2003] (Figure 8. a) andForehead Retina System in SIGGRAPH2006 [Kajimoto et al.
2006] (Figure 8. c). In this study, we mounted this electrotactile
display on the master hand and realized highly realistic haptic
display.
As a result, practical applications of these devices have become
possible. We integrated them and constructed a master-slave
system for “Haptic Telexistence.” In this system, there are two
features that conventional systems don’t have. First, the masterand slave hands enable us unconstrained operation. In addition,
we can perceive an exact touch sensation of an object by thehaptic sensor and display. The “Haptic Telexistence” system
realizes the remote interaction to have a real-time sensation ofbeing at a place.

5User Experience

BUTTERFASS, B., GREBENSTEIN, M., LIEU, H., and HIRZINGER, G.
2001. DLR-Hand Ⅱ: Next Generation of a Dextrous Robot
Hand, In Proceedings of IEEE International Conference on
Robotics and Automation, Seoul, May, 109–114.
HOSHINO, K. and KAWABUCHI, Y. 2005. Pinching at finger tips forhumanoid robot hand, Journal of Robotics and Mechatronics,
17, 6, 655–663.
KAJIMOTO, H., INAMI, M., KAWAKAMI, N., and TACHI, S. 2003.
SmartTouch: A New Skin Layer to Touch the Non-Touchable,
In Proceedings of ACM SIGGRAPH 2003, San Diego.
KAJIMOTO, H., KAWAKAMI, N., MAEDA T., and TACHI, S. 2004.
Electro-Tactile Display with Tactile Primary Color Approach,
In Proceedings of International Conference. on Intelligent
Robots and Systems(IROS).
KAJIMOTO, H., KANNO, Y., and TACHI, S. 2003. Forehead Retin
System, In Proceedings of ACM SIGGRAPH 2003, Boston.
KAMIYAMA, K., KAJIMOTO, H., KAWAKAMI, N., and TACHI, S.
2004. Evaluation of a Vision-based Tactile Sensor, In
Proceedings of International. Conference on Robotics andAutomation (ICRA2004).

In order to experience “Haptic Telexistence”, we will construct a
robotic hand master-slave system, as mentioned in the above
sections. A participant wears the master hand and controls theslave hand. When the slave hand touches an object, a participant
can sense highly realistic haptic sensations. Through the naturalinteractions with remote objects, the participants can feel likehe/she is present in a remote place. In this system, only one
participant can experience “Haptic Telexistence.” The experience
time is approximately 10 min.

KAMIYAMA, K., MIZOTA, T., VLACK, K. KAJIMOTO, H., KAWAKAMI
N., and TACHI, S. 2004. GelForce – A Vision Based TactileSensor, In Proceedings of ACM SIGGRAPH 2004, LosAngeles.

In addition, we will construct other master-slave systems for
multiple participants to experience “Haptic Telexistence.” Bytaking advantage of the small size of the electrotactile display andGelForce, we will construct two-fingered simple master-slavesystem. We can construct two or more sets of this system. The
experience time of these systems is approximately 5 min.
Therefore, we expect that many participants will be able toexperience our proposed “Haptic Telexistence” simultaneously.

LEE, M. H. and NICHOLLS, H.R 1999. Tactile Sensing for
Mechatronics: A State of the Art Survey, Mechatronics, 9,
1–31.

Credits
This work was supported by associate professor Kajimoto (TheUniv. of Electro-Communications) who developed theelectrotactile display, Dr. Kamiyama and Nitta corp. whodeveloped GelForce, and Dr. Kawabuchi (the Kawabuchi
Mechanical Engineering Laboratory, Inc.) who developed themechanism of both the slave and master hands.

KWON, D-S., WOO, K. Y., and CHO, H. S. 1999. Haptic control ofthe Master Hand Controller for a Microsurgicaltelerobot. In
Proceedings of IEEE Robotics and Automation, 3,1722–1727.

METHIL, N.S., SHEN, Y., ZHU, D., POMEROY, C.A., MUKHERJEE, R.,
Xi, N. and MUTKA, M. 2006. Development of supermedia
Interface for Telediagnostics of Breast Pathology, In
Proceedings of IEEE International Conference on Roboticsand Automation, Orlando, Florida.
MINSKY, M. 1980. Telepresence, Omni. June. 45–52.

References

NAKAGAWARA, S., KAJIMOTO, H., KAWAKAMI, N., and TACHI, S.,
Kawabuchi, Y. 2005. An Encounter-Type Multi-Fingered
Master Hand Using Circuitous Joints, In Proceedings ofIEEE International Conference on Robotics and Automation(ICRA2005), Barcelona, Spain.

BENALI-KHOUDJA M., HAFEZ M., ALEXANDRE J.M., and KHEDDAR
A. 2004. Tactile Interfaces: A State of the Art Survey, In
Proceedings of International Symposium on Robotics, March,
Paris, 23–26.

TACHI, S., and ABE, M. 1982. Study on Tele-Existence (1): Designof Visual Display. In Proceedings of the 21st Annual
Conference of the Society of Instrument and Control
Engineers (SIEC), Tokyo, Japan, July, 167–168.

BOUZIT, M., BURDEA, G., POPESCU, G., and ROIAN, R. 2002. The
Rutgers Master Ⅱ - New Design Force-Feedback Glove, In
Proceedings of. IEEE/AMSE Transactions on Mechtronics, 7,2, 256–263.