ICROS-SICE International Joint Conference 2009
August 18-21, 2009, Fukuoka International Congress Center, Japan

Teleoperation System with Haptic Feedback for Physical Interaction
with Remote Environment
Dzmitry Tsetserukou1, Katsunari Sato1, Naoki Kawakami1, Susumu Tachi2
1

The University of Tokyo, 2Keio University, Tokyo, Japan(Tel : +81-3-5841-6917; E-mail: dima_teterukov@ipc.i.u-tokyo.ac.jp, Katsunari_Sato@ ipc.i.u-tokyo.ac.jp,
kawakami@star.t.u-tokyo.ac.jp, tachi@tachilab.org)
Abstract: The paper focuses on the teleoperation system TeleTA with haptic feedback designed to realize haptic
communication with remote human, to achieve high level of maneuverability of robot arm in unstructured environment,
and to increase effectiveness of physical Human-Robot interaction. The human operator directs the robot arm to thetarget position, while manipulator sensory system provides safe physical interaction with environment. In order toachieve awareness of collision, to support operator with valuable information (stiffness and shape of contacting object),
and to realize robot-mediated haptic communication, we developed the innovative tactile and haptic displays, namely,
BraTact (for presenting the cutaneous information) and FlexTorque (for presenting the kinaethetic feedback).
Keywords: Teleoperation, haptic display, tactile display.

1. INTRODUCTION
Humans can perform dexterous manipulations inremote place by means of teleoperation. The
teleoperation is also intended for realization complex
cooperation tasks with human beings (physicalcollaborative interaction) and haptic communication.
Remote robot must ensure stable contacting with object
in case of collision and safe physical interaction. The
additional important information about objects (shape,
stiffness, location, fixation, etc.) can be gathered while
contacting. The presentation of obtained knowledge
about environment to a human operator in real time canconsiderably improve the performance of teleoperation.
The delivering force feedback in the case of collisionand during physical interaction is absolutely necessary.
However, typical teleoperation system provides force
feedback only to human hand [1]. Moreover, sensory
system of slave robot does not provide safe interaction
along entire structure. Additionally, master arm
integrates motors of high power. In the case of
malfunction it can harm operator.
The paper focuses on the teleoperation system
TeleTA (Teleoperetion system with wearable Tactileand haptic display on the operator arm surface providing
Awareness about contact state, contacting object
properties, and interactive forces between slave robot
and environment) designed by us to achieve high levelof maneuverability of robot arm and to perform
cooperative tasks with humans in a safe manner.
Distributed optical joint torque sensors and local
admittance controllers endow our robot arm with thedistinctive capability of safe interaction withsurroundings along entire manipulator surface(including joints). The applied force vector is calculated
from the values of joint torques and contact point
coordinates [2].
Master side (Fig. 1(a)) includes exoskeleton robot
arm with 6 DOF (Fig. 1(b)), tactile display BraTact topresent cutaneous stimulation when collision with
object is detected, and haptic display FlexTorque topresent forces during physical collaborative tasks andPosition,orientation
velocity
Force,
shape,
stiffnessa) Operator
cockpit.
Haptic displayb) Master arm.

- 3353 -Tactiledisplay

Opticaltorquesensors
Tactileskinc) Remote robot iSoRA.

Fig. 1 Robot teleoperation system TeleTA.
haptic communication. The orientation of the humanelbow is controlled by the tilt sensor.
Developed teleoperated robot iSoRA has 4-DOF:
Roll, Pitch, Yaw joints of a shoulder, and Pitch joint of
an elbow. Each joint is equipped with optical torquesensor directly connected to the output shaft ofharmonic drive and magnetic encoders.
Proposed sensory system of slave robot (tactile skin
to detect contact point and torque sensors distributed
into each joint for measurement of applied force)patterns on the human tactile system. Our sense of touch
can be separated into kinesthetic and coetaneous.
Kinesthetic stimulations, produced by forces exerted onbody, are sensed by mechanoreceptors in the joints,
tendons, and muscles enabling us to estimate forces
being applied to body [3]. On the contrary,mechanoreceptors in the skin layers are responsible forcutaneous stimulation sensation enabling stimuli
localization.
The robot arm is covered with Kinotex tactile sensor[4]. In order to recognize the contact region, we
employed the watershed algorithm. However, applied
force sensitivity of Kinotex is very low due to highnon-linearity of the output, and limited sensing range.
The task of load measurement is accomplished by thePR0002/09/0000-3353 ¥400 © 2009 SICEdeveloped optical torque sensors enabling the torquemeasurement with high accuracy [5]. The spring
members attached to the first, second, and third/fourth
joints were designed to measure torque of r 12.5 Nm, r
10.5 Nm, r 4.5 Nm, and have resolution of 10.77 mNm,
9.02 mNm, and 4.31 mNm, respectively.
In order to realize realistic haptic interaction (holding,pushing and contacting the object) with environmentand haptic communication with human beings (e.g.
handshaking), the force feedback is required. The aim ofour research is to develop wearable tactile and haptic
display to present realistic feedback (cutaneous andkinesthetic stimulus) to the operator’s arm.
In the remainder of the paper we describe thedevelopment of wearable device FlexTorque aimed atforce feedback, and ergonomic tactile display BraTact
for presentation of contact location and physicalproperties of contacting object.

2. DESIGN OF HAPTIC DISPLAY FlexTorque
Most of the force feedback master devices are similarin sizes to slave robot and are equipped with powerfulactuators. Such systems pose dangerousness for humanoperator and in case of failure during bilateral control
can harm human. In the last years there have been
several attempts to make the force feedback devices
more compact, safe, and wearable.
In [6], an exoskeleton-type master device wasdesigned based on the kinematic analysis of human arm.
Pneumatic actuators generate torque feedback. The
authors succeeded in making the lightweight andcompact force reflecting master arm. However, theforce-reflection capability of this device is not enoughto present contact forces effectively. Moreover,pneumatic actuators have large time delay, complicatedcontrol, and limited workspace.
An artificial pneumatic muscle-type actuator was
proposed [7]. Wearable robotic arm with 7 DOF andhigh joint torques was developed. Robotic arm uses
parallel mechanisms at the shoulder part and at wrist
part similarly to the muscular structure of human upper
limb. It should be noted, however, that dynamic
characteristics of such pneumatic actuator possess
strong nonlinearity and load-dependency, and, thus, anumber of problems need to be resolved for its
successful application. The large weight of 4 kg andcumbersome structure significantly degrade the initialconcept of human-friendly and wearable design.
In order to achieve human-friendly and wearable
design of haptic display, we analyzed the amount of
torque to be presented to the operator arm. Generally,
there are three cases when torque feedback is needed.
The first case takes place when haptic communication
with remote human needs to be realized. For example,
the person handshakes the slave robot and joint torques
are presented to the operator. Such interaction results invery small torque magnitude (in the range of 0-1.5 Nm).
The second situation takes place when slave robot
transports heavy object. Here, the torque values are- 3354 -much higher than in previous case and torque magnitudedepends on the load weight. However, continuouspresentation of high torques to the operator will result inhuman muscle fatigue. We argue that downscaled torqueindicating direction of the force would be informative
enough. The third and the worst case of contact state in
term of interactive force magnitude is collision. It is
obvious that high force will be produced when robot
collides with moving object. However, the result ofcollision with fixed object (as it is often the case) isimmediate discontinuation of the operator’s arm motion.
Therefore, the power of torque display must be enoughto only fixate the operator arm. For the case of collision
with movable obstacle, the haptic display should induce
human’s arm motion in the direction of the impact force,
decreasing thus the possible damages.
The idea behind novel torque display FlexTorque is toreproduce human muscle structure, that allows us toperform dexterous manipulation and safe interaction
with environment in daily life. Main functions of themuscles are contraction for locomotion and skeletal
movement. A muscle generally attaches to the skeleton
at both ends. Origin is the muscle attachment point to
the more stationary bone. The other muscle attachment
point to the bone that moves as the muscle contracts is
Insertion. Muscle is connected to the periosteum
through tendon (connective tissue in the shape of strap
or band). The muscle with tendon in series acts like arope pulling on a lever when pulling tendons to movethe skeleton (Fig. 2).
Text=Fext u dext
OriginTflex=Fflex u dflex

MuscleFext

Fflex
Tload=Fload u dload
dext

dflex

TendonInsertion

Fload

dload
Tnet= Tflex - Text

Fig. 2 Structure and action of a skeletal muscle.
When we hold a heavy object in a palm, its weight
produces torques in the wrist, elbow, and shoulder joint.
Each muscle generates a torque at a joint that is the
product of its contractile force and its moment arm at thatjoint to balance gravity force, as well as inertial forces,
and contact forces. Thus, we can feel object weight.
Because muscles pull but cannot push, hinge joints(e.g. elbow) require at least two muscles pulling in
opposite direction (antagonistic muscles) [3]. The torqueproduced by each muscle at a joint is the product ofcontractile force (F) and moment arm at that joint (d).
The net torque Tnet is the sum of the torques produces byeach antagonistic muscle. Movement of human limbs is
produced by coordinated work of muscles acting onskeletal joints. The structure of the developed torquedisplay FlexTorque is presented in Fig. 3.

Let us consider the calculation procedure of the net
torque value. The layout of the forces and torques
applied to the forearm during flexion is given in Fig. 5.

Motor holder

Tm
Ft

DC Motor

dt
Belt

lD

Ftx
df

Upper
arm

Fty

Tn

Belt fixator

Fig. 5 Diagram of applied forces and torques.

Forearm

The tension force Ft of the belt can be derived from:
Ft

Fig. 3 FlexTorque on the human’s arm surface.

(1)

where Tm is the motor torque, i is the gear ratio, and r is
the shaft radius.
The net torque Tn acting at the elbow joint is:

FlexTorque is made up of two DC motors (muscles)
fixedly mounted into plastic Motor holder unit, Belts(tendons), and two Belt fixators. The operation principleof the haptic interface is as follows. When DC motor isactivated, it pulls the belt and produces force Fflex
generating the flexor torque Tflex. The oppositely placed
DC motor generates the extensor torque Text. Therefore,
the couple of antagonistic actuators produce a net torqueat operator elbow joint Tnet. We defined the position of
the Insertion point to be near to the wrist joint in orderto develop large torque at the elbow joint.
The position of the operator’s arm, when flexor
torque is generated, is shown in Fig. 4 (where T stands
for angle of forearm rotation in relation to upper arm).
Directionof belt
tension

Tmi ,
r

Tn

Fty d fFt d f cos D ,(2)where df is the moment arm.
The angle D varies according to the relative positionof the forearm and upper arm. It can be found using thefollowing equation:§ l 2  d 2f  d t2 ·cos 1 ¨
¸¸ ,
¨
2
ld
f
©
¹

D

(3)

where dt is the distance from the pivot to the Origin; l is
the length of belt, it can be calculated from the rotation
angle of the motor shaft. The detailed view of the
FlexTorque is presented in Fig. 6.

Direction of
motor shaft
rotation

DC Motor
Motor holder
Stopper

Pulleys
Timing belt

T = 10q

T = 55q

T = 110q

Supporter
Shaft

StopperFig. 4 Positions of the human’s arm under flexor torque.
Fig. 6 3D exploded view of driving unit of FlexTorque.

- 3355 -Each unit is compact and light in weight (60 grams).
This was achieved due to the use of plastic andduralumin materials in manufacturing the main
components. The Supporter surface has concave profile
to match the curvature of human arm surface (Fig. 7).

Fig. 7 Driving unit of FlexTorque.
In the case of collision, the limb must be at rest. In
such a case, the net torque produced by the muscles is
opposed by another equal but opposite torque Tload. The
vibration of the human arm (e.g. simulation of driving
the heavy truck) can be modeled through alternate
repeatable jerks of torque of antagonistic motors. Thus,
operator can perceive the roughness of road surface.
The FlextTorque enables the creation of muscle
stiffness. By contracting belts before the perturbation
occur we can increase the joint stiffness. For example,
during collision of remote robot with moving object thetension of the belt of one driving units drops abruptly
and the tension of the belt pulling the forearm in the
direction of the impact force increases quickly.
The essential advantage of the structure of
FlexTorque device is that heaviest elements (DC motors,
shafts, and pulleys) are located on the part of upper arm,
which is nearest to the shoulder. Therefore, operator’s
arm undergoes very small additional loading. The rest ofcomponents (belts, belt fixators) are light in weight anddo not load the operator’s muscles considerably. We
propose to use term “Karate Haptics” to such kind of
novel devices because they allow presenting the forces
to the human arm without using additional interfaces in
the human hands.
The developed apparatus features extremely safe
force presentation to the operator’s arm. While
overloading, the belt is physically disconnected from the
motor and the safety of the human operator is
guaranteed. The FlexTorque also provides gravity
compensation to simplify the teleoperation tasks.

arranged in regular grid pattern. It should be mentioned
here, that heavy tactile display placed on the arm
surface degrades the mobility and increases joint muscle
loading. Therefore, our objective was to find out theapproach to effective presentation of tactile information.
Chen et al. [10] examined the human ability tolocalize a single vibration source on dorsal and volarsides of the forearm near wrist. An important finding
was that on average only 4 tactor locations could becorrectly identified on both sides of the wrist.
The developed tactile bracelet BraTact (Braceletwith Tactors) incorporates six vibration motors withholders linked by elastic band. To enhance thelocalization ability, we arrange the tactors in a zigzag
pattern (Fig. 8).
Vibration motor
HolderCover

Lugs
Dummy tactor
Wiring holeFig. 8 3D model of tactile display.
The inner surface of the holder has concave profile tomatch the curvature of human arm surface. The
designed tactile bracelet is shown in Fig. 9.

Fig. 9 Tactile display BraTact.

3. DEVELOPMENT OF THE ERGONOMICTACTILE DISPLAY BraTact

The small flat coreless vibration motors FM34F withdiameter of 12 mm and thickness of 3.4 mm produce
tactile stimulation on human skin.

When it comes to the contact presentation on thehuman arm, tactile displays are preferable toforce-feedback devices. They can convey contact cues[8], direction, and distance information. Recently, haptic
gadgets displaying information on a large area of thehuman body are gaining increased attention byresearchers. In [9], reconfigurable, wearable system
TactaVest for delivering vibro-tactile stimuli ispresented. It can be seen from the papers presented
above that, typically, the huge amount of actuators are

4. USER STUDY METHODOLOGYApparatus.
The developed tactile display BraTact was used topresent the shape, and stiffness of object. The smallest
participant’s forearm circumference was 21 cm, so the
distance between tactors always was 3.5 cm or more, i.e.
larger than a single receptive field (about 2.5 cm).

- 3356 -Participants.
A total of 5 males and 1 female with no previous
knowledge about experiment were examined. Their age
varied from 23 to 31. None of the subjects reported any
sensory difficulties.
Procedure and Stimuli.
Object shape presentation. Object shapeinformation is important for operator. While colliding,the objects with smooth shape attenuate the impact
forces. Thus, knowledge of object shape can improve
performance of motion planning in clutteredenvironment. The intuitive approach of representation of
simple object shapes (circle (Pattern A), rectangle (B),triangle (C)) and shape primitives (arc (D), line (E),point (F)) is presented in Fig. 10. The dashed ovals
depict the active tactors.

Circle (A)Rectangle (B)Triangle (C)Arc (D)Line (E)

Point (F)Level 2 (Rubbersponge)Level 1(Sponge)

Level 4 (Wood)

Level 3 (Rubber)

Level 5 (Metal)

Fig 11. Tactile presentation of the object stiffness.

Fig. 10 Tactile presentation of the object shape.
Dynamic Shapes Experiment was aimed at presenting
the shapes in a more transparent manner. The duration
time of tactor activation for all shapes was the same - 2
sec. For round, rectangular and triangular shape, thetactors were activated one by one in counter-clockwise
direction. This allows the human to draw in his mind thepath represented by tactor. The arc was presented bysimultaneous activation of all three tactors, but with
time-varying intensities with duration of 2 sec.
Moreover, intensity level in the middle tactor was larger
than in neighboring ones. To draw the line, we tried toreproduce the apparent motion illusion. For that, twotactors were activated simultaneously but with opposite
time-varying intensities levels. That is, when the first
tactor had the maximum vibration intensity the opposite
tactor had the lowest intensity and vice versa. The point
was presented by activation of single tactor with ramp
signal.
Stiffness presentation. In Static Stiffness Experiment
all tactors were activated simultaneously with the same
intensity level of 2 sec. duration. Five vibration intensity
levels (Level 1 (167 Hz) (Pattern A), Level 2 (188 Hz)(B), Level 3 (208 Hz) (C), Level 4 (229 Hz) (D), Level
5 (250 Hz) (E)) representing object stiffness from very
low rate to very high were chosen. The darkness level ofgray color in Fig. 11 represents the vibration amplitudelevel.

- 3357 -In Dynamic Stiffness Experiment all tactors wereactivated simultaneously for given level but with
time-varying intensity. The time-dependent pattern waschosen to represent the contact point motion duringcontact transient (when we collide with stiff object ourarm immediately stops moving forward). The stiffest
material, aluminum, was presented as short ramp
impulse with duration of 0.3 sec. The less stiff material
the longer duration and smaller amplitude of impulse
were presented to human’s forearm skin. To mimic
nonlinearity of the sponge material, the sine wave
pattern was presented with duration of 2 sec.
The experiment procedure is as follows. To mask
auditory cues of the tactor vibration, subjects wore
headphones producing pink noise of 65 dBA. They wereasked to sit down at the table and grasp vertical stick at
the edge of the table to establish static reference
position. Subjects were informed that the experiment
aimed at testing their ability to discriminate between
various patterns. All the participants were given 18 trials
practice sessions before experiment for object
presentation and 15 trials for stiffness presentation.
In total, 30 stimuli (6 patterns were repeated 5 times
in a random order) were presented during object shapediscrimination experiment and 25 stimuli (5 patterns
were repeated 5 times in a random order) for object
stiffness recognition experiment. After each stimulus,
the subject marked the table cell corresponding to thepattern had been detected.

5. RESULTS AND DISCUSSION
The results of user study are listed in Table 1 - Table 4.
Table 1 Group mean percentage of recognition of shapein dynamics.
Group Mean
Percentage, %Actual Pattern
A
B
C
D
E
F

Subject Response
A
100.0
0
0
3.3
3.3
0

B
0
83.3
3.3
0
3.3
3.3

C
0
13.3
96.7
0
3.3
0

D
0
0
0
73.3
3.3
0

E
0
0
0
10.0
86.7
3.3

F
0
3.3
0
13.3
0
93.3

Table 2 Group mean percentage of recognition of shapein static condition.
Group Mean
Percentage, %Actual Pattern
A
B
C
D
E
F

presentation of weight and stiffness of object to the user.
In the future we will conduct thorough user study of thedeveloped haptic display FlexTorque.
The tactile display BraTact supporting thepresentation of tactile information by means of thebracelet with tactors was developed. The results of theuser study revealed that the Dynamic mode ofpresentation of the object properties was more intuitive,
and, therefore, resulted in very high level of
discrimination accuracy. The future step is elaborationof the concept of “Dynamic tactile drawing”, whengraphical information is presented by tactile patterns
generated discretely in time.

Subject Response
A
60.0
33.3
10.0
0
0
0

B
26.7
26.7
33.3
3.3
10.0
0

C
6.7
26.7
43.3
6.7
20.0
0

D
3.3
3.3
0
76.7
16.7
0

E
3.3
10.0
13.3
10.0
53.3
3.3

F
0
0
0
3.3
0
96.7

Table 3 Group mean percentage of recognition of
stiffness in dynamics.
Group Mean
Percentage, %Actual Pattern
A
B
C
D
E

Subject Response
A
100.0
0
0
0
0

B
0
80.0
16.7
0
0

C
0
13.3
73.3
20.0
0

D
0
6.7
10.0
70.0
3.3

E
0
0
0
10.0
96.7

Acknowledgments. The research is supported in partby a Japan Society for the Promotion of Science (JSPS).

REFERENCES

Table 4 Group mean percentage of recognition of
stiffness in static condition.
Group Mean
Percentage, %Actual Pattern
A
B
C
D
E

Subject Response
A
73.3
13.3
0
0
0

B
20.0
50.0
16.7
0
0

C
6.7
16.7
50.0
23.3
6.7

D
0
20.0
20.0
53.3
23.3

E
0
0
13.3
23.3
70.0

The ANOVA results revealed that it was significantly
easier for participants to recognize stiffness of materials
B (p=0.001<0.05) and E (p=0.04<0.05) during Dynamic
Condition than in Static Condition. Shape patterns A, B,
and C were significantly easier to recognize during
Dynamic Condition than in Static Condition(p=0.006<0.05, p=0.002<0.05, and p=0.007<0.05,
correspondingly). The important point is that therecognition rate of the line presented by apparent
motion illusion was much more higher than that of line
presented in Static mode. Based on this evidence, we
can use this effect to draw more complex shapes.

6. CONCLUSIONS AND FUTURE WORKThe increasing complexity of tasks performed byteleoperation requires to deliver and to display tactile
information to the human operator effectively in orderto ensure safe and efficient interaction with environment.
The sensory system of the developed slave robot
supports the local admittance controllers generatingcompliant motion with information on exerted force.
We developed a novel haptic interface FlexTorquethat enables realistic physical interaction with Real(throughteleoperation
system)and
VirtualEnvironments. The main features are: (1) it does not
restrict the motion of the human arm, (2) it has wearable
design, and (3) it is extremely safe in operation. This
haptic display can find the future applications also in
Virtual Reality systems for realization of realistic
physical interaction, haptic communication, and- 3358 -[1] B. Stanczyk and M. Buss, “Development of aTelerobotic System for Exploration of Hazardous
Environment”, in Proc. IEEE/RSJ Int. Conf. onIntelligent Robots and Systems, Sendai, 2004, pp.
2532-2537.
[2] D. Tsetserukou, N. Kawakami, and S. Tachi, “AnApproach to Contact Force Vector Determination and its
Implementation to Provide Intelligent Tactile Interaction
with Environment”, in Proc. Int. Conf. EuroHaptics,
Madrid, 2008, pp. 151-156.
[3] E.R. Kandel, J.H. Schwartz, and T.M. Jessell.
Principles of Neural Science. New York: McGraw-Hill,
2000.
[4] Optic fiber tactile sensor Kinotex, Nitta Corporation.
http://www.nitta.co.jp/[5] D. Tsetserukou, R. Tadakuma, H. Kajimoto, and S.
Tachi, “Optical Torque Sensors for Local ImpedanceControl Realization of an Anthropomorphic Robot Arm”,
Journal of Robotics and Mechatronics, Vol. 18, No. 2,
pp.121-130, April 2006.
[6] S. Lee, S. Park, W. Kim, C-W. Lee, “Design of aForce Reflecting Master Arm and Master Hand using
Pneumatic Actuators”, in Proc. IEEE Int. Conf. on
Robotics and Automation, Leuven, 1998, pp.
2574-2579.
[7] Y. Jeong, Y. Lee, K. Kim, Y-S. Hong, and J-O Park,“A 7 DOF Wearable Robotic Arm Using Pneumatic
Actuators”, in Proc. 32nd Int. Symp. on Robotics, Seoul,
2001, pp. 388-393.
[8] R.D. Howe, D.A. Kontarinis, and W.J. Peine, “ShapeMemory Alloy Actuator Controller Design for Tactile
Displays”, in Proc. 34th IEEE Conf. on Decision andControl, 1995, pp. 3540-3544.
[9] R.W. Lindeman, Y. Yanagida, H. Noma, and K.
Hosaka, “Wearable Vibrotactile System for Virtual
Contact and Information Display”, Virtual Reality, Vol.
9, pp. 203-213, 2006.
[10] H-Y. Chen, J. Santos, M. Graves, K. Kim, and H.Z.
Tan, “Tactor Localization at the Wrist”, in Proc. Int.
Conf. EuroHaptics, Madrid, 2008, pp. 209-218.