An Approach to Contact Force Vector Determination
and Its Implementation to Provide Intelligent
Tactile Interaction with Environment
Dzmitry Tsetserukou, Naoki Kawakami, and Susumu Tachi
Department of Information Physics and Computing, University of Tokyo,
7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8656 Japan
dima_teterukov@ipc.i.u-tokyo.ac.jp,{kawakami, tachi}@star.t.u-tokyo.ac.jp

Abstract. The force vector information is essential while collision avoidanceplanning through tactile feedback in autonomous or teleoperation mode. In thispaper, we concentrate on the approach to calculation of the applied force vector
at any point of the robot arm and its technical realization. Based on the presented force vector calculation flow, we obtained the experimental results validating the ability of the proposed method to trace the contact point position andto estimate force vector robustly and accurately.
Keywords: Human-Robot interaction, sensitive robot arm.

1 Introduction
Recently, the new generation of robots called “service robots” has been advancing
rapidly and it is considered that in near future they will be introduced in human daily
life environment to assist with hospital care, surgery operations, high-risk maintenance tasks, construction works, and office affairs. Along with voice and image
recognition, the robot tactile ability becomes salient link in the chain of multimodal
interaction with humans. Moreover, the robot sensor system enabling detection of themechanical contact with object in unstructured dynamic environment is needed, because the occlusion in video and proximity sensor system can happen.
Conventional approaches to controlling the physical interaction between a manipulator and environment are based on impedance control of a robot arm according toapplied force vector measured at the manipulator wrist [1]. However, the rest parts of
the robot body (forearm, elbow, upper arm, shoulder, and torso) are presenting thesignificant danger not only for human being, but also for the robot structure itself. The
ability to sense the physical contact can be attained by covering the robot surface withtactile skin [2].
With tactile sensing technology continuously evolving, different types of tactile
sensors such as piezoelectric-polymer-film-based [2], flexible-polyester-sheet-based,
optical [3], etc. have been designed. Such sensors can only detect the contact area andcoarsely measure the spatial distribution of pressures perpendicular to a sensory area.
Therefore, they are unable to supply contact force orientation information. The most
M. Ferre (Ed.): EuroHaptics 2008, LNCS 5024, pp. 151–156, 2008.
© Springer-Verlag Berlin Heidelberg 2008

152

D. Tsetserukou, N. Kawakami, and S. Tachi

appropriate application of such tactile sensors is recognition of contact area gravity
center and contact image pattern.
To realize safe physical contact of entire robot arm structure with human and toguarantee the collision avoidance during teleoperation, our primary idea is concentrated on design of a whole-sensitive robot arm (by using artificial skin anddistributed torque sensors in each joint). When contact with environment occurs,
manipulator automatically generates compliant motion according to the measured
external force vector. The newly developed anthropomorphic manipulator having 4DOF arm and 8-DOF hand (Fig. 1(c)) is capable to safely interact with environment
wherever contact occurs on the robot arm surface. An operator sitting in cockpit (Fig.
1(a)) controls remote robot arm through manipulation of master robot arm (Fig. 1(b)).

a) Operator cockpit.

b) Master arm.

Position
Orientation
Velocity

Tactilesensors

Opticaltorquesensorsc) Developed whole-sensitive robot arm.

Fig. 1. Robot teleoperation system

The paper focuses on approach to determination of the applied force vector at eachpoint of the robot arm. This information is essential to plan collision avoidance trajectory, structure the environment, and communicate with humans intelligently. The
proposed method takes advantages of the tactile sensor, enabling to detect coordinatesof the contact point, and the distributed joint torque sensors to measure exerted
torques and force vector direction on the entire robot arm surface (including joints).
This method originates from human tactile system. Very fine special resolution of thehuman skin allows performing tactile discrimination of surface texture by fingertips.
However, it has moderate ability to estimate the exerted force, the job that muscles
do. They generate torque at the skeletal joints balancing such external forces as gravity, mechanical constraints of joints, inertial forces, and contact forces. Each muscle
produces a torque at a joint that is the product of its contractile force and its moment
arm at that joint [4]. Hence, by estimation of the tension of the corresponding muscles, we can define the acting force as well as the force direction.
The remainder of the paper is structured as follows. The sensory system and approach to contact point detection are presented in Section 2. The equations supporting analytical basis for the contact force vector calculations and experimental results

An Approach to Contact Force Vector Determination153

are given in Section 3. In Section 4, we briefly conclude the paper and discuss thefuture work.

2 Sensory System of the Developed Robot Arm
The robot arm is covered with Kinotex tactile sensor measuring the pressure intensity
through amount of backscattered light by photodetector [5]. The sensitivity, resolution, and dynamic range of this artificial skin are comparable to those of a human. The
taxels displaced with 21.5 mm in X and 22 mm in Y direction make up 6x10 array.
In order to recognize the contact region, we employed the watershed algorithm, animage processing segmentation technique that splits an image into areas based on thetopology of the image [6]. The adopted algorithm is processed as follows: departingfrom the local maximum, the new taxel is included into the contact area only if within
the 3x3 array surrounding it there is an other taxel of greater or equal pressure intensity already included into the water stream.
The accurate estimation of the contact point can be obtained by computing the center of gravity of the contact pattern c(xc,yc) of the neighborhood Ω by:⎛ ∑ xi f i ( xi , yi ) ∑ yi fi ( xi , yi ) ⎞⎜ x , y∈Ω⎟,x , y∈Ωc ( xc , yc ) = ⎜,f i ( xi , yi )∑ fi ( xi , yi ) ⎟⎟⎜ x∑
x , y∈Ω⎝ , y∈Ω⎠(1)

where fi(xi,yi) is the pressure intensity level of the taxel i with coordinates (xi,yi) [7].
Calculation findings of image gravity center are listed in Table 1.
Table 1. Calculation results of image gravity centeryc

xc
Digital image Visual imageY,mmPressure intensityfi,j1570114

190

135113261303755

56859

463

553

796

04375

106

91
X,mm6xi,jfi,j6yi,jfi,jxc

yc

482985 607541

94.4

118.8

In order to facilitate the realization of torque measurement in each arm joint, wedeveloped new optical torque sensors having high reliability, high accuracy (even inelectrically noisy environment), easy mounting procedure, low price and compact
sizes [8]. The novelty of our method is application of the ultra-small size photointerrupter (PI) RPI-121 as sensitive element to measure relative motion of sensor components. The spring components were manufactured from one piece of AISI 4135 steel
using wire electrical discharge machining. The optical torque sensor is set between

154D. Tsetserukou, N. Kawakami, and S. Tachi

the driving shaft of the harmonic transmission and driven shaft of the joint (Fig. 2(a)).
When the load is applied to the robot joint, the magnitude of the output signal fromthe PI corresponds to the exerted load.
8PIOutput voltage U, VShield

2T1Spring

Harmonicdrive

Calibration dataBest fit line765432a) Optical torque sensor-4-20Torque T, Nm24b) Calibration results

Fig. 2. Torque sensor of the elbow joint and calibration results

The spring members attached to the first, second, and third/forth joints were designed to measure torque of ± 12.5 Nm, ± 10.5 Nm, ± 4.5 Nm, and have resolution of
10.77 mNm, 9.02 mNm, 4.31 mNm, respectively. Each sensor was calibrated bymeans of attachment of reference weights to the lever arm. Calibration results for theelbow joins detector are given in Fig. 2 (b). Non-linearity of 2.5 % of Full Scale wascalculated using maximum deviated value from the best-fit line.

3 Contact Force Vector Calculation Algorithm
In this section, we consider the problem of computing the force vector at any contact
point on entire teleoperated robot arm based on the contact point coordinates andoutput signal from the torque sensors. TheZ0+
coordinate systems fixed to the joints andYaw
Y0connected to the arbitrary contact points are+
X0 +
Pitch
shown in Fig. 3. The forces ifi acting in theRoll
X2
Z1
J2J1coordinate system of each joint produce
Y1Xmoments ini (Eq. 2). The joint torques are
1Z 2 Y2
L F2
derived by taking Z component of the moments
x 2 J3applied to the link (Eq. (4)).
L2
F2z2 y2

Y4
Y3

Z4

F4z4
Y5
Z5
X5

ni = i +1i R i +1ni +1 + i PCi × i Fi + i Pi +1 × i +1i R i +1 fi +1 (2)

J4X3

X4iZ3
y4i

L F4
L4x4

Fig. 3. Coordinate system

f i = i +1i R i +1 fi +1 + i Fi(3)τ i = i niT i Zˆi ,(4)where i+1iR is the matrix of rotation between
links i+1 and i calculated using DenavitHartenberg notation; iPi+1, iPCi are the vectors

An Approach to Contact Force Vector Determination

155

locating the origin of the coordinate system i+1 in the system i, and contact point,respectively; iFi is the contact force.
Let us consider the case when external force is applied to the upper arm of the robot. Using the Eq. (2), (3) we have:2⎡ 0 ⎤ ⎡ F2 x ⎤ ⎡ LF 2 F2 z ⎤⎡ s2⎢⎥⎢⎥⎢⎥1n2 = ⎢ LF 2 ⎥ × ⎢ F2 y ⎥ = ⎢ 0 ⎥ ; n1 = ⎢⎢ 0⎢⎣ 0 ⎥⎦ ⎢⎣ F2 z ⎥⎦ ⎢⎣ − LF 2 F2 x ⎥⎦⎢⎣ c2c20− s20⎤ ⎡ LF 2 F2 z ⎤ ⎡ LF 2 F2 z s2 ⎤1 ⎥⎥ ⎢⎢ 0 ⎥⎥ = ⎢⎢ − LF 2 F2 x ⎥⎥ . (5)0⎥⎦ ⎢⎣ − LF 2 F2 x ⎥⎦ ⎢⎣ LF 2 F2 z c2 ⎥⎦The components of the applied force vector and force magnitude are derived from:F2 x = τ 2 ( − LF 2 ) ; F2 z = τ 1 ( LF 2 c2 ) ; F2 =( F2 x ) + ( F2 z ) .
22(6)The value of the applied force projection on the axis Y is not required because theupper arm cannot move in this direction. For the case when the force acts on the forearm we have the following algorithm to obtain the contact force vector:
F4 x = τ 4 LF 4 ; F4 z = τ 3 ( LF 4 s4 ) ;F4 y = (τ 2 + F4 z ( LF 4 c3c4 + L2 c3 ) + F4 x ( LF 4 s3 + L2 s3c4 ) ) ( L2 s3 s4 ) ,(7)where c2, c3, c4, s3, and s4 are abbreviations for cos(θ2), cos(θ3), cos(θ4), sin(θ3), andsin(θ4), respectively; L2, LF2, and LF4 are the lengths of the moment arms.
Contact point location y, mm

1.5
1.00.5
0.0-0.5
Applied torque W
Applied torque W

-1.0-1.5

Contact force components F2, Na) 04

81216Time t, sec

128
40-4
F2F2zF2x-8-120b)

48Time t, sec

1216200
18016014012010080c)
Contact force components Fz2 , N

Applied torque WEXT, Nm

2.0120140160180200220

Contact point location x, mm

86420-2-4-6-8-12-10d)

Contact force components Fx2 , N-8-6-4

Fig. 4. Torque sensor of the elbow joint and elbow joint assembly

-20

156

D. Tsetserukou, N. Kawakami, and S. Tachi

To evaluate the feasibility of the proposed approach, the experiments were conducted for calculation of the contact force vector acting on the upper arm.
During the experiment the object was moved along the upper arm surface according to the curve shown in Fig. 4(c). While contacting, the torques values measured at
the first and second joint (τ1, τ2) varies in accordance with Fig. 4(a). The contact force
vector projections (F2x, F2z) and force vector length F2 obtained from Eq. (6) are represented in Fig. 4(b). The trajectory of the end point of the contact force vector isshown in Fig. 4(c).

4 Discussion of Experimental Results and Future WorkThe experimental results show that the developed algorithm and implemented sensory
system of the robot arm allow measuring the force direction at any point of the robot
arm accurately and robustly. For further enhancement of the contact point positionestimation, the tactile sensor with higher spatial resolution is required.
In the future research we are going to employ the information on force vector andcontact point to plan the compliant collision avoidance trajectory autonomously andprovide tactile feedback to the operator through electro-stimulating device. We consider that presenting the information about direction and intensity of the contact force
vector can greatly improve the collision avoidance strategy realized by operator, incomparison with the conventional contact area tactile feedback. On the next stage wewill also develop such kind of control that allows rounding the contacting surface byusing force vector value and pressure distribution at the contact area.
Acknowledgments. The research is supported in part by a Japan Society for the Promotion of Science (JSPS) Postdoctoral Fellowship for Foreign Scholars.

References
1. Mitsunaga, N., Miyashita, T., Ishiguro, H., Kogure, K., Hagita, N.: Robovie IV: A Communication Robot Interacting with People Daily in an Office. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5066–5072. IEEE Press, San Diego (2006)
2. Caccavael, F., Natale, C., Siciliano, B., Villani, L.: Six-DOF Impedance Control Based on
Angle/Axis Representation. IEEE Trans. on Robotics and Automation. 15(2), 289–300(1999)
3. Um, D., Stankovic, B., Giles, K., Hammond, T., Lumelsky, V.: A Modularized Sensitive
Skin for Motion Planning in Uncertain Environments. In: IEEE International Conference on
Robotics and Automation, pp. 7–12. IEEE Press, Leuven (1998)
4. Kandel, E.R., Schwartz, J.H., Jessell, T.M.: Principles of Neural Science. McGraw-Hill,
New York (2000)
5. Optic Fiber Tactile Sensor Kinotex, Nitta Corporation, http://www.nitta.co.jp/english/product/mechasen/sensor/kinotex_top.html
6. Serra, J.: Image Analysis and Mathematical Morphology. Academic Press, London (1982)
7. Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Addison-Wesley, Reading (1992)
8. Tsetserukou, D., Tadakuma, R., Kajimoto, H., Tachi, S.: Optical Torque Sensors for Implementation of Local Impedance Control of the Arm of Humanoid Robot. In: IEEE Int.
Conf. on Robotics and Automation, pp. 1674–1679. IEEE Press, Orlando (2006)