The 2010 IEEE/RSJ International Conference onIntelligent Robots and Systems
October 18-22, 2010, Taipei, Taiwan

A Tactile Sensing for Estimating the Position and Orientationof a Joint-Axis of a Linked Object
Kazuya Matsuo, Kouji Murakami, Katsuya Niwaki, Tsutomu Hasegawa, Kenji Tahara, and Ryo Kurazume
Abstract— This paper describes a tactile sensing to estimate
the position and orientation of a joint-axis of a linked object.
This tactile sensing is useful when a multi-jointed multi-fingered
robotic hand manipulates a tool which has a joint. Thisestimation requires sensing of the location of a contact point
and the direction of an edge of the tool as contact information
measured by a robotic fingertip. A conventional hard fingertip
with a force sensor can measure only the location of a contact
point. In contrast, we have already developed a robotic fingertip
with a force sensor and a soft skin, and it can measure not only
the location of a contact point but also the direction of an edgeof an object. The estimation of a joint-axis of a linked object
is demonstrated by using the soft fingertip.

I. I NTRODUCTION
A multi-jointed multi-fingered robotic hand has potential
capability of dexterous manipulation [1], [2]. A human handdemonstrates high dexterity by handling various tools. If therobotic hand can use a tool, it improves its dexterity. However, it is difficult for the robotic hand to use a tool. Because ameasurement of the relative position and orientation between
a target object, a tool, and the hand is required for handling atool. So few work is reported on the robotic hand handling atool [3]. If a tool has a joint, tool handling is more difficult.
Because the robotic hand system has to additionally consider
the relative position and orientation between the direction ofa joint-axis of the tool and the target object. For example,
when the robotic hand cuts a paper with scissors, blades areperpendicular to the paper. In other words, the direction of
the joint-axis of scissors is kept to be parallel to the paper(Fig. 1). Mutual occlusion will often occur because a target
object, a tool, and the robotic hand are very close to eachother while the hand uses the tool. Then, a vision system
will easily fail to measure their position and orientation. So,
we propose a tactile sensing of estimating the position andorientation of a joint-axis of a tool.
Katz et al. proposed a method of estimating the positionof a joint-axis of a tool through point features tracking using
a vision sensor [4]. In this method, a vision system needs tocontinuously track feature points on the surface of a tool
while a robotic hand uses the tool. However, the visionsystem will often fail to track feature points due to occlusion,lighting condition, and complex background during a tool
handling. In contrast to this, a tactile sensing can avoid
these failure factors of the vision sensing because a tactile
sensor directly contacts a target. So, we try to replace avisual feature tracking by a tactile sensing of the loci of
The authors are with Faculty of Information Science and ElectricalEngineering, Kyushu University, 744 Motooka, Nishi-Ku, Fukuoka, 8190395, JAPAN. matsuo@irvs.is.kyushu-u.ac.jp

978-1-4244-6676-4/10/$25.00 ©2010 IEEE(a) right usage.
Fig. 1.

(b) wrong usage.

Direction of the joint-axis of scissors.

contact points (Fig. 2). Katz’s method estimates the positionof a joint-axis of a tool on the assumption that visual point
features do not move on the surface of the tool. However, thecontact point often moves on the surface of the object due
to slip and rolling during a tool handling process. Therefore,
error of the estimation will increase if contact points are usedinstead of visual point features.

Fig. 2. Tactile sensing for estimating the position and orientation of a
joint-axis of a linked object.

Fig. 3.

Contact point moves on an edge of a link of an object.

We propose another approach using a geometrical constraint between the joint-axis of the tool and an edge of alink of the tool. A distance between the joint-axis and theextended line of the edge is always constant during a tool
handling process. We assume that the contact point moves
on the edge (Fig. 3). Under these constraint conditions,
we can estimate the position and orientation of the jointaxis from tactile information. A measurement of the tactile1884

information including the location of a contact point and the
direction of an edge is required for the implementation ofthis approach. A conventional hard robotic fingertip with a
six-axis force/torque sensor can measure only the locationof a contact point. In contrast, we have already proposed amethod of measuring the direction of an edge in addition to
the location of a contact point by using a robotic fingertip
with a six-axis force/torque sensor and a soft skin [5], [6].
In this paper, we propose a tactile sensing of estimating
the position and orientation of a joint-axis of a tool. The
method is robust against not only the failure factors with thevision sensing but also the movement of a contact point on
an edge.

To find the solution to these equations, substitute the values:
ai = xi + tei and a = p + su.
(xi − p + tei − su) · ei = 0(5)(xi − p + tei − su) · u = 0.
By solving simultaneous equations (5),22

u +v +w

2

s−t=

sd =

(p q r)T + s(u v w)T
p + su,

(2)

=

1,

(3)

(xi − p) · ei|ei |2(8)(9)(xi − p) · ei , td = 0.
|ei |2

a

d

u

(10)

edge at the i thmeasurement

joint-axis

p

=
=

(7)

is obtained by solving simultaneous equations (5). Then,the position and orientation of the joint-axis is estimatedfrom equations (8) by using sd and td . These sd and tdare expressed as:(1)

where p = (p q r) is the position of a point on the jointaxis, u = (u v w)T , (|u| = 1) is the orientation of the jointaxis, and s is a scalar parameter. The minimum of |ai − a|is the distance between the edge and the joint-axis, whereai = (ai bi ci )T is a point on the i th edge (Eq. (1)) anda = (a b c)T is a point on the joint-axis (Eq. (2)). When|ai −a| is the minimum value, (ai −a) ⊥ ei and (ai −a) ⊥ u
are valid.
(ai − a) · ei = 0(4)(ai − a) · u = 0.
T

=

(6)

where d is the distance between the edge and the jointaxis, sd and td are s at equation (6) and t at equation(7), respectively. Seven parameters (p = (p q r)T , u =(u v w)T , and d) are estimated by using equation (3) andequations (8) obtained through six tactile measurements.
The denominators of sd and td are equal to zero when theedge and the joint-axis are parallel to each other (ei = u).
When ei is equal to u,

where xi = (xi yi zi )T is the location of the contact point at
the i th measurement, ei = (exi eyi ezi )T , (|ei | = 1) is the
direction of the edge of the object at the i th measurement,
and t is a scalar parameter. Moreover, the joint-axis of theobject is expressed as:(x y z)Tt|ei |2 (xi − p) · u − (ei · u)(xi − p) · ei,|ei |2 |u|2 − (ei · u)(ei · u)(xi − p) · u − |u|2 (xi − p) · ei|ei |2 |u|2 − (ei · u)|xi − p + td ei − sd u| = d,

We measure the location of a contact point and the
direction of an edge of the object as contact information.
The contact information is measured by a robotic fingertip
equipped with a 6-axis force/torque sensor and a soft skin.
The contact information is measured several times during arotation of the joint of the object. Then, the position andorientation of the joint-axis is estimated from the contact
information. We assume that the object has one revolute joint
and two links.
The distance between the edge of the object and the jointaxis of the object is constant during a rotation of the revolutejoint. Therefore, we can estimate the position and orientationof the joint-axis of the object by using the measured contact
information (Fig. 4). The contact information is measured N
times during the rotation of the joint. The edge of the object
is expressed as:(x y z) = xi + tei ,=are obtained. The distance between the edge and the jointaxis is expressed as:

II. E STIMATION M ETHOD OF P OSITION AND
O RIENTATION OF J OINT-A XIS

T

s

ai

ei

xi

( u = e = 1)iFig. 4. Defined vectors to calculate the position and orientation of the jointaxis: xi is the position vector of the contact point at the i th measurement,
ei is the direction vector of the edge of the object at the i th measurement,pi is the position vector of a point on the joint-axis, u is the orientation of
the joint-axis, and d = |ai − a| is the distance between the edge and thejoint-axis.

III. M EASUREMENT OF C ONTACT I NFORMATION
We formalize a method of measuring both the location ofa contact point and the direction of an edge of an object byusing a robotic fingertip with a six-axis force/torque sensor
and a soft skin.

1885

A. Contact ModelIn the case of contact with deep penetration of an object
into the soft fingertip, the geometric property of the object
affects the transmissible friction moment at the contact area.
We formalize a contact model between a soft fingertip and a
stick-like object. This contact model is an extension to Mason
and Salisbury’s contact model [7]. When the soft fingertip
comes into contact with a stick-like object that includes,for example, a sharp wedge, a rounded wedge, or the edgeof a plate, the shape of the contact area on the fingertip
becomes heteromorphic. The heteromorphic contact with alimited strip area caused by the local geometry of an object
transmits primarily two-dimensional friction moments {qn ,qt } in addition to a three-dimensional force {pn , pt , po }.
The transmissible force and friction moment are shown in
Fig. 5. The area of oblique lines indicates the contact areaon the soft fingertip in Fig. 5. qn is parallel to the surface
normal n of the fingertip at the contact area. Here, qn isreferred to as the normal component of the friction moment.
qt is perpendicular to both the surface normal n and theorientation of the stick-like object. We call qt the tangential
component of the friction moment. Then, we refer to thiscontact as soft line contact.

friction moment q is obtained from the balance equation offorce and moment as:(11)

m = q + r × p.

(12)

When the contact is a soft line contact, q consists of qn
and qt , where qt is the tangential component of the friction
moment, which is additionally transmitted to the fingertip
due to the penetration of the stick like object into the soft
fingertip. Then, qt is obtained from the friction moment q
measured at the contact by subtracting qn , as shown in Eq.
(14).
We can measure the orientation of the stick-like object as
the outer product of n and qt . n and qt are expressed as:
n =

S(r)
,
  S(r)

qt = q − (q · n)n,

(13)
(14)

where  is the gradient operator, and S(r) is the function
describing the surface of the fingertip.

Fig. 6.

Fig. 5.

f = p,

Geometric relationship for soft line contact.

IV. E XPERIMENTS

Soft line contact.

A. Experimental Setup
When a typical hard spherical fingertip contacts an object, the contact always transmits only a three-dimensional
force pn , pt , po . Features of the soft fingertip mentioned
above provide more capability of tactile sensing than a hardfingertip.
B. Measurement of Edge Direction
Measurement of the orientation of a stick-like object isformalized as follows. The geometrical relationship for thesoft line contact is shown in Fig. 6. Let Bxyz , f , and m be
the reference frame, the measured force, and the measured
moment, respectively. Let r, n, p, and q be the contact
location on a soft fingertip, the surface normal at r, thecontact force at r, and the friction moment at r, respectively.
Each vector is expressed with respect to Bxyz . The reference
frame Bxyz is attached to the six-axis force/moment sensor
reference frame in our implementation. The force f andmoment m exerted by the fingertip are measured by the sixaxis force/moment sensor. The position r of the contact pointon the fingertip is calculated using Bicchi’s method [8]. The

The experimental setup is shown in Fig. 7. An object
composed of two-links connected by one joint is on the table
as shown in Fig. 8. The left grip of the object is fixed tothe table. The right grip can be freely rotated. The rotation
axis is estimated through the tactile sensing between the soft
fingertip and the right grip. We have developed a robotic
fingertip with a soft skin. Fig. 9 shows the inner structure
and its appearance. The fingertip is composed of an innershell and a soft skin. The inner shell is made of aluminum.
The inner shell is a cylinder with a hemisphere on its top.
Its radius is 11 mm. The soft skin is made of silicone rubber
covering the inner shell. The skin has constant thickness. The
thickness of the skin is 5 mm. The inner shell is fixed to a
six-axis force/torque sensor (BL AUTOTEC, LTD. NANO
5/4). The exerted force/moment to the fingertip is measured
by the force/torque sensor. The soft fingertip is connectedto the probe of the 3-D coordinate measuring machine(microcord C604, Mitutoyo Corp.). The relative position andorientation between the 3-D coordinate measuring machinecoordinate system and the fingertip coordinate system are1886known. Then, both the location of a contact point and the
direction of an edge measured in the fingertip coordinate
system can be expressed in the 3-D coordinate measuringmachine coordinate system.

is obtained from equation (15). By rearranging equation (16)about p, q, and d,
⎛ ⎞
p
(17)
(eyi −exi 1)⎝ q ⎠ = eyi xi − exi yidis obtained. N equations are obtained by measuring thelocation of a contact point and the direction of an edge of theobject N times. We express the N equations by rearranging
them as:⎞⎛⎞⎛ey1 −ex1 1 ⎛ p ⎞ey1 x1 − ex1 y1
⎟
⎜ ..
..
.. ⎟⎝ q ⎠ = ⎜..
⎠ . (18)
⎝ .
⎝
.
.
. ⎠
d
eyN xN − exN yN
eyN −exN 1
We express this equation as:

Fig. 7.

Ax = b,Experimental setup.

(19)

where A is a N -by-3 matrix. Equation (19) can be solved
using the pseudo-inverse matrix A+ of the matrix A as
follows:x =A+=

A+ b + (I − AT A) ξ,(20)(AT A)−1 AT ,(21)

where I is the identity matrix, ξ is an arbitrary vector. x
which brings minimum Ax−b can be calculated by using
A+ . The position of the joint-axis of the object on x-y plane(p, q) is estimated as this x.
Fig. 8.

Object composed of two-links connected by one joint.

Fig. 9.
Inner structure of a robotic fingertip with a soft skin and its
appearance.
Fig. 10. Proposed method: Estimation method of the position of the jointaxis (p, q) based on measurement of the locations of contact points (xi , xj ,xk ) and the directions of edges of an object (ei , ej , ek ).

B. Formalization for experiments
1) Proposed method: In the experiments, the robotic fingertip rotates the right grip on x-y plane (Fig. 10). Therefore,
let xi = (xi yi 0)T , p = (p q 0)T , ei = (exi eyi 0)T ,and u = (0 0 1)T be substituted in the formalization ofsection II. Then, sd = 0 and td = −(xi − p) · ei =−exi (xi − p) − eyi (yi − q) are obtained. By using thesesd and td ,
 (1 − e2xi )(xi − p) − exi eyi (yi − q) (15)
 (1 − e2yi )(yi − q) − exi eyi (xi − p)  = d
is obtained. Since ei is a unit vector,
eyi (xi − p) − exi (yi − q) = d(16)2) Simple method: For comparison, we show anotherestimation method of the position of a joint-axis of an object.
In this method, we assume that a contact point does not moveon the surface of the object during a rotation of the joint of
the object. We call this method simple method.
The distance between a contact point on the object andthe joint-axis of the object is constant during the rotation of
the revolute joint. Therefore, the position of the joint-axis
is on the perpendicular bisector of a line segment that isbounded by two contact points before and after the rotationof the joint. So, the position of the joint-axis can be estimatedas the location of the intersection point of more than one

1887perpendicular bisector (Fig. 11). The location of a contact
point is measured N times during the rotation of the joint.
The perpendicular bisector is expressed as:y=−yi + yj
xj − xi
xi + xj)+, (yi = yj ), (22)(x −yj − yi22where (xi yi ) and (xj yj ) are the locations of the contact
points at the i th and j th measurements, respectively. Byrearranging this equation about x and y,(xj − xi yj − yi )

x
y

=

1 2
(x − x2i + yj2 − yi2 )
2 j

(23)

is obtained. N C2 similar equations are obtained by measuring the location of a contact point N times. We express the
N C2 equations by rearranging them as:⎛⎞x2 − x1
y2 − y1
⎜
⎟ x
..
..
⎝
⎠ y
.
.
xN − xN −1 yN − yN −1
⎛
⎞
1
2
2
2
2
2 (x2 − x1 + y2 − y1 )
⎜
⎟
..
=⎝
(24)
⎠.
.
1
2
2
2
2
2 (xN − xN −1 + yN − yN −1 )We express this equation as:
Ax = b,(25)

where A is a N C2 -by-3 matrix. Same as section IV-B.1),
the position of the joint-axis on x-y plane (p, q) is estimatedby using the pseudo-inverse matrix A+ of the matrix A as
follows:x = A+ b,(26)3) The robotic fingertip is moved away from the right
grip. Then, the right grip is rotated counterclockwise
by human hand by 30 degrees.
4) The location of a contact point and the direction of anedge are measured three times by repeating from 1)to 3). Then, the position of the joint-axis of the object
is estimated based on the measurement of the contact
information.
At the step 2), the contact information is measured when theforce of 5 [N] is exerted for penetration of the edge into thesoft fingertip.
D. Experimental Result
We investigated the influence of the movement of thecontact point on the edge. We estimated the position of thejoint-axis of the object in two cases with different conditions.
First case is that the contact point moves on the edge of theright grip. The other case is that the contact point does not
move on the edge. We estimated the position of the joint-axis
ten times in each case. The estimation results are shown in
TABLE I. Results in the case that the contact points move areshown in Fig. 12. Results in the case that the contact points
do not move are shown in Fig. 13. The left images are resultsof the proposed method. The right images are results of thesimple method. The blue circle indicates the true value of theposition of the joint-axis (p, q). The true value of (p, q) is(17 mm, 105 mm). The red x indicates the estimated positionof the joint-axis. The green squares indicate the measured
locations of the contact points. The green lines indicate themeasured directions of the edges of the right grip.
TABLE IT HE ESTIMATION RESULTS OF THE POSITION OF THE JOINT-AXIS (p, q).

where x = (x y)T = (p q)T .

The estimation results of the position of the joint-axis (p, q) [mm]in the case thatin the case thatthe contact points movethe contact points do not moveproposed method
simple method
proposed method
simple method(21, 99)(18, 76)(23, 109)(13, 100)(24, 109)(21, 79)(19, 103)(14, 101)(20, 105)(22, 79)(23, 109)(6, 100)(25, 108)(22, 78)(25, 114)(20, 101)(21, 108)(21, 77)(22, 111)(19, 102)(17, 100)(20, 76)(21, 105)(21, 101)(22, 111)(18, 76)(27, 115)(21, 101)(20, 103)(21, 79)(19, 105)(19, 101)(20, 104)(19, 76)(26, 113)(14, 100)(24, 110)(25, 80)(19, 105)(13, 101)
The average error of the estimation results of the position of the joint-axis [mm]
6.0
27.7
7.1
5.9

Fig. 11. Simple method: Estimation method of the position of the jointaxis (p, q) based on measurement of the locations of contact points (xi , xj ,xk ).

C. Sequence of experiment
The sequence of the experiment is as follows:
1) The robotic fingertip touches the right grip of theobject.
2) The location of the contact point and the direction of
the edge of the object are measured.

The position of the joint-axis was successfully estimatedby using the proposed method in both cases. The errorof the estimation depends on the accuracy of the tactile
measurement including the location of the contact point andthe direction of the edge. Repetition of tactile measurements
will improve the accuracy of the estimation. In contrast, theestimation error of the simple method is large in the case thatthe contact points move (in Fig. 12). The movement of thecontact point causes the increase of the error in the case of
the simple method.

1888

proposed method
Fig. 12.

simple method

Estimation results in the case that the contact points move.

proposed method
Fig. 13.

simple method

Estimation results in the case that the contact points do not move.

V. C ONCLUSION
We propose a tactile sensing of estimating the position andorientation of a joint-axis of an object. This tactile sensing isbased on the measurement of both the position of a contact
point and the direction of an edge of an object by using arobotic fingertip with a six-axis force/torque sensor and asoft skin. The estimation of a joint-axis of a linked object isdemonstrated by using the soft fingertip. The tactile sensing
is robust against not only failure factors with the vision
sensing but also the movement of a contact point on thesurface of the object.
R EFERENCES[1] A. Bicchi: “Hands for Dexterous Manipulation and Robust Grasping:
A Difficult Road Toward Simplicity”, IEEE Trans. on Robotics andAutomation, Vol. 16, No. 6, pp. 652–662, 2000.
[2] A. M. Okamura, N. Smaby, and M. R. Cutkosky: “An Overview ofDexterous Manipulation”, Proc. of IEEE Int’l Conf. on Robotics andAutomation, pp. 255–262, 2000.

1889[3] T. B. Martin, R. O. Ambrose, M. A. Diftler, R. Platt Jr., and
M. J. Butzer: “Tactile Gloves for Autonomous Grasping with theNASA/DARPA Robonaut”, Proc. of IEEE Int’l Conf. on Robotics andAutomation, pp. 1713–1718, 2004.
[4] D. Katz and O. Brock: “Manipulating Articulated Objects with Interactive Perception”, Proc. of IEEE Int’l Conf. on Robotics andAutomation, pp. 272–277, 2008.
[5] K. Murakami and T. Hasegawa: “Tactile Sensing of Edge Directionof an Object with a Soft Fingertip Contact”, Proc. of IEEE Int’l Conf.
on Robotics and Automation, pp. 2582–2588, 2005.
[6] K. Murakami and T. Hasegawa: “Novel Fingertip Equipped with Soft
Skin and Hard Nail for Dexterous Multi-fingered Robotic Manipulation”, Proc. of IEEE Int’l Conf. on Robotics and Automation, pp.
708–713, 2003.
[7] M. T. Mason and J. K. Salisbury: Robot hands and the mechanics ofmanipulation, Cambridge, MA, MIT Press, pp. 9–23, 1985.
[8] A.Bicchi: “Intrinsic Contact Sensing for Soft Fingers”, Proc. of IEEE
Int’l Conf. on Robotics and Automation, pp. 968–973, 1990.