Passivity-based Dynamic Visual Force Feedback Control
for Fixed Camera Systems
Hiroyuki Kawai, Toshiyuki Murao and Masayuki Fujita
Image

Abstract— This paper investigates passivity based 3D dynamic visual force feedback control for fixed camera systems.
In our approach, we can control not only the position but also
the orientation of the robot hand with a contact force by using
visual information. The proposed method can be regarded as
an extension of the hybrid position/force control to the hybrid
vision/force control. The main contribution of this paper is toshow that the 3D dynamic visual force feedback system has
the passivity which allows us to prove stability in the senseof Lyapunov. Both the passivity of the manipulator dynamics
and the passivity of the visual feedback system are preserved.
Finally simulation results on 3DOF planar manipulator arepresented to verify the stability of the 3D dynamic visual force
feedback system and understand our proposed method simply.

Camera

gch

5c
gco

5h

Force
Sensor

gwh

gwc
gho

5w
World Frame
5oContact Surface
Target Object
gwo

Fig. 1.

Visual force feedback system with a fixed camera configuration.

I. I NTRODUCTION
Robotics and intelligent machines need sensory information to behave autonomously in dynamical environments.
Visual information is particularly suited to recognize unknown surroundings. Vision based control of robotic systems
involves the fusion of robot kinematics, dynamics, andcomputer vision to control the motion of the robot in an
efficient manner. The combination of mechanical control
with visual information, so-called visual feedback control orvisual servoing, is important when we consider a mechanicalsystem working under dynamical environments [1][2].
For the theoretically problem of three dimensional(3D)visual servo control based on the robot control theory,
Kelly et al. [3] considered a simple image-based controller
under the assumption that the objects’ depths are known.
Chen et al. [4] addressed the field-of-view problem for3D dynamic visual feedback system using an image-space
navigation function. In our previous works, we discussed
the dynamic visual feedback control for 3D target tracking
based on passivity [5][6]. On the other hand, applicationsof visual feedback system are also increasing in many
fields. For example, recent applications of visual feedback
system include the autonomous injection of biological cells[7], laparoscopic surgery [8] and others. Although visual
information is necessary in order to recognize environments,
only visual information is not enough to complete tasks inthese applications. For example, not only visual information
but also force information are needed to inject DNA to
H. Kawai is with Department of Robotics, Kanazawa Institute of Technology, Ishikawa 921-8501, Japan hiroyuki@neptune.kanazawa-it.ac.jp
T. Murao is with Master Program of Information Systems Architecture,
Advanced Institute of Industrial Technology, Tokyo 140-0011, Japan
M. Fujita is with Department of Mechanical and Control Engineering,Tokyo Institute of Technology, Tokyo 152-8550, Japan

biological cells. Hence, integrating visual feedback control
with force control is important for the modern robot.
Xiao et al. [9] developed sensor fusion scheme forcontrolling an end-effector to follow an unknown trajectory on a contact surface. Baeten et al. [10] addresseda hybrid control structure for the eye-in-hand vision andforce control. Although many practical methods are reportedwith experimental results, rigorous results have hardly beenobtained in terms of the nonlinear control aspects. For thisproblem, Dean-León et al. [11] have combined image-based
visual feedback control with force control and discussed thestability of the nonlinear system. The authors have proposed
passivity based visual force feedback control law for force
control with target tracking [12]. Although these control
laws guarantee Lyapunov stability and are effective for thevisual force feedback system, they are restricted to planar
manipulators.
This paper deals with 3D visual force feedback control for
fixed camera systems as depicted in Fig. 1. In our proposed
method, we can control not only the position but also theorientation of the robot hand with a contact force in thevisual force feedback system. The main contribution of thispaper is to show that the 3D visual force feedback system
has the passivity which allows us to prove stability in thesense of Lyapunov. Both the passivity of the manipulator
dynamics and the passivity of the visual feedback system arepreserved in the 3D visual force feedback system. Finallysimulation results are shown to verify the stability of the
proposed method.
II. V ISUAL F EEDBACK S YSTE M
This section mainly reviews our previous works ([5], [6])via the passivity based visual feedback control. Throughout

this paper, we use the notation eξ̂θab ∈ R3×3 to represent
the change of the principle axes of a frame Σb relative toa frame Σa . ξab ∈ R3 specifies the direction of rotationand θab ∈ R is the angle of rotation. For simplicity weˆ ab to denote ξˆab θab . The notation ‘∧’ (wedge) is theuse ξθ
skew-symmetric operator such that ξ̂θ = ξ × θ for the vector
cross-product × and any vector θ ∈ R3 . The notation ‘∨’(vee) denotes the inverse operator to ‘∧’, i.e., so(3) → R3 .
Recall that a skew-symmetric matrix corresponds to an axisof rotation (via the mapping a → â). We use the 4 × 4
matrixeξ̂θab pab
gab =
(1)
0
1
as the homogeneous representation of gab = (pab , eξ̂θab ) ∈SE(3) describing the configuration of a frame Σb relative toa frame Σa . The adjoint transformation associated with gab
is denoted by Ad(gab ) [14].
A. Basic Representation for Visual Feedback System
Visual feedback systems with a fixed camera configuration
typically use four coordinate frames which consist of a world
frame Σw , a target object frame Σo , a camera frame Σc anda hand (end-effector) frame Σh as in Fig. 1. Then, gwh =(pwh , eξ̂θwh ) ∈ SE(3), gwc = (pwc , eξ̂θwc ) ∈ SE(3) and
gwo = (pwo , eξ̂θwo ) ∈ SE(3) denote the rigid body motion
from Σw to Σh , from Σw to Σc and from Σw to Σo , respectively. Similarly, the relative rigid body motion from Σc toΣh , from Σc to Σo and from Σh to Σo can be represented
by gch = (pch , eξ̂θch ) ∈ SE(3), gco = (pco , eξ̂θco ) ∈ SE(3)and gho = (pho , eξ̂θho ) ∈ SE(3), respectively. as shown in
Fig. 1. The objective of the visual feedback control is tobring the actual relative rigid body motion gho to a givenreference gd = (pd , eξ̂θd ) which is constant in this paper. In
other words, our goal is to determine the motion of hand byusing the visual information.
The relative rigid body motion from Σc to Σo can be led
by using the composition rule for rigid body transformations([14], Chap. 2, pp. 37, eq. (2.24)) as follows:−1
gco = gwc
gwo .

(2)

The relative rigid body motion involves the velocity of eachrigid body. To this aid, let us consider the velocity of a rigid
body as described in [14]. We define the body velocity of theb
T
T T
camera relative to the world frame Σw as Vwc= [vwcωwc] ,where vwc and ωwc represent the velocity of the origin
and the angular velocity from Σw to Σc , respectively ([14]Chap. 2, eq. (2.55)).
Differentiating (2) with respect to time, the body velocityof the relative rigid body motion gco can be written as follows(See [5]):
b
b
b
Vco
= −Ad(gco−1 V) wc + Vwo(3)bis the body velocity of the target object relative towhere VwobΣw . In the case of the fixed camera configuration, i.e. Vwc=

0, the model of the relative rigid body motion gco can be
rewritten as
b
b
Vco
= Vwo
.

(4)

Roughly speaking, if both the camera and the target object
move, then the relative rigid body motion gco will be derivedbfrom the difference between the camera velocity Vwc
and thebtarget object velocity Vwo. Hence, the model of the relative
rigid body motion from Σc to Σo equals the target objectbvelocity Vwo.
B. Estimation Error System
The relative rigid body motion gco can not be immediately
obtained in the visual feedback system, because the target
object velocity is unknown and furthermore can not be measured directly. Hence, we consider the estimation problemof the relative rigid body motion gco . The visual feedback
control task requires information of the relative rigid body
motion gco . Since the measurable information is only theimage information f(gco ) in the visual feedback system, weconsider a nonlinear observer in order to estimate the relative
rigid body motion gco from the image information f(gco ).
Firstly, using the basic representation (4), we choosebestimates ḡco and V̄coof the relative rigid body motion andvelocity, respectively asbV̄co= ue .

(5)

The new input ue is to be determined in order to drive thebestimated values ḡco and V̄coto their actual values.
In order to establish the estimation error system, we define
the estimation error between the estimated value ḡco and theactual relative rigid body motion gco as−1
gee = ḡco
gco .

(6)

Using the notation eR (eξ̂θ ), the vector of the estimation error
is defined as ee := [pTee eTR (eξ̂θee )]T . Note that ee = 0 iff
pee = 0 and eξ̂θee = I3 . Therefore, if the vector of theestimation error is equal to zero, then the estimated relative
rigid body motion ḡco equals the actual relative rigid body
motion gco .
Suppose the attitude estimation error θee is small enoughthat we can let eξ̂θee  I +sk(eξ̂θee ). Therefore, using a firstorder Taylor expansion approximation, the estimation error
vector ee can be obtained from image information f(gco )and the estimated value of the relative rigid body motion ḡco
as follows [5]:¯ee = J † (ḡco )(f − f),(7)

where f¯ is the estimated value of image information. In thesame way as the basic representation (4), the estimation error
system can be represented bybb−1 ue + V
Vee
= −Ad(gee
wo .
)

(8)

It should be noted that if the vector of the estimation error isequal to zero, then the estimated relative rigid body motionḡco equals the actual one gco .

bVwo

C. Control Error Systems
In this subsection, let us consider the dual of the estimation
error system, which we call the control error system, in orderto establish the visual feedback system. We assume that gwc
and gwh can be obtained accurately by a prior calibration
procedure, then the estimated value of gho is calculated−1
as ḡho = gchḡco where ḡco is the estimated value which
discussed in the previous subsection. Here, we define thecontrol error between the actual relative rigid body motion
gho and desired one gd as
gec = gd−1 gho .

(9)It should be noted that gho can not be measured directory.
Similar to the definition of ee , the vector of the control error
is defined as ec := [pTec eTR (eξ̂θec )]T .
Here we have to consider the way of deriving gec (9),because gho can not be measured directory. Using gee , thecontrol error can be transformed as−1−1
gec = gd−1 gho = gd−1 ḡho
ḡho gho = gd−1 ḡho
gee .

(10)

In Equation (10), gd and ḡho While the estimation error
vector ee can be obtained as Equation (7), the estimation
error matrix gee cannot be directly obtained, because gee
is defined using non-measurable value gco as Equation (6).
Therefore, we consider the way of deriving gee from ee .
Because of the definition of the estimation error vector
ee , i.e., ee := [pTee eTR (eξ̂θee )]T , the position estimation error
pee can be derived directly from ee . Under the condition− π2 ≤ θee ≤ π2 , ξθee can be derived as follows [6]:ξθee =

sin−1 eR (eξ̂θee )eR (eξ̂θee )eR (eξ̂θee ).

(11)

d

ue
Fig. 2.

VisualFeedback
SystemeceeBlock diagram of the visual feedback system.

b
Lemma 1: [5] If Vwo= 0, then the visual feedback
system (13) satisfies
 T
uTceνce dt ≥ −βce , ∀T > 0(14)0

where νce is defined as νce := −e and βce is a positive
scalar.
Using the following positive definite function, we canprove Lemma 1.
Vce = E(gec) + E(gee )(15)

where E(gab ) := 12 pab 2 + φ(eξ̂θab ) and φ(eξ̂θab ) :=1ξ̂θab) is the error function of the rotation matrix
2 tr(I − e(see, e.g., [13]).
The block diagram of the passivity of the visual feedback
system is shown in Fig. 2. Let us take uce as the input andνce as its output in Fig. 2. Thus, Lemma 1 implies that thevisual feedback system (13) is passive from the input uce tothe output νce as in the definition in [15].
III. DYNAMIC V ISUAL F ORCE F EEDBACK C ONTROL
A. Dynamic Visual Force Feedback System

Hence, gee can be derived from ee through ξθee .
The reference of the relative rigid body motion gd isbb
constant in this paper, i.e., ġd = 0, hence, Vec= Vho. Thus,the control error system can be represented asbbb−1 Ad −1 V
Vec
= −Ad(gec)(g ) wh + Vwo .

b
Vwh

(12)

This is dual to the estimation error system.
D. Passivity of Visual Feedback System
Combining (8) and (12), we construct the visual feedback
system as follows: b   −Ad(gec
−1
0
Vec
I
)
b
=
u
+
Vwo
ce
b
−1
0
−Ad(gee
Vee
I
)
(13)
T

b T
) uTe
denotes the control
where uce := (Ad(g−1 ) Vwhdinput. For the design of the visual feedback system, it isb
can be directly chosen.
assumed that the hand velocity Vwh
Let us define the error vector of the visual feedback system
as e := [eTc eTe ]T which consists of the control error vector
ec and the estimation error vector ee .
Next, we show an important relation between the input
and the output of the visual feedback system.

The dynamics of n-link rigid robot manipulators with theend-effector constraint can be written as follows [16]M (q)q̈ + C(q, q̇)q̇ + g(q) = τ + Jϕ(q)λ(16)

where q, q̇ and q̈ are the joint angle, velocity and acceleration,
respectively, τ is the vector of the input torque. M (q) ∈Rn×n is the manipulator inertia matrix, C(q, q̇) ∈ Rn×n
is the Coriolis matrix and g(q) ∈ Rn is the gravity vector.
λ ∈ R is the contact force, Jϕ (q) ∈ Rn is the normalized
Jacobian of the kinematic constraint ϕ(q) = 0 ∈ R anddefined as follows.

T
∂ϕ(q)
T
∈ Rn
(17)
Jϕ (q)q̇ = 0, Jϕ (q) =∂qEquation (16) possesses several important properties which
will be used in the sequel. The manipulator
dynamics (16)T
is passive from τ to q̇, that is 0 τ T q̇dt ≥ −βm whereβm is a positive scalar. Moreover, Ṁ(q) − 2C(q, q̇) is skewsymmetric by defining C(q, q̇) using the Christoffel symbols.
Now, we propose the control law for the manipulator asτ = M (q)q̈r + C(q, q̇)q̇r + g(q)+JbT (q)AdT(g−1 ) ec − Jϕ λd + us + Jϕ uF (18)dwhereq̇r := Qϕ (q)q̇d + αJϕ(q)Fe .
α is positive constant and
 t
 t

eλ dτ ∈ RFe :=λ − λd dτ =
0

(19)

(20)

0

where eλ := λ − λd is the force error. Then, the following
relation holds with respect to about the force errorḞe = eλ .

(21)

Because we consider the single point contact in this paper,
the projection matrix Qϕ (q) can be simply defined as
Qϕ (q) = I − Jϕ (q)JϕT (q)(22)

which arises on the tangent space at the contact surface
ϕ(q) = 0 [16].
bOn the other hand, the body velocity of the hand Vwh
isgiven byb
Vwh
= Jb (q)q̇(23)

where Jb (q) is the body manipulator Jacobian [14]. Moreover, we design the reference of the joint velocity asq̇d := Q†ϕ (q) Jb† (q)uh − αJϕ (q)Fe

(24)

T
T T
where uh = [vuhωuh
] is the desired body velocity of
the hand which will be obtained from the visual feedback
system.
We define the error vector with respect to the joint velocityof the manipulator dynamics as

s := q̇ − q̇r ∈ Rn

(25)

Here, we know that the following relation holds [16]JϕT s = −αFe .

(26)

Using (13)(16) and (18), the visual force feedback systemwith manipulator dynamics (we call the dynamic visual force
feedback system) can be derived as follows:⎤⎡⎤ ⎡−M −1 Cs − JbT AdT(g−1 ) ec − Jϕ eλṡd⎥⎢ Ḟe ⎥ ⎢
⎥
eλ
⎢
⎥=⎢
⎥
⎢
b ⎦
⎣ Vec
⎦
⎣
−Ad(g−1 ) Jb s
ho
b
Vee
0
⎤
⎤
⎡
⎡
M −1 M −1 Jϕ
0
0
0
⎥
⎥
⎢
⎢ 0
0
0
0
⎥u + ⎢ 0 ⎥w
+⎢
⎦
⎣
⎣ 0
0 −Ad(gec
−1
0
I ⎦
)
I
0
0
0
−Ad(gee
−1
)
(27)
where

⎡

⎤
⎤
⎡
us
s
⎢ Fe ⎥
⎥
⎢
uF
b
⎥
⎥
⎢
x := ⎢⎣ ec ⎦ u := ⎣ Ad(g−1 ) uc ⎦ w := Vwo ,deeue

are defined as the state, the input and the disturbance of the
dynamic visual force feedback system, respectively. Here,
we formulate the manipulator control problem as follows:
Control problem : For the dynamic visual force feedback
system with the fixed camera configuration described by (27),design a control input u such thatlim s = 0, lim Fe = 0, lim ec = 0 and lim ee = 0.

t→∞

t→∞

t→∞

t→∞

B. Passivity of Dynamic Visual Force Feedback System
Before constructing the dynamic visual force feedback
control law, we derive an important lemma.
Lemma 2: If w = 0, then the dynamic visual force
feedback system (27) satisfies
 T
uT ν ≥ −β, ∀ T > 0(28)0where ν := N x, N := diag{In , −α, −I6 , −I6 } and β is apositive scalar.
Proof: Consider the following positive definite function
V =

1
1 T
s M s + αFe2 + E(gec ) + E(gee ).
2
2

(29)

Differentiating (29) with respect to time yields
1V̇ = sT M ṡ + sT Ṁ s + αFe Ḟe2+pTec eξ̂θec e−ξ̂θec ṗec + eTR (eξ̂θec )eξ̂θec ωec+pTee eξ̂θee e−ξ̂θee ṗee + eTR (eξ̂θee )eξ̂θee ωee
⎡
⎤⎡
⎤
M 0
0
0
ṡ
⎢ 0 α
⎥ ⎢ Ḟe ⎥ 1 T
00⎥⎢⎥= xT ⎢⎣ 0 0 Ad(eξ̂θec )⎦ ⎣ V b ⎦ + 2 s Ṁs.
0
ec
b
0 0
0
Ad(eξ̂θee )Vee(30)
Observing that the skew-symmetry of the matrices p̂ec andp̂ee , i.e., pTecp̂ec e−ξ̂θd ωud = −pTec (e−ξ̂θd ωud )∧ pec = 0,
pTee p̂ee ωue = −pTee ω̂ue pee = 0, the above equation along
the trajectories of the system (27) can be transformed intoV̇ = −sT C ṡ + sT JbT AdT(g−1 ) ec + sT Jϕ eλ + αFe eλd1−eTc Ad(g−1 ) Jb s + sT Ṁ s
ho

2
⎤
M 0
0
0
⎢ 0 α
⎥
0
0
⎥
+xT ⎢
⎣ 0 0 Ad(eξ̂θec )
⎦
0
0 0
0
Ad(eξ̂θee )⎤⎡00
M −1 M −1 Jϕ
⎥
⎢ 0
0
0
0
⎥u
× ⎢
⎦
⎣ 0
−1
0 −Ad(gec
0
)
−1
0
0
0
−Ad(gee)1 T= s (Ṁ − 2C)s + eTλ (Jϕ s + αFe ) + xT N T u
2= xT N T u.
(31)
⎡

T

Ad(gd-1)ec

uc
us
uF
Fig. 3.

Manipulator
Dynamics& Auxiliary
Controllerb(=Jb(q)q)
_
Vwh

s
Fe

ue

Camera Framezcw
VisualFeedback
System

5c

xc

ec

gch

ee
gco
Gwo

(32)

where β is a positive scalar that only depends on the initialstates of s, Fe , ec and ee .
The block diagram of the passivity of the 3D dynamic visual
force feedback system is shown in Fig. 3.
Remark 1: The visual feedback system (13) satisfies thepassivity property as described in (14). It is well known
that the manipulator dynamics (16) also has the passivity.
In Lemma 2, the inequality (28) says that the dynamic
visual force feedback system (27) is passive from the input
u = [uTs uTF (Ad(g−1 ) uc)T uTe ]T to the output ν = [sT −dαFeT − eTc − eTe ]T as shown in Fig. 3.
C. Passivity-based Dynamic Visual Force Feedback Control
We now propose the following control input for theinterconnected system:(33)

where Ks := diag{ks1, · · · , ksn}, kF , Kc := diag{kc1 , · · · ,kc6 } and Ke := diag{ke1, · · · , ke6} denote the positive gain
matrices.
Theorem 1: If w = 0, then the equilibrium point x = 0
for the closed-loop system (27) and (33) is asymptotic stable.
Proof: In the proof of Lemma 2, we have alreadyderived that the time derivative of V along the trajectoryof the system (27) is formulated as (31). Using the control
input (33), (31) can be transformed intoV̇ = −xT N T KN x.

zo

zh
Gwh

5h
xh

gho

(34)

This completes the proof.
Theorem 1 shows the stability via Lyapunov method forthe dynamic visual force feedback system. It is interesting
to note that stability analysis is based on the passivity as
described in (28). Our proposed method is valid for the 3D
dynamic visual force feedback system, while previous works[11][12] consider the 2D dynamic visual force feedback
control. Hence, we can control not only the position but also
the orientation of the robot hand with a contact force in thevisual force feedback system.

5o

yo

Image
q3

q2

World Frame
zw
5w
q1
xw
yw

yh gwoHand Framexo0u = −Kν = −KN x
K := diag{Ks , kF , Kc, Ke } ∈ Rn+13

gwc

yc

Block diagram of the 3D dynamic visual force feedback system.

Integrating (31) from 0 to T , we obtain T
uT νdτ = V (T ) − V (0) ≥ −V (0) = −βincludingfour feature points

3DOF Robot
Manipulator

Contact Point

Target Object Frame
Fig. 4. Coordinate frames for dynamic visual force feedback system withthree degree of freedom manipulator.

IV. S IMULATION RESULTS
The simulation results on 3DOF planar manipulator as
depicted in Fig. 4 are shown in order to understand our
proposed method simply, though it is valid for 3D dynamic
visual force feedback systems.
We present results for the stability analysis with a static
target object. The simulation is carried out with the conditions pwo = [0.47 0.05 0]T [m], ξθwo = [0 0 0]T [rad],pwc = [0.47 0.05 1]T [m], ξθwc = [0 0 0]T [rad]. The
lengths of the three links of the manipulator are l1 = 0.2[m], l2 = 0.2 [m] and l3 = 0.1 [m], respectively. The initial
angles of the manipulator is set as q1 (0) = −π/2 [rad],q2 (0) = π/2 [rad] and q3 (0) = π/2 [rad]. In other words, theinitial relative rigid body motion is pho = [0.15 − 0.27 0]T[m], ξθho = [0 0 −π/2]T [rad]. The desired force λd and thedesired relative rigid body motion gd = (pd , eξ̂θd ) are givenby λd = 5 [N], pd = [0.03 0 0]T [m] and ξθd = [0 0 0]T[rad] in this simulation. The initial errors of force and vision
are calculated as λe = −5 [N], pec = [0.12 − 0.27 0]T [m],ξθec = [0 0 − π/2]T [rad], pee = [0.53 0.95 0]T [m] andξθee = [0 0 π/4]T [rad], respectively.
The controller parameters for Equation (33) were empirically selected as Ks = diag{10, 10, 10}, kF = 25, α = 1,
Kc = diag{40, 40, 20, 20, 20, 40} and Ke = 50I6 . The
simulation results are shown in Figs. 5 and 7. Figs. 5–7
illustrate the control error ec , the estimatoin error ee , andthe contact force λ, respectively. In Figs. 5 and 6, we focus
on the errors of the translations of x and y and the rotationof z, because the errors of the translation of z and therotations of x and y are zeros ideally on the 3DOF planar
manipulator. The control error ec and the estimation error
ee tended to zero, thus we can confirm that the relative
rigid body motion gho coincided with the desired one gd byusing image information. In Fig. 7, the pulse siglnal means
the contact transition at around 0.2 [s]. The contact force λtended to 5 [N], i.e., converged to the desired one λd . Fromthese figures, the asymptotic stability can be also confirmed.

5

0
0.4

0

0.2

0.4

0.6

0.8

1.0

0
-0.4
0.6
0
-0.6
-1.2

0

0.2

0.4

0.6

0.8

1.0

0

0.2

0.4
0.6
time [s]0.81.0peex [m]peey [m]NGeez [rad]

0.8
0.4
0
0

0.2

0.4

0.6

0.8

1.0

0

0.2

0.4

0.6

0.8

1.0

0

0.2

0.4
0.6
time [s]

0.8

1.0

0.5
0
0.8
0.4

4
3
2
1
0

Fig. 5. The control error ec which defined the error between the relative
rigid body motion of the robot hand gho and desired one gd . Initial control
errors are pecx = 0.12 [m], pecy = −0.27 [m] and ξθecz = −π/2 [rad],respectively

1

contact force λ [N]pecx [m]pecy [m]NGecz [rad]

6

0.2

0

Fig. 6.
The estimation error ee which defined the error between theestimated value ḡco and the actual relative rigid body motion gco . Initial
estimation errors are peex = 0.53 [m], pecy = −0.95 [m] and ξθecz =π/4 [rad], respectively

V. C ONCLUSIONSThis paper considers 3D visual force feedback control
for fixed camera systems. In our approach, we can control
not only the position but also the orientation of the robot
hand with a contact force by using visual information.
The proposed method can be regarded as an extension of
the hybrid position/force control to the hybrid vision/force
control. The main contribution of this paper is to show thatthe visual force feedback system has the passivity whichallows us to prove stability in the sense of Lyapunov. Boththe passivity of the manipulator dynamics and the passivityof the visual feedback system are preserved in the visual
force feedback system. Finally simulation results are shownto verify the stability of the proposed method.

Contact Transition
00.2
Fig. 7.

0.4
0.6
time [s]0.81.0The contact force trajectory λ.

R EFERENCE S[1] F. Chaumette and S. Hutchinson, “Visual Servo Control, Part I: BasicApproaches,” IEEE Robotics & Automation Magazine, Vol. 13, No. 4,
pp. 82–90, 2006.
[2] F. Chaumette and S. Hutchinson, “Visual Servo Control, Part II:
Advanced Approaches,” IEEE Robotics & Automation Magazine,
Vol. 14, No. 1, pp. 109–117, 2007.
[3] R. Kelly, R. Carelli, O. Nasisi, B. Kuchen and F. Reyes, “Stable
Visual Servoing of Camera-in-Hand Robotic Systems,” IEEE Trans.
Mechatronics, Vol. 5, No. 1, pp. 39–48, 2000.
[4] J. Chen, D. M. Dawson, W. E. Dixon and V. K. Chitrakaran, “Navigation Function-based Visual Servo Control,” Automatica, Vol. 43,
No. 7, pp. 1165–1177, 2007.
[5] M. Fujita, H. Kawai and M. Spong, “Passivity-based Dynamic VisualFeedback Control for Three Dimensional Target Tracking: Stability
and L2 -gain Performance Analysis,” IEEE Transactions on Control
Systems Technology, Vol. 15, No. 1, pp. 40–52, 2007.
[6] T. Murao, H. Kawai and M. Fujita “Predictive Visual Feedback Control
with Eye-in/to-Hand Configuration via Stabilizing Receding Horizon
Approach,” Proc. of the 17th IFAC World Congress on AutomaticControl, 2008(to appear).
[7] S. Yu and B. J. Nelson, “Autonomous Injection of Biological Cells
Using Visual Servoing,” In:D. Rus and S. Singh (Eds), Experimental
Robotics VII, Springer–Verlag, pp. 169–178, 2001.
[8] K. Omote et al., “Self-Guided Robotic Camera Control for Laparoscopic Surgery Compared with Human Camera Control,” The
American Journal of Surgery, Vol. 177, No. 4, pp. 321–324, 1999.
[9] D. Xiao, B. K. Ghosh, N. Xi and T. J. Tarn, “Sensor-based Hybrid
Position/Force Control of a Robot Manipulator in an Uncalibrated
Environment,” IEEE Transactions on Control Systems Technology,
Vol. 8, No. 4, pp. 635–645, 2000.
[10] J. Baeten, H. Bruyninckx and J. D. Schutter, “Integrated Vision/Force
Robotic Servoing in the Task Frame Formalism,” International Journalof Robotics Research, Vol. 22, No. 10–11, pp. 941–954, 2003.
[11] E.C. Dean-León, V. Parra-Vega and A. Espinosa-Romero, “Visual
Servoing for Constrained Planar Robots Subject to Complex Friction,”
IEEE/ASME Trans. on Mechatronics, Vol. 11, No. 4, pp. 389–400,
2006.
[12] H. Kawai, T. Murao and M. Fujita, “Passivity-based Visual ForceFeedback Control for Planar Manipulators with Eye-in-Hand Configuration,” Proc. of 16th IEEE International Conference on Control
Applications pp. 1480-1485, 2007.
[13] F. Bullo and A.D. Lewis, Geometric Control of Mechanical Systems,
Springer–Verlag, 2004.
[14] R. Murray, Z. Li and S. S. Sastry, A Mathematical Introduction to
Robotic Manipulation, CRC Press, 1994.
[15] A. van der Schaft, L2 -Gain and Passivity Techniques in Nonlinear
Control (2nd ed.), Springer–Verlag, 2000.
[16] Y. Liu , K. Kitagaki, T. Ogasawara and S. Arimoto, “Model-based
Adaptive Hybrid Control for Manupilators Under Multiple GeometricConstraints,” IEEE Trans. on Control Systems Technology, Vol. 7,
No. 1, pp. 97–109, 1999.