Passivity-based Synchronized Visual Feedback
Control for Eye-to-Hand Systems
Toshiyuki Murao

Hiroyuki Kawai

Masayuki Fujita

Master Program of Innovation
for Design and Engineering
Advanced Institute ofIndustrial Technology
Tokyo 140-0011, Japanmurao-toshiyuki@aiit.ac.jp

Department of Robotics
Kanazawa Institute of Technology
Ishikawa 921-8501, Japanhiroyuki@neptune.kanazawa-it.ac.jp

Department of Mechanicaland Control Engineering
Tokyo Institute of Technology
Tokyo 152-8550, Japanfujita@ctrl.titech.ac.jp

Abstract—This paper investigates passivity based synchronizedvisual feedback control for eye-to-hand systems. Firstly the briefsummary of the estimation and the control error systems is given
with the basic representation of a relative rigid body motion. Secondly we construct the synchronized visual feedback system withan eye-to-hand configuration by combining a synchronization
error system. Next, we derive the passivity of the synchronizedvisual feedback system. Based on passivity, stability analysis isdiscussed. Finally the validity of the proposed control law canbe confirmed by the simulation results.

I. I NTRODUCTION
Robotics and intelligent machines need sensory information
to behave autonomously in dynamical environments. Visual
information is particularly suited to recognize unknown surroundings. In this sense, vision is one of the highest sensing
modalities that currently exist. The combination of mechanicalcontrol with visual information, so-called visual feedback
control is important when we consider a mechanical system
working in dynamical environments [1].
In classical visual servoing, many practical methods arereported by two well known camera configurations, i.e., aneye-in-hand configuration or an eye-to-hand (fixed camera)
one [1]. Recently, new camera configurations combined eachclassical one have been proposed. Flandin et al. [2] addressed
an eye-in-hand and an eye-to-hand cooperation approach thateach camera information is partitioned into the positioning
task and the orientation one, respectively. In [3], the occlusion
problem has been tackled by using multi eye-in-hand and eyeto-hand cameras. The authors discussed passivity based visualfeedback control with an eye-in/to-hand configuration which
consists of two robot manipulators, i.e., a work manipulator
and a camera one [4][5]. A common feature among thesevisual feedback systems with a new camera configuration [2]–[5] is to consist of multiple robot manipulators.
On the other hand, some efforts have recently been madeto design multicomposed system which is shared information
among robot manipulators. Especially, the mutual synchronized control problem can be formulated as to design interconnections and controllers for the robot manipulators in thesystem, such that the motion of the manipulator has to besynchronized with respect to not only the desired trajectory
but also the motion of other manipulators. In production
process tasks that cannot be carried out by a single robot,
either because of the complexity of the task or limitations of
the robot, the use of multirobot systems working in mutualsynchronization, e.g., cooperative schemes, has proved to bea good alternative [6]. One of the advantages of synchronized
control is that the controller can reduce the magnitude ofthe error for the whole system in the transient stage throughsynchronous manner, especially in the presence of nonidenticalexternal disturbances.
One recent representative work on synchronization of robot
manipulators is in [6]. In this work, a scheme which needs
only position measurements was proposed to solve the problem of position synchronization of multiple cooperative robot
systems, and the controller was shown to be semi-globally
exponentially stable. In [7], a leader-follower synchronization
output feedback control scheme for the ship replenishment
problem was proposed. Chung et al. [8] provided a methodthat eliminates both the all-to-all coupling and the feedback of
the acceleration terms. Sun et al. designed an adaptive crosscoupled controller [9] and a model-free one [10] to stabilize
multi-axis motions for synchronizing mechanical systems.
Visual feedback, however, is not considered here. The scheme,
which applies to mutual synchronized control for the visualfeedback system with multiple robot manipulators, should
allow us to extend the technological application area.
This paper deals with synchronized visual feedback control
for eye-to-hand systems as depicted in Fig. 1. In this paper, forsimplicity, the synchronized visual feedback system consistsof only two manipulators and a fixed camera, while it canbe easily extended the system which consists of more manipulators and/or a movable camera [4][5]. In our proposedsystem, not only the convergence of the relative rigid body
motion from the hand to the target object to the desired
one, but also that of between manipulators are guaranteed,since the synchronization error system is constructed. Basedon passivity, stability analysis for the synchronized visualfeedback system is discussed. Finally simulation results areshown to verify the performance of the proposed method.

Camera

gch1

5c
gwc
gco
gww1
5w

World
Frame

gwo

gch2

5o

gww2

g h 2oTargetObject5h1
gw1h1

g h 1o5w1
Manipulator 1

5h2
gw2h2
5w2
Manipulator 2

Fig. 1. Synchronized visual feedback system with eye-to-hand configuration.

II. E STIMATION AND C ONTROL E RROR S YSTEMS
Throughout this paper, we use the notation eξ̂θab ∈ R3×3
to represent the change of the principle axes of a frame Σb
relative to a frame Σa . ξab ∈ R3 specifies the direction ofrotation and θab ∈ R is the angle of rotation. For simplicity
we use ξ̂θab to denote ξˆab θab . The notation ‘∧’ (wedge) is theˆ = ξ × θ for the vectorskew-symmetric operator such that ξθcross-product × and any vector θ ∈ R3 . The notation ‘∨’(vee) denotes the inverse operator to ‘∧’, i.e., so(3) → R3 .
Recall that a skew-symmetric matrix corresponds to an axisof rotation (via the mapping a → â). We use the 4 × 4 matrixgab =eξ̂θab
0

pab
1


(1)

as the homogeneous representation of gab = (pab , eξ̂θab ) ∈SE(3) describing the configuration of a frame Σb relative toa frame Σa . The adjoint transformation associated with gab isdenoted by Ad(gab ) [11].
A. Generation of Desired Relative Rigid Body Motion
The visual feedback system considered in this paper has
two robot manipulators and a fixed camera as depicted inFig. 1, where the coordinate frames Σw , Σc , Σo Σwi andΣhi represent the world frame, the camera frame, the object
frame, the base frame of the i-th manipulator, and the i-th
hand (end-effector of the i-th manipulator) frame, respectively.
Here, the subscript i denotes the index of the manipulators,
i.e. i = 1, 2 in this paper. Then, the relative rigid body motion
from Σc to Σo can be represented by gco . Similarly, the rigid
body motions gwc , gwo , gww1 and gww2 , and the relative rigid
body motions gch1 , gch2 , gw1 h1 , gw2 h2 , gh1 o and gh2 o are
represented, respectively, as shown in Fig. 1.
The objective of synchronized visual feedback control isthat both manipulators track the target object while the manipulators should mutually synchronize. In other words, it is
to bring the actual relative rigid body motions gh1 o and gh2 o
to given constant desired ones gh1 d and gh2 d , respectively.
It should be noted that the desired motions gh1 d and gh2 d are

related each other because of the mutual synchronous behavior.
Therefore, we state how to set the desired rigid body motions
for synchronized visual feedback control.
Firstly, we consider the reference manipulator as depicted
in Fig. 2, which is the fundamental manipulator to track thetarget object. The desired value of the synchronized visualfeedback system has only one degree of freedom. We define
it as ghd which is the relative rigid body motion from thereference hand frame Σh to the object frame Σo . Next, we
set the reference target objects which have the coordinate
frames Σoi as depicted in Fig. 2, to fulfill gwi oi = gwo (Thismeans gooi = gwwi , too.). Meanwhile, if the control objectiveis achieved, the motion of all manipulators is corresponded
to the reference manipulator i.e. gwi hi = gwh because of
the mutual synchronous behavior. This means to satisfy the
relationship ghhi = gwwi , too. Hence, we can set ghi di using
ghd as follows:−1
ghi di = gww
g g
.
(2)
i hd wwi
We define the control error gehi between the actual relative
rigid body motion ghi oi and the desired one ghi di as
g hi o i ,gehi = gh−1
i di

(3)

in other words, pehi = e−ξ̂θhi di (phi oi − phi di ) and eξ̂θehi =e−ξ̂θhi di eξ̂θhi oi . Note that phi oi = phi di and eξ̂θhi oi = eξ̂θhi diiff gehi = I4 , i.e., pehi = 0 and eξ̂θehi = I3 . Using the notationeR (eξ̂θab ) := sk(eξ̂θab )∨ := 12 (eξ̂θab −e−ξ̂θab )∨ which denotes
the error vector of the rotation matrix eξ̂θab , the vector of thecontrol error is given by ehi := [pTehi eTR (eξ̂θehi )]T . Note thatehi = 0 iff pehi = 0 and eξ̂θehi = I3 . Therefore, if the vectorof the control error is equal to zero, then the actual relative
rigid body motion ghi oi equals the desired relative rigid body
motion ghi di .
B. Visual Motion Observer and Estimation Error System
This subsection mainly reviews our previous works [12][13]via the passivity based visual feedback control.
In order to achieve the control objective, the relative rigid
body motion ghi oi is needed. ghi oi can be represented asfollows:−1
gco gwwi .
(4)
ghi oi = gchi
In (4), since gchi and gwwi can be given by the measured
information, we consider to get the relative rigid body motion
gco 1 . The relative rigid body motion from Σc to Σo can be led
by using the composition rule for rigid body transformations([11], Chap. 2, pp. 37, eq. (2.24)) as follows:−1gwo .
gco = gwc

(5)

The relative rigid body motion involves the velocity of eachrigid body. To this aid, let us consider the velocity of a rigid
body as described in [11]. We define the body velocity of theb
T
T T
camera relative to the world frame Σw as Vwc= [vwcωwc] ,where vwc and ωwc represent the velocity of the origin and the
1 We assume that g
wc and gwhi can be obtained accurately by a priorcalibration procedure.

Manipulator 1

gww1
5w1

gh1d1

5o
Reference 1
Target Object 1

gww1

5h2

5h
gw1h1

gwo

Manipulator 2

Reference Manipulator

5h1

gww1
gh1d

ghd
gwo

gw2h2
gh2d2

5w

gww2

5o

gww2

5w2

gwo

gh2d

TargetObjectFig. 2.

gww2

gwh

5o
Reference 2
Target Object 2

Generation of desired relative rigid body motion.

angular velocity from Σw to Σc , respectively ([11] Chap. 2,
eq. (2.55)).
Differentiating (5) with respect to time, the body velocityof the relative rigid body motion gco in the case of the fixedbcamera configuration (i.e. Vwc= 0) can be written as follows(See [13]):
b
b
Vco
= Vwo
(6)
b
where Vwois the body velocity of the target object relativeto Σw . Equation (6) is a basic representation for the threecoordinate frames of the visual feedback system.
The visual feedback control task requires information of
the relative rigid body motion gco . Since the measurable
information is only the image plane coordinate f (gco ) ∈ R2m
which is given by the perspective projection of the m featurepoints onto the image plane, we consider a visual motion
observer in order to estimate the relative rigid body motion
gco from the image information f (gco ).
b
Using (6), we choose estimates ḡco and V̄coof the relative
rigid body motion and velocity, respectively asbV̄co= ue .
(7)

The new input ue is to be determined in order to drive thebestimated values ḡco and V̄coto their actual values.
In order to establish the estimation error system, we define
the estimation error between the estimated value ḡco and theactual relative rigid body motion gco as
−1
gco .
(8)
gee = ḡcoUsing the notation eR (eξ̂θab ), the vector of the estimation error
is defined as ee := [pTee eTR (eξ̂θee )]T . Therefore, if the vectorof the estimation error is equal to zero, then the estimatedrelative rigid body motion ḡco equals the actual relative rigid
body motion gco .
Suppose the attitude estimation error θee is small enoughthat we can let eξ̂θee  I +sk(eξ̂θee ). Using a first-order Taylor
expansion approximation, the estimation error vector ee canbe obtained from image information f (gco ) and the estimatedvalue of the relative rigid body motion ḡco as follows ([12]):ee = J † (ḡco )(f − f¯),
(9)where J(ḡco ) is the image Jacobian, f¯ is the estimated valueof image information and † denotes the pseudo-inverse. In thesame way as the basic representation (6), the estimation error
system can be derived asbb−1 ue + V
= −Ad(gee
Vee
wo .
)

(10)

It should be noted that if the vector of the estimation error isequal to zero, then the estimated relative rigid body motionḡco equals the actual one gco .
C. Control Error System
In this subsection which mainly reviews our previous
work [5], let us consider the dual of the estimation error
system, which we call the control error system.
In order to achieve the control objective, we have to derivegehi defined as (3). Using gee , the control error can betransformed as
ghi oi=gh−1
g −1 gco gwwi=gh−1
g −1 ḡco gee gwwi .(11)gehi=gh−1
i di
i di chii di chi
In (11), ghi di , gchi , ḡco and gwwi are known information.
While the estimation error vector ee can be obtained as (9),the estimation error matrix gee cannot be directly obtained,
because gee is defined using non-measurable value gco as (8).
Therefore, we derive gee from ee .
Because of the definition of the estimation error vector ee ,i.e., ee := [pTee eTR (eξ̂θee )]T , the position estimation error pee
can be derived directly from ee . Under the condition − π2 ≤θee ≤ π2 , ξθee can be derived as follows [5]:ξθee =

sin−1 eR (eξ̂θee )eR (eξ̂θee )eR (eξ̂θee ).

(12)Hence, gee can be derived from ee through ξθee .
The reference of the relative rigid body motion ghi di isbconstant in this paper, i.e., ġhi di = 0, hence, Veh= Vhbi oi .
iThus, differentiating (3) with respect to time, the control error
system can be represented asbbb= −Ad(g−1 ) Ad(g−1 ) Vwh
+ Vwo
.
(13)
Veh
i
i
ehi

hi di

This is dual to the estimation error system.
III. PASSIVITY- BASED S YNCHRONIZED V ISUAL
F EEDBACK C ONTROLFor assembly tasks in modern manufacturing and space
applications, a scheme which reduces not only the error
concerning the control objective but also that of synchronous
manner must have the great potential [9]. For the visualfeedback system which consists of two manipulators and afixed camera shown as Fig. 1, the straightforward extension of
the controller using only the estimation and the control errorsystems in [13] can be also proposed without synchronous
manner. However, although it can be treated the tracking
problem, it can not be ensured the synchronous behavior. Thissection considers synchronized visual feedback control basedon passivity which is a main contribution in this paper.
A. Synchronization Error System
In this subsection, we derive the synchronization error
system in order to impose the synchronous manner. Firstly, wedefine the error gehij = (pehij , eξ̂θehij ) between the relative
rigid body motion of one manipulator ghi oi and that of anothermanipulator ghj oj asgehij = gh−1
ghi oi , (i, j) = (1, 2), (2, 1).
(14)
j oj
Moreover, the synchronization error is defined using the element of the predefined error (14) and the coupling gains which
denote the interactions between manipulators in the system asfollows:ξ̂θehij
kpij pehij
gesij = kRij e(15)
01
where kpij is the coupling gain for the position and kRij isthat for the rotation. Using the notation eR (eξ̂θab ), the synchronization error vector is defined as esij := [pTesij eTR (eξ̂θesij )]T .
Differentiating (15) with respect to time, the synchronization error system can be represented asbb= −Ksij Ad(g−1 ) Ad(g−1 ) Vwh
Ves
ij
i
ehi

hi di

b
+Ksij Ad(g−1 ·gh d ) Ad(g−1 ) Vwh
j
hi oi
hj dj
j j

b
−1
+Ksij Ad(gww) I − Ad(g−1 ·gh o ) Vwo , (16)
i

hi o

j

2
I} for all i, j = 1, 2, i=j.
where Ksij := diag{kpij kRij I, kR
ij

B. Synchronized Visual Feedback Systemas
Mi (qi )q̈i + Ci (qi , q̇i )q̇i + gi (qi ) = τi + τid(17)

where Mi ∈ R
is the inertia matrix, Ci ∈ R
is theCoriolis matrix, gi ∈ Rn is the gravity vector, and qi , q̇i and q̈iare the joint angle, velocity and acceleration, respectively. τi isthe vector of the input torque, and τid represents a disturbance
input. Since the manipulator dynamics is considered, the handb
= Jib (qi )q̇i , where Jib (qi ) is
body velocity is given by Vwhithe body manipulator Jacobian [11].
Next, we propose the control law for the manipulator asτi = Mi (qi )q̈id + Ci (qi , q̇i )q̇id + gi (qi )TT+Jib(qi )AdT(g−1 ) ehi + Jib(qi )AdT(g−1 ) Ksij esij
n×n

n×n

hi di

C. Passivity of Synchronized Visual Feedback System
Before proposing a synchronized visual feedback control
law, we derive an important lemma.
Lemma 1: If w = 0, then the synchronized visual feedback
system (19) satisfies
 T
uT ν ≥ −β, ∀ T > 0
(20)
0

where
ν := N Ks x⎡⎤
I 0 0 00
0
0
⎢0 I 0 0
0
0
0 ⎥
⎢
⎥
T
⎢0 0 −I 0 −AdT −1
Ad(g−1 )0 ⎥(gh o ·gh1 d1 )
N = ⎢
⎥
eh
2 2
1
⎢
⎥
AdT(g−1 ) −AdT(g−1 ·gh d ) 0 ⎦⎣0 0 0 −I
eh2

The dynamics of the rigid body manipulators can be writtenT−Jib(qi )AdT(g−1 ) Ksji esji + uξibshould befrom the aforementioned error systems. Thus, Vwhireplaced by uhi d .
Using (10), (13) and (16)–(18), the synchronized visual feedback system can be derived as the equation (19)at the bottom of the next page, where the input u :=[uTξ1 uTξ2 (Ad(g−1 ) uh1 d )T (Ad(g−1 ) uh2 d )T uTe ]T . We define
h1 d1
h2 d2
the state and the disturbance of the synchronized visualfeedback system as x := [ξ1T ξ2T eTh1 eTh2 eTs12 eTs21 eTe ]T and
T
T
b T T
w := [τ1dτ2d(Vwo) ] , respectively.
Remark 1: If the coupling gains kpij = kRij = 0 andthe synchronization error esij is not considered, then thesynchronized visual feedback system (19) can be regarded
as the non-synchronized one which consists of two manipulators and a fixed camera. Non-synchronized system is astraightforward extension of the eye-to-hand visual feedback
system [13]. Thus, the visual feedback system with the eyeto-hand configuration [13] is included as the special case of
the system (19).

hj oj

(18)

hi oi

where q̇id and q̈id represent the desired joint velocity and acceleration, respectively. The new input uξi is to be determinedin order to achieve the control objective.
Let us define the error vector with respect to the joint
velocity of the manipulator as ξi := q̇i − q̇id . Moreover, we†(qi )uhi ddesign the reference of the joint velocity as q̇id := Jib
where uhi d is the desired body velocity which will be obtained

0 0 0 00
Ks = diag{I, I, I, I, Ks12 , Ks21 , I}

h1 o1

0

2 2

−I

and β is a positive scalar.
Proof: Consider the following positive definite function11
V = ξ1T M1 ξ1 + ξ2T M2 ξ2 + E(geh1 ) + E(geh2 )2211E(ges12 ) +E(ges21 ) + E(gee ), (21)
+
kR12
kR21
where E(gab ) := 12 pab 2 + φ(eξ̂θab ) and φ(eξ̂θab ) :=1ξ̂θab) is the error function of the rotation matrix (see,2 tr(I − ee.g., [14]). Differentiating (21) with respect to time yields
11V̇ = ξ1T Ṁ1 ξ1 + ξ2T Ṁ2 ξ2
2⎡
2
⎤
⎡ ˙ ⎤
0
0
0
0
0
M1 0
⎢ 0 M2 00000 ⎥ ξ1⎢⎥⎢ ˙ ⎥⎢ 0 0 Ad ξ̂θeh 0000 ⎥⎢ ξb2 ⎥1)⎢⎥⎢ V ⎥(e⎢ 0 0⎢ eh1 ⎥
0 Ad ξ̂θeh2 000 ⎥⎥⎢ V b ⎥
+xT⎢
(e
)
⎢
⎥⎢ eh2 ⎥
⎢ 0 0
⎢ b ⎥
0 ⎥
0
0 Ad ξ̂θeh12 0⎢⎥⎢Ves12 ⎥(e)⎢⎥⎣V b ⎦
00
0 Ad ξ̂θeh21 0 ⎦ es21
⎣ 0 0
(e
)
b
Vee
0 0
0
0
0
0 Ad
= xT Ks N T u,(eξ̂θee )(22)

using the skew-symmetry of the matrices p̂eh1 , p̂eh2 , p̂eh12 ,p̂eh21 , p̂ee , Ṁ1 −2C1 and Ṁ2 −2C2 . Note that we have utilized
the fact that eR (kRij eξ̂θab ) = kRij eR (eξ̂θab ). Integrating (22)from 0 to T , we obtain T
uT νdτ = V (T ) − V (0) ≥ −V (0) = −β(23)0where β is a positive scalar that only depends on the initialstates of ξ1 , ξ2 , eh1 , eh2 , es12 , es21 and ee .
Remark 2: Lemma 1 would suggest that the synchronizedvisual feedback system is passive from the input u to the
output ν as in the definition in [15]. Note that Lemma 1 canbe shown to connect with the each passivity property of themanipulator dynamics, the estimation error system, the control
error system and the synchronization error system.
D. Stability Analysis for Synchronized Visual Feedback System
We now propose the following control input for the interconnected system:(24)
u = −Kν = −KN Ks x
K := diag{Kξ1 , Kξ2 , Kh1 , Kh2 , Ke }where Kξi := diag{kξi 1 , · · · , kξi n }, Khi := diag{khi 1 , · · · ,khi 6 } and Ke := diag{ke1 , · · · , ke6 } denote the positive gain
matrices.
Theorem 1: If w = 0, then the equilibrium point x = 0 for
the closed-loop system (19) and (24) is asymptotic stable.
Proof: In the proof of Lemma 1, we have already derived
that the time derivative of V along the trajectory of the system(19) is formulated as (22). Using the control input (24), (22)can be transformed into
(25)
V̇ = −xT Ks N T KN Ks x.
This completes the proof.
Theorem 1 guarantees the stability of synchronized control
using a Lyapunov function for the eye-to-hand synchronizedvisual feedback system (19) which is a highly nonlinear

system. It is interesting to note that stability analysis isbased on passivity as described in (20). For the tasks which
need the synchronous manner, the control performance shouldbe improved compared to the eye-to-hand non-synchronizedvisual feedback control law, because the synchronized control
design considers the mutual synchronization error. For thevisual feedback system, it is difficult that the desired trajectoryin joint space is given explicitly, since the motion of thetarget object is unknown. Although the synchronization error
is defined in the joint space in the previous works [6]–[10],we define it in the work space by making the reference
manipulator and the reference target objects as shown in
Fig 2. Additionally, L2 -gain performance analysis for thesynchronized visual feedback system (19) can be consideredbased on the dissipative systems theory in the presence ofdisturbances, similar to our previous works [4][12][13].
IV. S IMULATIONSIn this section, the validity of the proposed control law
can be confirmed by comparing the simulation results on two
2DOF manipulators with a static target object. In this paper, we
present simulation results for the synchronized visual feedback
control, compared with the non-synchronized one which is thestraightforward extension of the controller in [13]. The controller parameters for the control law u (24) were empiricallyselected. The coupling gains for mutual synchronization werechosen as kp12 = kR12 = kp21 = kR21 = 0.75. When we set
the coupling gains, it is important not to select too large values.
In the case of the non-synchronized control law, the same gains
are set in order to compare the both control laws simply, except
for the coupling gains kp12 = kR12 = kp21 = kR21 = 0. In thissimulation, we tested the system by adding a force disturbanceτ1d = [100 50]T [Nm] to only the manipulator 1 from 0.2–0.3[s] in order to confirm the advantage of the proposed law.
Figs. 3 and 4 illustrate the hand control errors eh1 (Dashed)and eh2 (Dotted), and the synchronization error es21 (Solid),

„
«3
2
−1
−1 T
T
T
T
−M
C
ξ
+
M
J
e
+
Ad
K
e
−
Ad
K
e
Ad
1
1
−1
−1
s
s
−1
s
s
h
1b
12
12
21
21
1
1
1
(gh d )
(gh o )
(gh o )
7
2
3 6
1 1
2 2
1 1
„
«7
6
ξ̇1
6
7
−1
−1
T
T
T
T
6 ξ̇2 7 6−M2 C2 ξ2 + M2 J2b Ad −1 eh2 + Ad −1 Ks21 es21 − Ad −1 Ks12 es12 7(gh d )
(gh o )
(gh o )
6 b 7 6
7
2
2
1
1
2
2
6 Veh 7 6
7
6
6
7
1 7
J
ξ
−Ad
−1
1
1b
(gh o )
6 V b 7=6
71
1
6 eh2 7 6
7−Ad(g−1 ) J2b ξ2
6V b 7 6
7h2 o2
6 es12 7 6
7
b
7
4Ves
5 6
−K
Ad
(J
ξ
−
J
ξ
)
−1
12
1
2
1b
2b
(gh o )
6
7
21
1 1
b
6
7
Vee
−K21 Ad(g−1 ) (J2b ξ2 − J1b ξ1 )
4
5
h2 o2

0
3
2 −1
3 2M −1 0
0
M1
0
0
0
0
1
−1
−1
6
7
0
6 0 M27 6 0 M2
0
0
0
7
6
7 6 0
7
0
I
0
−Ad(g−1 )
0
0
6 0
7 6
7
6
7 6 0
eh1
0
I
6 0
7 6
„
«7
7
0
0
−Ad(g−1 )07u++6
7w (19)
eh2
6
7 6
Ad
0
0
K
I
−
Ad
−1
−1
6
7
s
12
(g
)
(g
·g
)
6 0
7
ww1
h2 o
0
−K
0
h
o
s12 Ad(g −1 ) Ks12 Ad(g −1 ·g
6
1
)
6
7 6
„
«7
eh1
h1 o1 h2 d2
7
6
7 6
7
0 Ks21 Ad(g−1 ·g−Ks21 Ad(g−1 )
0
4 0
5 4 0
0 Ks21 Ad(gww
I − Ad(g−1 ·g ) 5−1)
h2 o2 h1 d1
eh2
)
h
o
h
o
2
1
2
0
0
0
0
−Ad(gee−1)00Ipelx [m]

0.1
0

-0.3
pely [m]

0.4

0

0.2

0.4

0.6

0

-0.4

V. C ONCLUSIONS
0

0.2

0

0.2

0.4

0.6

0.4

0.6

N^G elz [rad]

1
0

-1

time [s]Fig. 3. Control error eh1 (Dashed) and eh2 (Dotted), and the synchronization
error es21 (Solid) with the proposed control law.
pelx [m]

0.1
0

-0.3
pely [m]

0.4

0

0.2

0.4

0.6

0

0.2

0.4

0.6

0

0.2

0.4

0.6

N^G elz [rad]

1
0

-1

This paper considers synchronized visual feedback control
for eye-to-hand systems. In our proposed system, we can treat
not only the tracking problem but also the synchronized one,since it consists of the synchronization error system. Basedon passivity, stability analysis for the synchronized visualfeedback system is discussed. Finally simulation results areshown to verify the stability and the performance of the
proposed control law. The proposed control method which
poses no problem with respect to real-time computing can be
carried on in the experimental testbed, similar to [4][12].
R EFERENCES

0

-0.4

magnitude of the error for the whole system at the small
expense of the motion of the manipulators which are not added
the force disturbance, as a result, the synchronized visual feedback system becomes robust against the external disturbance
input. This validates one of the expected advantages of themutual synchronized control for the visual feedback system.

time [s]Fig. 4. Control error eh1 (Dashed) and eh2 (Dotted), and the synchronization
error es21 (Solid) with the non-synchronized control law.

respectively. In these figures, we focus on the errors of thetranslations of x and y and the rotation of z, because theerrors of the translation of z and the rotations of x and y are
zeros ideally on the defined coordinates. Figs. 3 and 4 denotethe errors applying the proposed synchronized control law andthe non-synchronized one, respectively.
In Figs. 3, the asymptotic stability applying the proposed
control law can be confirmed by steady state performance.
Moreover, it can be verified that es21 converges faster thanerrors of eh1 and eh2 in the synchronized visual feedback
control case in Fig. 3. This proves the mutual synchronization
behavior. Meanwhile, the errors eh1 and eh2 converge fasterthan es21 for the non-synchronized control case in Fig. 4.
Furthermore, Fig. 3 shows that the added force disturbancedoes not greatly affect the hand control error eh1 and the synchronization error es21 under the proposed synchronized control, compared with Fig. 4 in the case of the non-synchronized
control. It should be noted that the magnitude of the error
eh1 is specially small after adding the disturbance, since thecontrol law is designed by also considering the interaction
between manipulators. Our proposed controller reduces the[1] F. Chaumette and S. Hutchinson, “Visual Servoing and Visual Tracking,”
In:B. Siciliano and O. Khatib (Eds.), Springer Handbook of Robotics,
Springer-Verlag, pp. 563–583, 2008.
[2] G. Flandin, F. Chaumette and E. Marchand, “Eye-in-Hand/Eye-toHand Cooperation for Visual Servoing,” Proc. 2000 IEEE International
Conference on Robotics and Automation, pp. 2741–2746, 2000.
[3] V. Lippiello, B. Siciliano and L. Villani, “Position-based Visual Servoing
in Industrial Multirobot Cells using a Hybrid Camera Configuration,”
IEEE Trans. Robotics, Vol. 23, No. 1, pp. 73–86, 2007.
[4] T. Murao, H. Kawai and M. Fujita, “Passivity-based Control of VisualFeedback Systems with Dynamic Movable Camera Configuration,” Proc.
of the 44th IEEE Conference on Decision and Control and European
Control Conference, pp. 5360–5365, 2005.
[5] T. Murao, H. Kawai and M. Fujita, “Predictive Visual Feedback Control
with Eye-in/to-Hand Configuration via Stabilizing Receding Horizon
Approach,” Proc. of the 17th IFAC World Congress on AutomaticControl, pp. 5341–5346, 2008.
[6] A. Rodriguez-Angeles and H. Nijmeijer, “Mutual Synchronization of
Robots via Estimated State Feedback: A Cooperative Approach,” IEEE
Trans. Control Systems Technology, Vol. 12, No. 4, pp. 542–554, 2004.
[7] E. Kyrkjebø, K. Y. Pettersen, M. Wondergem and H. Nijmeijer, “Output
Synchronization Control of Ship Replenishment Operations: Theory andExperiments,” Control Engineering Practice, Vol. 15, No. 6, pp. 741–
755, 2007.
[8] S.-J. Chung and J.-J. E. Slotine, “Cooperative Robot Control and Concurrent Synchronization of Lagrangian Systems,” IEEE Trans. Robotics,
Vol. 25, No. 3, pp. 686–700, 2009.
[9] D. Sun and J. K. Mills, “Adaptive Synchronized Control for Coordination of Multirobot Assembly Tasks,” IEEE Trans. Robotics andAutomation, Vol. 18, No. 4, pp. 498–510, 2002.
[10] D. Sun, X. Shao and G. Feng, “A Model-Free Cross-Coupled Control
for Position Synchronization of Multi-Axis Motions: Theory and Experiments,” IEEE Trans. Control Systems Technology, Vol. 15, No. 2,
pp. 306–314, 2007.
[11] R. Murray, Z. Li and S. S. Sastry, A Mathematical Introduction to
Robotic Manipulation, CRC Press, 1994.
[12] M. Fujita, H. Kawai and M. W. Spong, “Passivity-based Dynamic VisualFeedback Control for Three Dimensional Target Tracking: Stability
and L2 -gain Performance Analysis,” IEEE Trans. Control Systems
Technology, Vol. 15, No. 1, pp. 40–52, 2007.
[13] H. Kawai, T. Murao and M. Fujita, “Passivity-based Control andEstimation of Dynamic Visual Feedback Systems with a Fixed Camera,”Proc. of the 43rd IEEE Conference on Decision and Control, pp. 4022–
4027, 2004.
[14] F. Bullo and A. D. Lewis, Geometric Control of Mechanical Systems,
Springer–Verlag, 2004.
[15] A. van der Schaft, L2 -Gain and Passivity Techniques in Nonlinear
Control (2nd ed.), Springer–Verlag, 2000.