Electronics and Communications in Japan, Vol. 92, No. 6, 2009
Translated from Denki Gakkai Ronbunshi, Vol. 128-C, No. 2, February 2008, pp. 286–294

Passivity-Based Control on Dynamic Visual Feedback Systems with Movable
Camera Configuration
TOSHIYUKI MURAO,1 HIROYUKI KAWAI,2 and MASAYUKI FUJITA3
1

Advanced Institute of Industrial Technology, Japan
2
Kanazawa Institute of Technology, Japan
3
Tokyo Institute of Technology, Japan

based one with an eye-in-hand configuration or a fixedcamera one [1, 2].
Recently, several camera configurations combining
each classical one have been proposed to overcome thedrawbacks which are that the eye-in-hand configuration
cannot have global view whereas the fixed camera one isdifficult to search throughout the workspace. Flandin andcolleagues [3] addressed an eye-in-hand and a fixed camera
cooperation approach where each camera information ispartitioned into the positioning task and the orientation one,respectively. Elena and colleagues [4] proposed a method
whereby the eye-in-hand camera executes the tracking of
the target and the fixed one determines the position of therobot arm. In Ref. 5, the occlusion problem has been tackled
by using multi eye-in-hand and fixed cameras.
On the other hand, instead of the use of additional
cameras, an eye-to-hand configuration which has a movable camera installed at some place other than the work
robot hand has also been proposed (including the fixedcamera configuration as a particular case). Actually, suchan eye-to-hand configuration using movable cameras allows effective visual servoing in complicated environments
where repetitive operations are performed, such as welding
or car coating [6]. There is also increasing demand forlaparoscopic surgery manipulators and other medical robots; in addition to manipulator position and orientation
control, such systems must provide for wide-area monitoring by a laparoscope or other device so as to maintain thefield of view [7].
Sharma and Hutchinson [6] introduced the control
method of a movable camera using motion perceptibility
which relates magnitude of the rate of change in an object’s
position to that in the image of that object. Nelson and
Khosla [8] described the movable camera control framework based on the camera’s field of view and other indicators. Muis and Ohnishi [9] proposed a method to gain
real-time visual servoing with a movable camera. However,

SUMMARY
This paper deals with control of dynamic visual feedback systems with a movable camera configuration. Thisconfiguration consists of a robot manipulator and a camerathat is attached to the end-effector of another robot manipulator. This system, which can be interpreted as the dynamic
visual feedback system with an eye-in-hand configuration
and a fixed camera one as a special case, can enlarge thefield of view. First, the dynamic visual feedback systemwith an eye-to-hand configuration is given with the fundamental representation of a relative rigid body motion. Second, we construct the dynamic visual feedback system witha movable camera configuration by combining the camera
control error system. Next, we derive the passivity of the
dynamic visual feedback system. Based on the passivity,
stability and L2-gain performance analysis are discussed.
Finally the validity of the proposed control law can be
confirmed by comparing the experimental results. © 2009
Wiley Periodicals, Inc. Electron Comm Jpn, 92(6): 1–11,
2009; Published online in Wiley InterScience (www.
interscience.wiley.com). DOI 10.1002/ecj.10091

Key words: visual feedback control; manipulator
dynamics; passivity; Lyapunov stability; L2-gain performance analysis.

1. Introduction
Visual feedback control has been used for rather
complicated systems in a wide application range, such as
factory automation and medical treatment. In classical visual servoing, many practical methods are reported by two
well-known approaches with two camera configurations,that is, position-based visual feedback control and image-© 2009 Wiley Periodicals, Inc.
1

these previous works [6, 8, 9] assume that the manipulator
dynamics is negligible and does not interact with the visualfeedback loop. Moreover, few results using movable cameras have been obtained for the stability of the closed-loopsystem and treated the tracking problem of moving target
objects.
In this paper, we consider a dynamic visual feedback
system using a movable camera as shown in Fig. 1. In
particular, we analyze the stability and control performanceof a system including one manipulator equipped with acamera (termed a camera manipulator), and another manipulator intended for actual operation (termed a work
manipulator). The system proposed in this paper includes,
as particular cases, the eye-in-hand configuration [10, 11]
and the fixed camera configuration [12] proposed previously by the authors.
The paper is organized as follows. In Section 2 we
explain a dynamic visual feedback system with the eye-tohand configuration (including the fixed camera configuration). Section 3 presents a dynamic visual feedback systemwith the movable camera configuration combined with anerror system for camera control. In addition, we propose acontrol law and perform an analysis of the stability andL2-gain performance. We report experimental results obtained with a 2-DOF manipulator in Section 4, and give asummary in Section 5.

Fig. 1. Visual feedback system with movable camera
configuration.

as the homogeneous representation of gij =^(pij, eξθij) ∈ SE(3) describing the configuration of a frame
Σj relative to a frame Σi. For example, the position andorientation in the hand frame Σh relative to the world frameΣw is gwh.
In our approach, gwh and gzc are obtained from thejoint angle of each manipulator, and gwz can be identified
because the bases of the manipulators are fixed. In addition,
gwc and gch can be obtained from the composition rules forthe homogeneous representation,* thus, the above positionand orientation data are known. On the other hand, gwo, gho,
and gco are unknown because the motion of the target object
is unknown.
In this section, we consider a visual feedback systemwith the eye-to-hand configuration. The control objective
is to always match the relative position and orientation ghoof the target object, relative to the hand frame, to its reference ghd. The relative position and orientation gho cannot be
obtained directly. Thus, we consider first the motion of thetarget object gco relative to the camera frame.
The relative position and orientation gco of the target
object, relative to the camera frame, can be expressed asfollows:

2. Dynamic Visual Feedback System with
Eye-to-Hand Configuration
2.1 Representation of rigid motion in visualfeedback system
In this paper, we consider a visual feedback system
including two manipulators (a camera manipulator and awork one) using five coordinate frames as shown in Fig. 1.
The five coordinate frames are: the world frame Σw basedon the work manipulator, the hand frame Σh, the base frameof the camera manipulator Σz, the camera frame Σc, and thetarget object frame Σo. Throughout this paper, we use the^notation eξθij ∈ R3×3 to represent the rotation matrix of aframe Σj relative to a frame Σi. ξij ∈ R3 specifies the
direction of rotation and θij ∈ R is the angle of rotation. For^θ to denote ξ^ θ . The notation “∧”
simplicity we use ξij
ij ij(wedge) is the skew-symmetric operator such thata^b = a × b for the vector cross-product × and any vector a,b ∈ R3, that is, a^ is a 3 × 3 skew-symmetric matrix. The
notation “∨” (vee) denotes the inverse operator to “∧,” thatis, so(3) → R3. Recall that a skew-symmetric matrix corresponds to an axis of rotation (via the mapping a D a^). We
use the 4 × 4 matrix(2)
In addition, the relative velocity of the target object, relative
to the camera frame, can be obtained as the time derivativeof Eq. (2):(3)
Here Vbij ∈ R6 is the body velocity of rigid motion, andAd(gij) ∈ R6 is the adjoint transformation of the homogeneous representation gij [13]. Equation (3) can be inter*

From the relations between the three coordinate frames,
gwc = gwzgzc, gch = g−1
wcgwh.

(1)2

preted as expressing the relative velocity Vbco of the target
object, relative to the camera frame, in terms of the difference between the camera velocity Vbwc and the object velocity Vbwo. Here Vbwc = 0 corresponds to a fixed camera. For thedetailed derivation, see Ref. 12.

2.3 Hand control error system
In this subsection, let us consider the hand control
error in order to achieve the control objective. The relative
position and orientation gho of the target object relative tothe hand frame cannot be obtained directly because _gco is
not available. Thus, we consider the estimated value gho*:

2.2 Nonlinear observer and estimation error
system(8)
Differentiating Eq. (8) with respect to time, the estimated body velocity from Σh to Σo can be obtained asfollows:

Here we consider the estimation error system. In thiscase, the image features are assumed to be acquired bymeans of a single pinhole camera. Although the visual
information f ∈ R2m obtained from the camera includes therelative position and orientation gco, the latter cannot be
obtained directly because the visual information is onlyis configured
two-dimensional.‡ Thus, a nonlinear observer_
in order to obtain the estimated value gco of the relative
position and orientation. Based_ on Eq. (3), the motion
model for the estimated value gco is defined as follows:

The hand control error geh can be_ defined as thedifference between the estimated value gho of the relative
position and orientation and the reference value ghd of thetarget object relative to the hand frame:

(4)

(10)

Here ue ∈ R6 is an additional input to stabilize the behaviorof the estimation error. The design of the input ue will be
explained later.
The error gee between actual relative position andorientation and its estimate is defined as

In addition, similarly to the estimation error vector
ee, the hand control error vector is defined as^
eh := [pTeh eTR(eξ θeh)]T.
Similarly to the estimation error system (7), the handcontrol error system can be obtained by taking the time
derivative of Eq. (10):

(9)

(5)

(11)

In addition, the estimation error vector is defined as^^^
ee := [pTeeeTR(eξθee)]T ∈ R6. Here eR(eξθ) := sk(eξθ)∨ ∈ R3,
^
^ξθ
^ξθ ∨
^
and sk(e ) := 1 / 2(e − e−ξθ); sk(eξθ) is a function thatextracts the skew-symmetric components of the rotation
matrix. Using the visual information f acquiredby the_camera, the estimated visual information f obtained fromthe_ nonlinear observer, and the pseudo-inverse matrix
J†(gco) ∈ R6×2m of the image Jacobian, the estimation error
vector ee can be expressed as follows [12]:Here Vbhd is the reference velocity of the target object gho
relative to the hand frame.
2.4 Dynamic visual feedback system with
eye-to-hand configuration
Here we configure a dynamic visual feedback systemof the eye-to-hand configuration with the work manipulator’s dynamics.
Assuming that the work manipulator has nh degrees
of freedom, its dynamics is expressed as follows [15]:(6)
If the estimation error vector ee is zero, then the estimate of
the relative position and orientation is equal to its actual
value.
The estimation error system is configured using thebasic representation of relative rigid body motion (3) andthe nonlinear observer (4). In particular, by taking the time
derivative of Eq. (5) and using Eqs. (3) and (4), the estimation error system is obtained as follows:

(12)
. .
Here qh, qh, qh ∈ Rnh are, respectively, the joint angle,
angular velocity, and angular acceleration of the work manipulator, τh ∈ Rnh is the input torque, τhd ∈ Rnh is thedisturbance input, Mh(qh) ∈ Rnh×nh is the inertia matrix,. .
Ch(qh, qh) qh ∈ Rnh expresses the Coriolis and centrifugal
forces, and gh(qh) ∈ Rnh is the gravity force.

(7)*

We assume that the camera is calibrated properly, and that−1gch = g−1
zc gwzgwh can be calculated accurately. In addition, the uncertaintyof the external parameters can be reduced by the estimation for the robot
hand aside from that for the target object [14].

‡Details of the camera model are given in Ref. 12. Here the image features
are acquired at m(m ≥ 4) feature points on the target object.

3

In addition, the velocity of the robot hand can be
expressed using the body manipulator JacobianJhb(qh) ∈ R6×nh as follows [13]:considered as an input, which means that the target object
might escape from the field of view. For example, in Ref.
12 we considered the fixed camera configuration, constituting a particular case of the eye-to-hand scheme at Vbwc = 0.
Usually, the fixed camera configuration is exploited in orderto provide a larger field of view than the eye-in-handscheme; however, a much wider field of view is required
when the target object moves greatly.
Thus, using the advantages of a movable camera, wedefine the camera velocity Vbwc as a system input so as tokeep the object within the field of view. As shown in Fig.
1, we consider control of a camera mounted on a manipulator hand. Also note that the results of comparative experimental verification of the movable camera configurationand the fixed camera configuration are given below in
Section 4.

(13)
On the other hand, denoting the reference hand velocity by.
uhd and the reference joint angular velocity by qhd, and using
the body manipulator Jacobian just as in the above expres.
sion, uhd = Jhb(qh)qhd can be written. The error of the joint
angular velocity of the work manipulator is defined as.
.
rh := qh − qhd ∈ Rnh. If this error rh is zero, the hand velocity
is equal to its reference value.
Now consider the following input torque applied to
the work manipulator:

(14)

..

3.1 Camera control error system

Here qhd is the reference value of the joint angular acceleration, and urh is an additional input to eliminate the error of
the joint angular velocity; its setting will be explained later.
Using Eqs. (7), (11), (12), and (14), the dynamic visualfeedback system with the eye-to-hand configuration can be
expressed as follows:

The control objective in this paper is always to keep
gho equal to ghd, while maintaining the relative position andorientation gco of the target object, relative to the camera
frame, at its reference value gcd so that the camera does not
lose the target object. However, as explained above, gco
cannot be obtained directly; thus, we use its estimated value_gco. Here we define_ the camera control error between theestimated value gco and the reference of the relative rigid
body motion gcd as(17)
In addition, the camera control error vector is defined as^
ec := [pTeceTR(eξθec)]T.
Just as in the hand control error system (11), we
derive the camera control error system by taking the time
derivative of Eq. (17):

(15)
where
(16)

(18)
where Vbcd is the desired body velocity of the relative rigid
body motion gco.

The control objective in the dynamic visual feedback system with the eye-to-hand configuration is to make the robot
hand follow the moving object. This objective is achieved
by maintaining zero states rh, eh, and ee.
(Note) Here camera motion is not discussed in thissection. When camera motion is ignored, the dynamic
visual feedback system with the eye-to-hand configuration(15) can be regarded as the dynamic visual feedback systemwith the fixed camera configuration [12].

3.2 Dynamic visual feedback system withmovable camera configuration
In this subsection, we consider the dynamics of thecamera manipulator in the same way as that of the work
manipulator. The dynamics of a camera manipulator withnc degrees of freedom is expressed as follows:

3. Dynamic Visual Feedback Control with Movable
Camera Configuration(19)

In the previous section, we considered dynamic visual feedback control with the eye-to-hand configuration. In
this system, however, the camera velocity Vbwc was not

It should be noted that all the symbols of the camera
manipulator have the same meaning as for the work one in
Eq. (12).

4

In addition, the velocity of the camera can be expressed using the body manipulator Jacobian
Jcb(qc) ∈ R6×nc as follows:

The movable camera configuration makes possible awider field of view than the fixed one, which offers a larger
robot workspace. In the previous work [12], the workspacecannot be expanded unless the camera is fixed at a position
providing a wide field of view. On the other hand, in thesystem proposed in this paper, the camera can track thetarget object at a short distance, and hence there is no needfor a wide view angle. As a result, the extraction of unnecessary features other than the target object can be avoided.
In addition, the visual feedback system with the movable
camera configuration without regard for manipulator dynamics can be developed by using only Eqs. (7), (11), and(18), which is suitable for such applications as mobile
robots. A block diagram of the dynamic visual feedback
system with the movable camera configuration is shown in
Fig. 2.

(20)
On the other hand, by denoting the reference camera veloc.
ity by ucd and reference joint angular velocity by qcd, andusing the body manipulator Jacobian just as in the above.
expression, ucd = Jcb(qc)qcd is obtained. The error of thejoint angular velocity of the camera manipulator is defined.
.
as rc := qc − qcd ∈ Rnc. Now consider the following input
torque applied to the camera manipulator:(21)..
where qcd is the reference value of the joint angular acceleration. Using Eqs. (15), (18), (19), and (21), the dynamic
visual feedback system with the movable camera configuration can be expressed as follows:

3.3 Passivity of dynamic visual feedback
systemHere we derive the properties of the dynamic visualfeedback system with the movable camera configuration inorder to propose a control law.
[Lemma 1] If w = 0, then the dynamic visual feedback
system (22) satisfies(24)
where n is defined as
(25)

(22)

where

(23)
The control objective in the dynamic visual feedback system with the movable camera configuration is always tomaintain the relative position and orientation of the target
object from the viewpoints of the camera and hand at theirrespective references. This objective is achieved when thestate x := [rTh rTc eTh eTc eTe ]T is kept at zero.

Fig. 2. Block diagram of dynamic visual feedback
system with movable camera configuration.

5and β0 is a positive scalar.
(Proof) Consider the following energy function:

diag{ke1, . . . , ke6} are the positive gain matrices of x, y, andz axes of the translation and the rotation for the hand control
error, the camera control one, and the estimation one,respectively.
Considering the passivity of the dynamic visual feedback system shown in Lemma 1, the following theorem canbe derived for stability.
[Theorem 1] If w = 0, then the equilibrium point x =0 for the closed-loop system (22) and (29) is asymptotic
stable.
(Proof) Treating energy function (26) as a Lyapunov
function candidate, time integration along the trajectory ofEqs. (22) and (29) gives the following with regard to Eq.
(27):

(26)
ξ^θij

ξ^θij

where E(gij) := 1 / 2||pij||2 + φ(e ) and φ(e ) := 1 / 2tr(I −^eξθij ) which is the error function of the rotation matrix [16].
Considering the skew-symmetry of the visual feedback
system and the manipulator dynamics, the time derivativeof the energy function V yields

(27)
(30)

Integration of both sides of the above expression gives

Since the gains Krh, Krc, Kh, Kc, and Ke are positive definite,
the matrix K is positive definite, the matrix N is nonsingular,
and the system equilibrium point x = 0 is asymptoticallystable.
"Thus, we have shown the stability via Lyapunov
method for the dynamic visual feedback system with the
movable camera configuration, which is composed of thework manipulator and the camera one. It is interesting tonote that stability analysis is based on the passivity as
described in Eq. (24).

(28)
where β0 is a nonnegative constant that depends only on theinitial state.
"
The above lemma suggests that the dynamic visualfeedback system (22) involves the passivity property. This
dynamic visual feedback system with the movable camera
configuration (22) can be treated as a dynamic visual feedback system with the eye-in-hand one so long as the work
manipulator is ignored. That is, the dynamic visual feedback system with the eye-in-hand configuration presented
in Refs. 10 and 11 is a particular case if the hand velocity
is Vbwh = 0, the reference value of the hand velocity is
uhd = 0, and the relative rigid body motion in the world
coordinates is gwz = I, while the hand control error eh isignored as well as the work manipulator. In addition, as
explained in Section 2, the case of the dynamic visualfeedback system with the fixed camera configuration [12]is also included. Therefore, the dynamic visual feedback
system with the movable camera configuration (22) includes, as particular cases, the two generally known dynamic visual feedback systems with the eye-in-handconfiguration and with the fixed camera one.

3.5 L2-gain performance analysis for dynamic
visual feedback system
Based on the dissipative systems theory, we consider
L2-gain performance analysis for the dynamic visual feedback system (22) in one of the typical problems, that is, thedisturbance attenuation problem.
In order to derive simple gain conditions, we redefine
Kc = kcI and Ke = keI where kc and ke are positive scalars. In
addition, we set the following conditions using a positive
scalar γ:(31)

3.4 Stability analysis for dynamic visualfeedback system

(32)
(33)

We now propose the following control input for theinterconnected system:

(34)
(35)

(29)
where Krh := diag{krh1, . . . , krhn h}and Krc := diag{krc1, . . . ,krcn c} denote the positive gain matrix for each joint axis of
the work manipulator and camera one, respectively. Kh :=diag{kh1, . . . , kh6}, Kc := diag{kc1, . . . , kc6}, and Ke :=

where
(36)

6

The following theorem can be established for control performance analysis.
[Theorem 2] Given a positive scalar γ and consider
the control input (29) with the gains Krh, Krc, Kh, kc, andke such that the inequalities (31) to (35) are satisfied, thenthe closed-loop system (22) and (29) has L2-gain ≤ γ.
(Proof) Consider the energy function V defined in Eq.
(26) as a storage function. Differentiating the positive definite function V along the trajectory of the closed-loopsystem and using the method of completing the squares
yield

(41)

Therefore, from the Schur complement, the necessary condition P ≥ 0 can be modified as the conditions (31) to (35)in Theorem 2.*"
In this control performance analysis, the motion of
the target object is treated as a disturbance; the influence ofthe object’s motion on the system is weaker with smaller γ.
In H∞-type control, we can consider some problems by
establishing the adequate generalized plant. This paper has
discussed L2-gain performance analysis for the disturbanceattenuation problem. The proposed strategy can be extended for the other type of generalized plants of the dynamic visual feedback systems.
4. Experimental Case Study(37)

4.1 Experimental setup

where W := diag{I, I, 0, 0, I}. Substituting control law (29),
we obtain

The experimental results on a 2-DOF manipulator(SICE-DD arm [15]) are shown in order to understand our
proposed method simply, though it is valid for 3D dynamic
visual feedback systems.
The manipulators and the coordinate frames areshown in Fig. 3. Five coordinate frames are defined for theleft manipulator (camera manipulator) equipped with thecamera, and right manipulator (work manipulator). Bothmanipulators are composed of Link 1 (0.2 m) and Link 2(0.3 m). The manipulators are controlled by a dSPACE
DS1005 digital control board (Power PC 750, 480 MHz).
The images are acquired by a Sony XC-HR57 CCD camera(60 fps), and imported by a PicPort-Stereo-H4D PCI-bus
frame grabber board (60 fps). Image processing is performed by the HALCON software; in particular, four object
points are calculated.
Because of space limitations, we omit discussion of
the experiments without disturbances used to verify Theorem 1. We describe the experiments with a moving target
object used to verify the L2-gain performance for various γ.
In addition, we compare the proposed method with theprevious method of Ref. 12 (visual feedback control withthe fixed camera configuration) in terms of field of view.
In the experiments, the target object moved in astraight line for 4 seconds, and then described a “figure 8”
motion for 5.6 seconds, as shown in Fig. 4. The initial

(38)
where
(39)
Assuming that P ≥ 0, integration of both sides gives(40)
Here β is a nonnegative constant that depends only on theinitial state.
Now P can be transformed as follows:*

Since the condition (31) to (35) is equivalent P > 0 and we assume theworst disturbance, these are only sufficient conditions.

7Fig. 3. Experimental arm (left: camera manipulator;
right: work manipulator).

settings were: pwh = [0.4732 0.1 0]T, ξθwh = [0 0 0]T, pwz =[0 0 −1.16] T, ξθwz = [0 0 0]T, pwc = [0.4732 0 −1.16] T, ξθwc
= [0 0 0]T. In addition, we set the following time-invariant^
desired position and orientation: phd = [0 0 −2]T, eξθhd = I,^
pcd = [0 0 −0.84]T, eξθcd = I. The error was assumed to be
0 at the initial state.

Fig. 5. Hand control error (left: Gain A; right: Gain B).

Comparative experimental results for the two gain
settings are presented in Figs. 5 to 7. Figures 5 and 6 show
the position errors of the x-axis and y-axis and the rotation
error of the z-axis in 3D space. Figure 5 represents the handcontrol error, and Fig. 6 represents the camera control error.
In addition, Fig. 7 shows the Euclidean norm ||x|| of the statex. The left and right plots in Figs. 5 and 6, and the top andbottom plots in Fig. 7, represent Gain A and Gain B,
respectively.
As indicated by the diagrams, Gain B with smaller γ
exhibits smaller errors and better follow-up performance,
which confirms the effectiveness of the proposed control
law.

4.2 Experimental results for L2-gain
performance analysis
In the experiments, the gains were designed as follows, considering the disturbance attenuation problem.
Step 1: The gain Kh for the hand control error and thegain kc for the camera control error are determined.
Step 2: The gain ke for the estimation error is set so
as to satisfy Eqs. (33) to (35).
Step 3: The gains Krh and Krc for the angular velocity
errors are set so as to comply with the given γ and Eqs. (31)and (32).
Actually, the gains were set as follows.
Gain A: γ = 0.388, kc = 10, ke = 20, Kh = diag{20, 20,
10, 10, 10, 20}, Krh = diag{5, 5}, Krc = diag{5, 5}Gain B: γ = 0.241, kc = 30, ke = 30, Kh = diag{50, 50,
25, 25, 25, 50}, Krh = diag{10, 10}, Krc = diag{10, 10}

Fig. 6. Camera control error(left: Gain A; right: Gain B).

Fig. 4. Trajectory of target object.

8

with Gain B and the fixed camera configuration. (The caseof the movable camera configuration with Gain A is omitted
here because of space limitations.) The range of fx and fy
shown in the diagrams corresponds to the actual measurement range provided by the camera in the experiments.
Comparison between the two configurations indicates thatthe feature points of the moving target object approach thelimits of the field of view in the case of a fixed camera, whilestaying in the middle area of the field of view in the case ofa movable camera. This means that the movable camera
tracks the target object properly.
Moreover, Nelson and Khosla proposed the following index to evaluate the camera’s field of view [17]:(42)Fig. 7. State norm (top: Gain A; bottom: Gain B).

Here [fxi fyi] are the coordinates of the feature points on theimage plane acquired by the camera, and [fxM fyM] represent
the respective maximum values. The closer the index is to1 which represents the minimum value, the nearer the target
object is to the image center. A high index means that thecamera is located so that the target object is not likely toescape from the field of view.
Figure 9 shows how the field of view index variedwith time in the experiments. The top and bottom diagrams
represent respectively the fixed camera configuration andthe movable camera configuration. The dashed and solidlines in the bottom diagram represent respectively Gain A
and Gain B. From the diagrams, we can verify that the fieldof view index with the proposed control law is less than thatwith the previous one. This result states that the target object
almost exists in the center of the camera, and the camera
can move not to miss the moving target object. Thus, we
may conclude that visual feedback control with the mov-

4.3 Experimental results for different camera
configurations
We also compared the field of views obtained by the
proposed movable camera configuration and by the previous fixed camera configuration. As explained in Section
3.3, the dynamic visual feedback system with the movable
camera configuration proposed in this paper includes theoretically, as a particular case, the previously proposed dynamic visual feedback system with the fixed camera
configuration. However, we have not yet considered thequantitative advantages of the movable camera configuration. Thus, we compared the two systems in terms of the
field of view. In the experiments, we used Gain A and Gain
B for the movable camera configuration, and ke, Kh, andKrh of Gain B for the fixed camera configuration.
Figure 8 shows the measured trajectories of fourfeature points of the target object: The left and right sides
represent respectively the movable camera configuration

Fig. 9. Field of view index [top: fixed camera; bottom:
movable camera (dashed line: Gain A;
solid line: Gain B)].

Fig. 8. Trajectory of feature points [left: movable
camera (Gain B); right: fixed camera].

9able camera configuration offers a wider field of view than
the fixed camera configuration.

6. Sharma R, Hutchinson S. Motion perceptibility andits application to active vision-based servo control.
IEEE Trans Robotics Automation 1997;13:607–617.
7. Shimoyama I et al. Robot frontier. Iwanami Shoten;
2005. (in Japanese)
8. Nelson BJ, Khosla PK. Visually guided manipulation
using active camera-lens systems. Proc 1997 IEEE
International Conference on Robotics and Automation, p 437–442.
9. Muis A, Ohnishi K. Eye-to-hand approach on eye-inhand configuration within real time visual servoing.
IEEE/ASME Trans Mechatron 2005;10:404–410.
10. Kawai H, Azuma T, Fujita M. Observer-based dynamic visual feedback control of robotic systems.
Trans Inst Syst Control Inf Eng 2004;17:39–47. (inJapanese)
11. Fujita M, Kawai H, Spong MW. Passivity-based dynamic visual feedback control for three dimensionaltarget tracking-stability and L2-gain performance
analysis. IEEE Trans Control Syst Technol
2007;15:40–52.
12. Kawai H, Azuma T, Fujita M. Passivity-based 3D
dynamic visual feedback control with a fixed camera.
Trans Inst Syst Control Inf Eng 2005;18:127–135. (inJapanese)
13. Murray R, Li Z, Sastry SS. A mathematical introduction to robotic manipulation. CRC Press; 1994.
14. Kawai H, Murao T, Fujita M. Passivity-based dynamic visual feedback control with uncertainty ofcamera coordinate frame. Proc 2005 American Control Conference, p 3701–3706.
15. Kobayashi H et al. Practical control of robot manipulators. Corona; 1997. (in Japanese)
16. Bullo F, Lewis AD. Geometric control of mechanicalsystems. Springer-Verlag; 2004.
17. Nelson BJ, Khosla PK. Integrating sensor placement
and visual tracking strategies. Proc 1994 IEEE International Conference on Robotics and Automation, p1351–1356.

5. Conclusions
In this paper, we considered a 3D dynamic visualfeedback system with the movable camera configuration.
Aiming at expansion of the robot’s workspace, the proposed system which utilizes a movable camera includes, as
particular cases, the eye-in-hand configuration and thefixed camera configuration. Stability and L2-gain performance analysis for the dynamic visual feedback system havebeen discussed based on passivity with the energy function.
We also carried out experiments to verify L2-gain performance analysis and the effectiveness of the movable camera
configuration.
REFERENCES
1. Hutchinson S, Hager GD, Corke PI. A tutorial on
visual servo control. IEEE Trans Robotics Automation 1996;12:651–670.
2. Chaumette F, Hutchinson S. Visual servo control, Part
I: Basic approaches. IEEE Robotics Automation Mag
2006;13:82–90.
3. Flandin G, Chaumette F, Marchand E. Eye-inhand/eye-to-hand cooperation for visual servoing.
Proc 2000 IEEE International Conference on Robotics and Automation, p 2741–2746.
4. Elena M, Cristiano M, Damiano F, Bonfe M. Variable
structure PID controller for cooperative eye-inhand/eye-to-hand visual servoing. Proc 2003 IEEE
International Conference on Control Applications, p989–994.
5. Lippiello V, Siciliano B, Villani L. Eye-in-hand/eyeto-hand multi-camera visual servoing. Proc 44th
IEEE Conference on Decision and Control and European Control Conference, p 5354–5359, 2005.

10

AUTHORS (from left to right)

Toshiyuki Murao received his B.E. degree from the Department of Electrical and Computer Engineering and M.E. degreefrom the Division of Electronics and Computer Science at Kanazawa University in 2003 and 2005, and D.Eng. degree from the
Department of Mechanical and Control Engineering at Tokyo Institute of Technology in 2008. Since 2006, he has been an
assistant professor at the Advanced Institute of Industrial Technology. His research focuses on the passivity-based visualfeedback control of robot systems.
Hiroyuki Kawai received his B.S. degree from the Department of Electrical and Computer Engineering and M.S. and
Ph.D. degrees from the Graduate School of Natural Science and Technology at Kanazawa University in 1999, 2001, and 2004.
From 2004 to 2005, he was a postdoctoral scholar at the Information Technology Research Center, Hosei University. He has
been an assistant professor at Kanazawa Institute of Technology since 2005. His research focuses on the visual feedback controlof robot systems and the networked cellular vision system. In particular, he is interested in the passivity-based control. He
received the IEEE Transactions on Control Systems Technology Outstanding Paper Award from the IEEE Control Systems
Society (CSS) in 2008.
Masayuki Fujita received his B.E., M.E., and D.Eng. degrees in electrical engineering from Waseda University in 1982,
1984, and 1987. From 1985 to 1992, he was with the Department of Electrical and Computer Engineering at Kanazawa
University. He was on the faculty of Japan Advanced Institute of Science and Technology as an associate professor from 1992
to 1998 and Kanazawa University as a professor in 1999. Since 2005, he has been a professor in the Department of Mechanicaland Control Engineering, Tokyo Institute of Technology. From 1994 to 1995, he held a visiting position in the Department ofAutomatic Control Engineering, Technical University of Munich, Germany. His research interests include robust control andits applications, cooperative control, and passivity-based visual feedback. He will serve as the General Chair of the 2010 IEEE
Multi-conference on Systems and Control. He currently serves as a member of the board of governors of the IEEE Control
Systems Society, an Editor of the SICE Journal of Control, Measurement, and System Integration, and an Associate Editor ofAutomatica. He also served as a director of SICE in 2006 and Associate Editor of the IEEE Transactions on Automatic Control
and Asian Journal of Control from 2005 to 2008. He received the 2008 IEEE Transactions on Control Systems Technology
Outstanding Paper Award, the Best Paper Award from the Society of Instrument and Control Engineers in 1997, and from theInstitute of Systems, Control and Information Engineers (ISCIE) in 2000.

11