Predictive Visual Feedback Control with Eye-in-Hand System
via Stabilizing Receding Horizon Approach
Toshiyuki Murao, Teruki Yamada and Masayuki Fujita
Abstract— This paper investigates vision based robot control
based on a receding horizon control strategy, as a first step fora predictive visual feedback control. Firstly, the brief summaryof the 3D dynamic visual feedback system with eye-in-handconfiguration is given. Next, a stabilizing receding horizon
control for the 3D dynamic visual feedback system, a highly
nonlinear and relatively fast system, is proposed. The stabilityof the receding horizon control scheme is guaranteed by using
the terminal cost derived from an energy function of the visualfeedback system. Furthermore, simulation results are assessedwith respect to the stability and the performance.

I. I NTRODUCTION
Robotics and intelligent machines need sensory information to behave autonomously in dynamical environments. Visual information is particularly suited to recognize unknown
surroundings. In this sense, vision is one of the highest
sensing modalities that currently exist. Vision based controlof robotic systems involves the fusion of robot kinematics,
dynamics, and computer vision to control the motion of therobot in an efficient manner. The combination of mechanicalcontrol with visual information, so-called visual feedback
control or visual servoing, is important when we consider amechanical system working in dynamical environments [1],[2].
In previous works, for the problem of three dimensional(3D) visual servo control, Kelly et al. [3] considered animage based controller under the assumption that the objects’
depths are known. Cowan et al. [4] addressed the fieldof-view problem for 3D dynamic visual feedback systemusing navigation functions. Although good solutions to the
set-point problem are reported in those papers, few results
have been obtained for the tracking problem of moving target
objects in the full 3D dynamic visual feedback system that
include not only the position and the orientation but also
the manipulator dynamics. The authors discussed passivity
based control for a moving target object in 3D workspacewith eye-in-hand configuration [5], [6]. However, the control
law proposed in [5] is not based on optimization, the desired
control performance cannot be guaranteed explicitly.
Receding horizon control, also recognized as model predictive control is a well-known control strategy in which
the current control action is computed by solving, a finite
horizon optimal control problem on-line [7]. A large number
T. Murao is with the Master Program of Information Systems Architecture, Advanced Institute of Industrial Technology, Tokyo 140-0011, JAPANT. Yamada and M. Fujita are with the Department of Mechanicaland Control Engineering, Tokyo Institute of Technology, Tokyo 152-8552,
JAPAN fujita@ctrl.titech.ac.jp

Camera5c

Robot
Manipulator

Image
gwc=
gco=
^ wc)
^ ) (pwc,eNG(pco,eNGco^
NG
5w
5o gwo=(pwo,e wo)
Target Object
World Frame
Fig. 1.

Eye-in-hand visual feedback system.

of industrial applications using model predictive control canbe found in chemical industries where the processes haverelatively slow dynamics. On the contrary, for nonlinear andrelatively fast systems such as in robotics, few implementations of the receding horizon control have been reported.
For the receding horizon control, many researchers havetackled the problem of stability guarantees. An approach
proposed by Parisini et al. [8] is based on using a quadratic
endpoint penalty of the form axT (t + T )P x(t + T ) forsome a > 0, some positive definite matrix P and a terminal
state x(t + T ). Jadbabaie et al. [9] showed that closedloop stability is ensured through the use of a terminal
cost consisting of a control Lyapunov function. Moreover,these results were applied to the Caltech Ducted Fan toperform aggressive maneuvers [10], [11]. Visual feedback,however, is not considered here. Predictive control could
be of significant benefit when used in conjunction withvisual servoing. With the incorporation of visual information,
the system could anticipate the target’s future position andbe waiting there to intercept it [12]. In [13], the authors
proposed stabilizing receding horizon control for the planar
visual feedback system. However, the visual feedback system
proposed in [13] is restricted to a planar manipulator, and thismethod can only treat the desired position problem.
In this paper, as a first step for a predictive visual feedback
control, stabilizing receding horizon control is applied to the3D visual feedback system in [5], a highly nonlinear andrelatively fast system. Compared with previous work [13],this 3D dynamic visual feedback system can treat not only
the position but also the orientation, so the possible application area should be undoubtedly increasing. This represents afirst step towards high performance visual servoing targeting
more aggressive maneuvers. The main idea is the use of theterminal cost derived from an energy function of the visualfeedback system.
Firstly, the brief summary of our prior work [5], which isthe 3D dynamic visual feedback system, is given. Next, a stabilizing receding horizon control for the 3D visual feedback
system using a control Lyapunov function is proposed. Then,the control performance of the stabilizing receding horizon
control scheme is evaluated through simulation results.
Throughout this paper, we use the notation eξ̂θab ∈ R3×3
to represent the change of the principle axes of a frame Σb
relative to a frame Σa . ξab ∈ R3 specifies the direction ofrotation and θab ∈ R is the angle of rotation. For simplicity
we use ξ̂θab to denote ξˆab θab . The notation ‘∧’ (wedge) is the
skew-symmetric operator such that ξ̂θ = ξ × θ for the vector
cross-product × and any vector θ ∈ R3 . The notation ‘∨’(vee) denotes the inverse operator to ‘∧’, i.e., so(3) → R3 .
Recall that a skew-symmetric matrix corresponds to an axisof rotation (via the mapping a → â). We use the 4×4 matrixeξ̂θab pab
(1)
gab =
0
1
as the homogeneous representation of gab = (pab , eξ̂θab ) ∈SE(3) describing the configuration of a frame Σb relative toa frame Σa . The adjoint transformation associated with gab
is denoted by Ad(gab ) [14].
II. T HREE D IMENSIONAL DYNAMIC V ISUAL F EEDBACKS YSTEM WITH E YE - IN -H AND C ONFIGURATIONIn this section, the brief summary of our prior work in [5]is given. An energy function and a stabilizing control law,which play an important role for a predictive visual feedback
control, are derived.
A. Basic Representation for Visual Feedback System
The visual feedback system considered in this paper has
the camera mounted on the robot’s end-effector as depicted
in Fig. 1, where the coordinate frames Σw , Σc and Σorepresent the world frame, the camera (end-effector) frame,
and the object frame, respectively. Then, the relative rigid
body motion from Σc to Σo can be represented by gco .
Similarly, gwc and gwo denote the rigid body motions fromthe world frame Σw to the camera frame Σc and from theworld frame Σw to the object frame Σo , respectively, as
shown in Fig. 1.
The objective of visual feedback control is to bring theactual relative rigid body motion gco to a given reference gd .
The reference gd for the relative rigid body motion gco is
assumed to be constant throughout this paper, because thecamera can track the moving target object in this case.
The relative rigid body motion from Σc to Σo can be led
by using the composition rule for rigid body transformations([14], Chap. 2, pp. 37, eq. (2.24)) as follows:−1
gco = gwc
gwo .

(2)

The relative rigid body motion involves the velocity of eachrigid body. To this aid, let us consider the velocity of a rigid
body as described in [14]. We define the body velocity of theb
T
T T
camera relative to the world frame Σw as Vwc= [vwcωwc] ,where vwc and ωwc represent the velocity of the origin
and the angular velocity from Σw to Σc , respectively ([14]Chap. 2, eq. (2.55)).
Differentiating (2) with respect to time, the body velocityof the relative rigid body motion gco can be written as follows
[5].
b
b
b
−1 V
= −Ad(gco
Vco) wc + Vwo(3)bwhere Vwois the body velocity of the target object relativeto Σw . We consider that it is the basic representation for
the three coordinate frames of the visual feedback system.
Roughly speaking, the relative rigid body motion gco willb
depend on the difference between the camera velocity Vwcbb
and the target object velocity Vwo , because Vco is defined as
the body velocity of the relative rigid body motion gco .

B. Estimation Error and Control Error Systems
The visual feedback control task requires information of
the relative rigid body motion gco . Since the measurable
information is only the image information f (gco ) in thevisual feedback system, we consider a nonlinear observer
in order to estimate the relative rigid body motion from theimage information f (gco ).
Firstly, using the basic representation (3), we choosebestimates ḡco and V̄coof the relative rigid body motion andvelocity, respectively as
b
V̄co

=

b
−1 V
−Ad(ḡco) wc + ue ,(4)

where ue is the new input in order to converge the estimatedvalue to the actual relative rigid body motion.
In order to establish the estimation error system, we define
the estimation error between the estimated value ḡco and theactual relative rigid body motion gco as−1
gco .
gee = ḡco(5)

Using the notation eR (eξ̂θ ) in [15], the vector of the estimation error is given by ee := [pTee eTR (eξ̂θee )]T . Note thatee = 0 iff pee = 0 and eξ̂θee = I3 . Therefore, if the vectorof the estimation error is equal to zero, then the estimatedrelative rigid body motion ḡco equals the actual relative rigid
body motion gco . The estimation error vector ee can beobtained from image information f (gco ) and the estimatedvalue of the relative rigid body motion ḡco . In the same way
as the basic representation for the visual feedback system,
the estimation error system can be represented bybb−1 ue + V
= −Ad(gee
Vee
wo .
)

(6)

Moreover, let us consider the dual of the estimation error
system, which we call the control error system, in order toestablish the visual feedback system. We define the control
error as follows:
gec = gd−1 ḡco ,(7)

which represents the error between the estimated value ḡcoand the reference of the relative rigid body motion gd . The
vector of the camera control error is defined as ec :=Visual Feedback System[pTec eTR (eξ̂θec )]T . The camera control error system can be
described bybb−1 V= −Ad(ḡcoVec) wc + ue .

u(8)

Combining (6) and (8), we construct the visual feedback
system as follows:  b 
−1−Ad(ḡco
I
0 b
Vec
)
Vwo
(9)
uce +
b =
−1
0
−Ad(gee
I
Vee
)

N

Vecb
Veeb

The manipulator dynamics can be written as

uk

(10)

where M ∈ R
is the inertia matrix, C ∈ R
is theCoriolis matrix, g ∈ Rn is the gravity terms, and q, q̇ and q̈
are the joint angle, velocity and acceleration, respectively.
τ is the vector of the input torque, and τd represents a
disturbance input [16]. Now, we will construct a dynamic
visual feedback system by connecting the visual feedback
system (9) and the manipulator dynamics (10). Since thecamera is mounted on the end-effector of the manipulator
in the eye-in-hand configuration, the body velocity of theb
is given bycamera Vwc
n×n

b
Vwc
= Jb (q)q̇Fig. 2.

KNx

Block diagram of the passivity based visual feedback control.

D. Energy Function and Stabilizing Control Law

M (q)q̈ + C(q, q̇)q̇ + g(q) = τ + τd
n×n

u

+

Passivity-based Control Law(Stabilizing Control Law)b T
) uTe ]T denotes the control input.
where uce := [(Vwc

C. Dynamic Passivity based Visual Feedback System with
Eye-in-hand Configuration

x

(11)

where Jb (q) is the manipulator Jacobian [14].
Next, we propose the control law for the manipulator asτ = M (q)q̈d +C(q, q̇)q̇d +g(q)+JbT (q)AdT(g−1 ) ec +uξ . (12)dwhere q̇d and q̈d represent the desired joint velocity and acceleration, respectively. The new input uξ is to be determinedin order to achieve the control objective.
Let us define the error vector with respect to the joint
velocity of the manipulator as ξ := q̇ − q̇d . Moreover, we
design the reference of the joint velocity based on the relation
between the camera velocity and the joint velocity (11) asq̇d := Jb† (q)ud where ud is the desired body velocity of
the camera which will be obtained from the visual feedbackbin (8) should be replaced by ud .
system. Thus, Vwc
Using (9)–(12), the visual feedback system with the manipulator dynamics (we call the dynamic visual feedback
system) can be derived as follows:⎤⎤ ⎡⎡−M −1 Cξ + M −1 JbT AdT(g−1 ) ec
ξ̇
d
⎥
b ⎦=⎢
⎣ Vec
−1 Jb ξ−Ad(ḡco
⎦
⎣
)
b
Vee
0
⎡ −1
⎤ ⎡ −1 ⎤
M
0
0
0
M
⎦
⎣
−1
I
0
0 ⎦w (13)+⎣ 0 −Ad(ḡcou+)
00 −Ad(gee−10I)
where u := [uTξ uTd uTe ]T . We define the state and the
disturbance of dynamic visual feedback system as x :=b T T
[ξ T eTc eTe ]T and w := [τdT (Vwo) ] , respectively.

In previous work [5], the passivity of the visual feedback
system (13) is derived by using the following energy function
V (x)V (x) =1 T
ξ M (q)ξ + E(gec ) + E(gee )2(14)where E(g) := 12 p2 + φ(eξ̂θ ) and φ(eξ̂θ ) := 12 tr(I − eξ̂θ )is the error function of the rotation matrix (see e.g. [15]).
Here, we consider the following control input
u = −Kν := uk , K > 0,
⎡
I
0
T
⎢
ν := N x := ⎣ 0 −Ad(g−1 )d0AdT(eξ̂θec )

⎤

0
0 ⎥
⎦ x.
−I

(15)
(16)

By differentiating V (x) along the trajectory of the system
and using the control input uk , the next equation is obtained
if w = 0 is satisfied.
V̇ = ν T u = −xT N T KN x.

(17)

Therefore, if w = 0, the equilibrium point x = 0 for theclosed-loop system (13) and (15) is asymptotic stable, i.e.,
uk is a stabilizing control law for the system. The block
diagram of the passivity based visual feedback control isshown in Fig. 2.
However, the stabilizing control law uk (15) is not basedon optimization, the desired control performance cannot
be guaranteed explicitly. In the next section, a stabilizing
receding horizon control based on optimal control theory is
proposed.
III. P REDICTIVE V ISUAL F EEDBACK C ONTROLThe objective of this section is to propose a predictive
visual feedback control based on optimal control theory.
A camera can provide more information than the current
derivation from a nominal position at the sample instant.
This property can be exploited to predict the target’s future
position and improve the control performance. As a firststep for a predictive visual feedback control, we proposea stabilizing receding horizon control based on optimizationin this paper.

A. Control Lyapunov Function
In this section, the finite horizon optimal control problem(FHOCP) for the visual feedback system (13) is considered.
The FHOCP for the visual feedback system (13) at time
t consists of the minimization with respect to the inputu(τ, x(τ )), τ ∈ [t, t + T ], of the following cost function
t+T

J(x0 , u, T )=l(x(τ ), u(τ ))dτ + F (x(t + T ))

(18)

t

l(x(t), u(t)) = qξ (t)ξ(t)2 + Eqc (gec (t)) + Eqe (gee (t))(19)+uT (t)R(t)u(t)F (x) = ρV (x)(20)
qξ (t)≥0, qp (t)≥0, qR (t)≥0, R(t) > 0, ρ > 0,
where Eq (g(t)) := qp (t)p(t)2 + qR (t)φ(eξ̂θ(t) ), with thestate x(t) = x0 . The speciality of the cost function (18)–(20) is that the terminal cost is derived from an energyfunction of the visual feedback system. Furthermore, therotation error related part of the stage cost is derived from theerror function φ(eξ̂θ ) instead of the commonly used quadratic
form eR (eξ̂θ )2 . For a given initial condition x0 , we denote
this solution of the FHOCP as u∗ (τ, x(τ )), τ ∈ [t, t + T ].
In receding horizon control, at each sampling time δ, theresulting feedback control at state x0 is obtained by solving
the FHOCP and settinguRH := u∗ (δ, x0 ).

(21)

The following lemma concerning a control Lyapunov
function is important to prove a stabilizing receding horizon
control. The definition for a control Lyapunov function S(x)is given by
inf Ṡ(x) + l(x, u) ≤ 0,
u(22)

where l(x, u) is a positive definite function [9].
Lemma 1: Suppose that w = 0, θec ≤ π2 , θee ≤ π2 andthe design parameter ρ satisfiesρ2 T T −1
2− x N R N x + qξ ξ + Eqc (gec ) + Eqe (gee )4
ρ2 T T −1= − x N R N x + qξ ξ2 + qpc pec 2
4+qRec φ(eξ̂θec ) + qpe pee 2 + qRee φ(eξ̂θee )ρ2≤ − xT N T R−1 N x + qξ ξ2 + qpc pec 242
T ξ̂θee+qReceTR (eξ̂θec ) + qpe p)ee  + qRee eR (e2ρ T −1T
N R N − Q x,
= −x
(24)
4
where we have used the fact that φ(eξ̂θ )≤eTR (eξ̂θ ) for allθ≤ π2 . Therefore, the condition inf u [Ṡ(x)+l(x, u)]≤0 will
be satisfied, if the assumption ρ2 I≥4QN −1 RN −T .
Lemma 1 shows the energy function ρV (x) of the visualfeedback system (13) can be regarded as a control Lyapunov
function in the case of ρ2 I≥4QN −1 RN −T .
B. Stabilizing Receding Horizon Control for the 3D VisualFeedback System
Suppose that the terminal cost is the control Lyapunov
function ρV (x), the following theorem concerning the stability of the receding horizon control holds.
Theorem 1: Consider the cost function (18)–(20) for thevisual feedback system (13). Suppose that w = 0, θec ≤ π2 ,θee ≤ π2 , and ρ2 I≥4QN −1 RN −T , then the receding horizon control for the visual feedback system is asymptoticallystabilizing.
Proof: Our goal is to prove that J(x∗ (t), uRH , T ),which is the cost-to-go applying the receding optimal control
uRH , will qualify as a Lyapunov function for the closed loop
system. Construct the following suboptimal control strategy
for the time interval [t + δ, t + T + δ] ∗
τ ∈ [t + δ, t + T ]u (τ )(25)ũ =uk (τ ) = − ρ2 R−1 N x τ ∈ [t + T, t + T + δ]

(23)

where uk is the stabilizing control law (15) with K = ρ2 R−1
for the visual feedback system. The associated cost iswhere Q := diag{qξ In , qpc I3 , qRc I3 , qpe I3 , qRe I3 }. Then,the energy function ρV (x) of the visual feedback system(13) can be regarded as a control Lyapunov function.

J(x∗ (t + δ), ũ, T )= J(x(t), u∗ , T ) + ρ[V (x(t + T + δ)) − V (x∗ (t + T ))]ρ2 I≥4QN −1 RN −T ,Proof: In [5], we have already shown that the time
derivative of V along the trajectory of the system (13)is formulated as (17). Using the positive definite functionl(x(t), u(t)) (19) and the stabilizing control law uk (15) with
K = ρ2 R−1 for the system, Equation (22) can be transformed
into
inf [Ṡ(x) + l(x, u)]u= inf ρV̇ + qξ ξ2 + Eqc (gec ) + Eqe (gee ) + uT Ru
u= inf ρxT N T u + qξ ξ2 + Eqc (gec ) + Eqe (gee ) + uT Ru
u 

T 

ρ
ρ
= inf u + R−1 N x R u + R−1 N x
u
2
2

t+δ

−

l(x∗ (τ ), u∗ )dτ +tt+T +δ

l(x∗ (τ + T ), uk )dτ,

t+T

(26)

where x∗ is the optimal state trajectory. This cost, which isan upper bound for J(x∗ (t + δ), u∗ , T ), satisfies
J(x∗ (t + δ), u∗ , T ) − J(x∗ (t), u∗ , T )≤ ρ[V (x(t + T + δ)) − V (x∗ (t + T ))]
t+δ

−
t

l(x∗ (τ ), u∗ )dτ +t+T +δ

t+Tl(x∗ (τ + T ), uk )dτ.
(27)

Using the positive definite function l(x(t), u(t)) (19) and thestabilizing control law uk (15) for the system, and dividing

JointVisual Feedback System

u

N

Vecb
Veeb

+

q AngleuxWorld Frame

zw

xw

5w
yw

yc 5c

Receding Horizon Control Law

uRH
Fig. 3.

xc

zc

Block diagram of the predictive visual feedback control.

both sides by δ and taking the limit as δ → 0, Equation (27)can be transformed intoJ(x∗ (t + δ), u∗ , T ) − J(x∗ (t), u∗ , T )
δ→0
δ
ρ2 ∗T
T −1
≤ − x (t + T )N R N x∗ (t + T )4∗∗+qξ ξ ∗ (t + T )2 + Eqc (gec(t + T )) + Eqe (gee(t + T ))∗T∗∗T∗−x (t)Qx (t)−2u Ruρ T −1N R N − Q x∗ (t + T )≤ − x∗T (t + T )4(28)−x∗T (t)Qx∗ (t) − u∗T Ru∗ .
limConsidering that the control input during first δ is uRH = u∗ ,by the assumption ρ2 I≥4QN −1 RN −T , the derivative ofJ(x∗ (t), uRH , T ) is negative definite. Therefore, we haveshown that J(x∗ (t), uRH , T ) qualifies as a Lyapunov function and asymptotic stability is guaranteed.
Theorem 1 guarantees the stability of the receding horizon control using a control Lyapunov function for the 3D
visual feedback system (13) which is a highly nonlinear
and relatively fast system. Since the stabilizing receding
horizon control design is based on optimal control theory,the control performance should be improved compared tothe simple passivity based control [5], under the conditionof adequate gain assignment in the cost function. It should
be noted that the error function φ(eξ̂θ ) of the rotation
matrix can be directly used in the stage cost (19). Moreover,compared with previous work [13], the main advantage ofthis approach is that the 3D dynamic visual feedback system
is not restricted to a planar manipulator, and can treat not
only the position but also the orientation. This allows us
to extend the technological application area. In this paper,
as a first step, we propose unconstrained stabilizing receding
horizon control schemes. In the near future, we will consider
constraints which represent one of the advantages of receding
horizon control, and develop it using level set, see [9]. The
assumptions θec ≤ π2 and θee ≤ π2 will be considered byusing constraints. The block diagram of the predictive visualfeedback control is shown in Fig. 3.
In the next section, the stabilizing receding horizon control
is applied to a 3D visual feedback system. It is expectedthat the control performance is improved using the receding
horizon control.

Camera Frame

xo

zo
5o

Object FrameyoFig. 4. Coordinated frames for 3D dynamic visual feedback systems with
2DOF manipulator.

IV. S IMULATIONSIn this section, the validity of the proposed control law canbe confirmed by comparing the simulation results. Although
we only discuss simulation results in this paper, it should be
noted that the model parameters were developed using actual
visual servo system.
The simulation results on 2DOF manipulator as depicted
in Fig. 4 are shown in order to understand our proposed
method simply, though it is valid for 3D dynamic visualfeedback systems. We defined the three coordinates whichwere described in Fig. 4. The target object has four featurepoints. The control objective is to bring the actual relative
rigid body motion gco to a given reference gd . In thispaper, we present results for the stability analysis with astatic target object. The simulation is carried out with theinitial condition q1 (0) = π/6 [rad], q2 (0) = −π/6 [rad],pwo = [0.3986 0 − 0.9]T [m], ξθwo = [0 0 − 0.5087]T[rad], pwc = [0.4732 0.1 0]T [m], ξθwc = [0 0 0]T [rad]. The
desired relative rigid body motion gd is pd = [0 0 − 0.9]T[m], ξθd = [0 0 0]T [rad], and the initial error condition x(0)is ξ(0) = [0 0]T [rad/s], pec (0) = [−0.0746 − 0.1 0]T [m],ξθec (0) = [0 0 − 0.5077]T [rad], pee (0) = [0 0 0]T [m],ξθee (0) = [0 0 − 0.001]T [rad] in this simulation.
In this simulation, we compare the performance of thereceding horizon control law proposed in Theorem 1 andthe passivity based control law uk (15). The weights ofthe cost function (18)–(20) were selected as qξ = 0.001,
qpc = 0.003, qRc = 0.001, qpe = 0.0003, qRe = 0.0001,
R = diag{0.1, 3.2, 0.07, 50, 15, 15, 15, 50, 300, 300, 30, 30,
30, 300} and ρ = 1 satisfy ρ2 I≥4QN −1 RN −T .
The controller parameters for the passivity based control law uk (15) were empirically selected as K =diag{10, 1, 2, 1, 0.2, 0.2, 0.2, 1, 1, 1, 1, 1, 1, 1}. To solve thereal time optimization problem, the software C/GMRES [17]is utilized. The control input with the receding horizon
control is updated every 20 [ms]. It must be calculated by thereceding horizon controller within that period. The horizon
was selected as T = 0.02 [s].
The simulation results are presented in Fig. 5. This figure
shows the actual control error er := [pTer eTR (eξ̂θer )]T , for theTABLE I

perx [m]0.05

VALUES OF THE INTEGRAL COST

0-0.1
pery [m]

0
0.05

2

3

4

5

6

0

1

2

3

4

5

6

0

1

2

3
time [s]

4

5

6

0

-0.1
NGerz [rad]

1

0

-0.5

Fig. 5. Actual control error: solid: with stabilizing receding control law;dashed: with passivity based control law.

position based control, which is the error vector between gco
and gd . In Fig. 5, the solid lines denote the errors applying
the proposed stabilizing receding horizon control, and thedashed lines denote those for the passivity based control law
uk (15). We focus on the errors of the translations of x andy and the rotation of z, because the errors of the translationof z and the rotations of x and y are zeros ideally on thedefined coordinates in Fig. 4.
In Fig. 5, the asymptotic stability can be confirmed bysteady state performance. Moreover, the rise time applying
the receding horizon control is shorter than that for thepassivity based control. The controller predicts the movementof the target object using the visual information, as a result,the manipulator moves more aggressively. This validates oneof the expected advantages of the stabilizing receding horizon
control for the visual feedback system.
The performance for parameter value T and ρ is comparedin terms of the integral cost in Table I. Since the costof the stabilizing receding horizon method is smaller thanthe passivity based control method under conditions of theadequate cost function, it can be easily verified that thecontrol performance is improved. With increasing weightof the terminal cost from ρ = 0.75 to ρ = 1.5 the cost
increases, too. With higher terminal cost the state value isreduced more strictly, using a large control input. In thissimulation, since the weights of the control input are larger
than those of the state, the cost increased consequently. As
the horizon length increases from T = 0.005 to T = 0.05,
the cost is reduced. In the case of T = 0.2, the calculation
can not be completed within one sampling interval, due tolimited computing power.
V. C ONCLUSIONSThis paper proposes a stabilizing receding horizon control
for a 3D visual feedback system, which is a highly nonlinear
and relatively fast system, as a first step for a predictive
visual feedback control. It is shown that the stability of
the receding horizon control scheme is guaranteed by using

Control Scheme
Passivity based Control
Receding Horizon Control (T = 0.02 [s], ρ = 0.75)
Receding Horizon Control (T = 0.02 [s], ρ = 1)
Receding Horizon Control (T = 0.02 [s], ρ = 1.5)
Receding Horizon Control (T = 0.005 [s], ρ = 1)
Receding Horizon Control (T = 0.05 [s], ρ = 1)

cost
1208
9.61
31.5
682
32.4
30.1

the terminal cost derived from an energy function of thevisual feedback system. In the simulation results, the control
performance of the stabilizing receding horizon control isimproved compared to that of the simple passivity based
control. In this paper, the stabilizing receding controller was
implemented for a low level inner loop, in the near future,
we would like to tackle the implementation on a high levelouter loop.
R EFERENCES[1] S. Hutchinson, G. D. Hager and P. I. Corke, “A Tutorial on Visual
Servo Control,” IEEE Trans. Robotics and Automation, Vol. 12, No. 5,
pp. 651–670, 1996.
[2] H. I. Christensen and P. Corke, “Editorial: Special Issue on Visual
Servoing,” The Int. Journal of Robotics Research, Vol. 22, No. 10–11,
pp. 779–780, 2003.
[3] R. Kelly, R. Carelli, O. Nasisi, B. Kuchen and F. Reyes, “Stable
Visual Servoing of Camera-in-Hand Robotic Systems,” IEEE Trans.
Mechatronics, Vol. 5, No. 1, pp. 39–48, 2000.
[4] N. J. Cowan, J. D. Weingarten and D. E. Koditschek, “Visual Servoing
via Navigation Functions,” IEEE Trans. Robotics and Automation,
Vol. 18, No. 4, pp. 521–533, 2002.
[5] H. Kawai and M. Fujita, “Passivity-based Dynamic Visual Feedback
Control for Three Dimensional Target Tracking: Stability and L2 -gain
Performance Analysis,” Proc. 2004 American Control Conference, pp.
1522–1527, 2004.
[6] M. Fujita, H. Kawai and M. W. Spong, “Passivity-based Dynamic
Visual Feedback Control for Three Dimensional Target Tracking:Stability and L2 -gain Performance Analysis,” IEEE Trans. Control
Systems Technology, 2006(to appear).
[7] D. Q. Mayne, J. B. Rawlings, C. V. Rao and P. O. M. Scokaert,“Constrained Model Predictive Control: Stability and Optimality,”
Automatica, Vol. 36, No. 6, pp. 789–814, 2000.
[8] T. Parisini and R. Zoppoli, “A Receding-Horizon Regulator for Nonlinear Systems and a Neural Approximation,” Automatica, Vol. 31,
No. 10, pp. 1443–1451, 1995.
[9] A. Jadbabaie, J. Yu and J. Hauser, “Unconstrained Receding-Horizon
Control of Nonlinear Systems,” IEEE Trans. Automatic Control, Vol.
46, No. 5, pp. 776–783, 2001.
[10] J. Yu, A. Jadbabaie, J. Primbs and Y. Huang, “Comparison of Nonlinear Control Design Techniques on a Model of the Caltech Ducted
Fan,” Automatica, Vol. 37, No. 12, pp. 1971–1978, 2001.
[11] A. Jadbabaie and J. Hauser, “Control of a Thrust-Vectored Flying
Wing: a Receding Horizon – LPV Approach,” International Journalof Robust and Nonlinear Control, Vol. 12, No. 9, pp. 869–896, 2002.
[12] A. E. Hunt and A. C. Sanderson, “Vision-based Predictive Robotic
Tracking of a Moving Target,” Technical Report, Carnegie Mellon
University, 1982.
[13] M. Fujita, Y. Kawai and Y. Nakaso, “An Experimental Study ofStabilizing Receding Horizon Control of Visual Feedback Systemwith Planar Manipulators,” Proc. 2005 International Workshop on
Assessment and Future Directions of Nonlinear Model Predictive
Control, pp. 319–326, 2005.
[14] R. M. Murray, Z. Li and S. S. Sastry, A Mathematical Introduction to
Robotic Manipulation, CRC Press, 1994.
[15] F. Bullo and A. D. Lewis, Geometric Control of Mechanical Systems,
Springer–Verlag, 2004.
[16] M. W. Spong, S. Hutchinson and M. Vidyasagar, Robot Modeling andControl, John Wiley & Sons, 2006.
[17] T. Ohtsuka, “A Continuation/GMRES Method for Fast Computationof Nonlinear Receding Horizon Control,” Automatica, Vol. 40, No. 4,
pp. 563–574, 2004.