Visual Motion Observer-based Pose Control with Panoramic Camera
via Passivity Approach
Hiroyuki Kawai, Toshiyuki Murao and Masayuki Fujita
Abstract— This paper considers the vision-based estimationand control with a panoramic camera via passivity approach.
First, a hyperbolic projection of a panoramic camera is presented. Next, using standard body-attached coordinate frames(the world frame, mirror frame, camera frame and object
frame), we represent the body velocity of the relative rigid body
motion (position and orientation). After that, we propose a visual motion observer to estimate the relative rigid body motion
from the measured camera data. We show that the estimation
error system with a panoramic camera has the passivity whichallows us to prove stability in the sense of Lyapunov. After
that, stability and L2 -gain performance analysis for the closedloop system are discussed. Finally, simulation and experimental
results are shown in order to confirm the proposed method.

I. I NTRODUCTION
Vision based control uses the computer vision data tocontrol the motion of a robot in an efficient manner. The
combination of mechanical control with visual information,so-called visual feedback control or visual servoing, is important when we consider a mechanical system working underdynamical environments [1].
Recently, Lippiello et al. [2] presented a position-based
visual servoing for a hybrid eye-in-hand/eye-to-hand multicamera configuration by using the extended Kalman filter anda multiarm robotic cell. Gans and Hutchinson [3] proposed a
hybrid switched-system control which utilizes image-based
and position-based visual feedback control. Hu et al. [4]considered a homography-based robust visual servo control
for the uncertainty of the camera calibration. In our previous
works, we discussed the dynamic visual feedback control
for 3D target tracking based on passivity [5][6]. Althoughthese previous works give us the new vision-based robot
control theory systematically, most of the works use a simple
perspective projection by a pinhole camera.
On the other hand, omnidirectional cameras are usefulfor recognizing unknown surroundings widely. Geyer andDaniilidis [7] presented a unifying theory for all central
panoramic systems, i.e., an equivalence of catadioptric andspherical projections. Mariottini et al. [8] reviewed theseveral epipolar geometry estimation algorithms by usinga omnidirectional camera and give us Epipolar Geometry
Toolbox which is a simulation environment with MATLAB.
Fomena and Chaumette [9] considered improvements onmodeling features for visual servoing using a spherical projection. Although these novel methods need a desired imagea priori, these works focus on the theoretical contributions.
From the more practical point of view, Vidal et al. [10]H. Kawai is with Department of Robotics, Kanazawa Institute of Technology, Ishikawa 921-8501, Japan hiroyuki@neptune.kanazawa-it.ac.jp
T. Murao is with Master Program of Innovation for Design and Engineering Advanced Institute of Industrial Technology, Tokyo 140-0011, Japan
M. Fujita is with Department of Mechanical and Control Engineering,Tokyo Institute of Technology, Tokyo 152-8550, Japan

MirrorCameraMirrorMirrorCameraCameraMobile Robot

Mobile Robot

Fig. 1.

Mobile Robot

Omnidirectional vision-based formation of mobile robots.

used motion segmentation techniques to estimate the positionof each leader for the vision-based formation control ofmobile robots with central panoramic cameras. Although
the vision-based pose synchronization has been proposed in
our previous work [11], a perspective projection model of apinhole camera is used in order to estimate relative poses.
This paper deals with vision-based control by using apanoramic camera for mobile robot systems as depicted
in Fig. 1. We propose the visual motion observer with apanoramic camera in order to estimate the relative rigid body
motion (position and orientation). The main contribution ofthis paper is to show that the estimation error system with apanoramic camera has the passivity which allows us to provestability in the sense of Lyapunov. After that, stability andL2 -gain performance analysis for the closed-loop system are
discussed. Finally, simulation and experimental results areshown in order to confirm the proposed method.
II. PANORAMIC C AMERA P ROJECTION
A. Hyperbolic Projection of Panoramic Camera
In this paper, we consider a panoramic camera which
consists of a pinhole camera and a hyperbolic mirror as
shown in Fig. 2. So, the pinhole camera catches reflected
images through the hyperbolic mirror. We first review thepanoramic camera model [8] in order to represent a image
feature in our framework. Visual feedback systems by using apanoramic camera use four coordinate frames which consistof a world frame Σw , a mirror frame Σm , a camera frame
Σc , and a object frame Σo as in Fig. 2. Let pmo ∈ R3 andeξ̂θmo ∈ SO(3) be the position vector and the rotation matrix
from the mirror frame Σm to the object frame Σo . Then, therelative rigid body motion from Σm to Σo can be represented
by gmo = (pmo , eξ̂θmo ) ∈ SE(3) 1 . Similarly, gwm =(pwm , eξ̂θwm ), gwc = (pwc , eξ̂θwc ) and gwo = (pwo , eξ̂θwo )denote the rigid body motions from the world frame Σw to
the mirror frame Σm , from the world frame Σw to the camera
frame Σc and from the world frame Σw to the object frame
Σo , respectively.
1 The notation of the homogeneous transform is referred to [5].

zoHyperbolic
Mirror

poi
zm

pmi

ym

pmh

5m

Mirror Frame

gcm

xm

h

gmo
zw

5o

gwo

MirrorFrameyo(constant)

Object Frame

5w

pch

Mobile Robot

Fig. 3.

yc
xc

Fig. 2. Panoramic camera model (pinhole camera and hyperbolic mirror).

Here, pwo is projected at s through the origin of the camera
frame Σc, after being projected at pmh through the origin of
the mirror frame Σm as shown in Fig. 2. Let a and b be thehyperbolic mirror parameters which satisfy
2x2mh + ymh(zmh + r)2
−
=1
(1)
a2
b2
√
with eccentricity r = a2 + b2 . Eq. (1) means a constraint
with respect to pmh . The transformation to obtain the projection s in the camera frame (see, Fig. 2) is given by
1  ξ̂θcm
(2)
s=
Λ e
(αe−ξ̂θwm (pwo − pwm )) + pcm
zchwhere α is a scalar and defined as s := [ fx fy 1 ]T ∈R3 where fx and fy are the coordinates of x-axis and yaxis onto the image plane, respectively [8]. In this paper,Λ is assumed as the ideal internal calibration matrix of thepinhole camera and defined as Λ := diag{λ, λ, 1} where λis a focal length.
From Fig. 2, the relation between the camera frame Σcand the mirror frame Σm can be represented asT, eξ̂θcm = I3(3)pcm = 0 0 2r
and it is assumed that these parameters are known. Moreover,pmh can be represented as follows:
pmh = αpmo

5oObject Frame

gwc

World Frames

CameraFrame

gwo(Target Robot)

zc

5c

gwm

xw

2r
PinholeCameragco

CameraFrame 5c

yw

gwm

gmo

5m
gcm

xo

5w
World Frame

Coordinate frames for mobile robots.

From Eqs. (2) and (3) and zch = 2r + zmh , the hyperbolic
projection of the panoramic camera can be represented as1α(pmo )pmo + pcm(7)s=Λ
2r + α(pmo )zmo
where we exploit zmh = α(pmo )zmo and the composition
rule, i.e., pmo = e−ξ̂θwm (pwo − pwm ).
B. Image Feature for Panoramic Camera
Let poi ∈ R3 and pmi ∈ R3 be the position vectors of
the target object’s i-th feature points (i = 1, · · · , n, (n ≥ 4))
relative to Σo and Σm , respectively (see Fig. 2). Using atransformation of the coordinates, we havepmi = gmo poi(8)where pmi and poi should be regarded, with a slight abuseof notation, as [pTmi 1]T and [pToi 1]T via the well-known
homogeneous coordinate representation in robotics, respectively (see, e.g., [12]).
The hyperbolic projection of the i-th feature point onto
the image plane gives us the image plane coordinate fi :=[ fxi fyi ]T ∈ R2 asλα(pmi )
xmi
fi =
(9)
2r + α(pmi )zmi ymi
where α(pmi ) means that pmo in Eq. (6) is replaced with
pmi = [xmi ymi zmi ]T . It is straightforward to extend thismodel to n image points by simply stacking the vectors of
the image plane coordinate, i.e.,f(gmo ) := [f1T · · · fnT ]T ∈ R2n .

(10)

(4)

and pm := [pTm1 · · · pTmn ]T ∈ R3n . We assume that multiple

Because αpmo has to satisfy the constraint (1), the following
relation holds(5)

feature points on a known object are given. Although the
problem of extracting the feature points from the target
object is interesting in its own right, we will not focus on
this problem and merely assume that the image feature are
obtained by well-known techniques [13]. From Eq. (8), theimage feature f only depends on the relative rigid body
motion gmo .

(6)III. B ODY V ELOCITY FOR PANORAMIC C AMERA

2
α2 x2mo + α2 ymo(αzmo + r)2
−
= 1.
2
2
a
b
Solving (5) for α, we obtainα(pmo ) =b2 (−rzmo ± apmo )22 − a2 x 2 − a2 y 2
b zmo
mo
mo

where α(pmo ) represents that α depends on pmo explicitly2 .
2 Although α(pmo ) has two solutions, the suitable solution will beselected by pmo [8].

The relative rigid body motion gmo is discussed in thissection, because the image feature f only depends on gmo .
We recall that visual feedback systems by using a panoramic
camera use four coordinate frames and the relative rigid

body motion from Σm to Σo can be represented by gmo =(pmo , eξ̂θmo ) as shown in Fig. 3.
The relative rigid body motion from Σm to Σo can be led
by using the composition rule for rigid body transformations([12], Chap. 2, pp. 37, eq. (2.24)) as follows:−1
gmo = gwm
gwo .

(11)

The relative rigid body motion involves the velocity ofeach rigid body. To this aid, let us consider the velocity
of a rigid body as described in [12]. We define the body
velocity of the mirror relative to the world frame Σw asb
TT= [vwmωwm]T , where vwm and ωwm represent the
Vwmvelocity of the origin and the angular velocity from Σw toΣm , respectively ([12] Chap. 2, eq. (2.55)).
Differentiating (11) with respect to time, the body velocityof the relative rigid body motion gmo can be written as
follows (See [5]):bbb= −Ad(gmo−1 Vwm + Vwo .
Vmo
)

(12)

b
is the body velocity of the target object relative towhere VwoΣw . Eq. (12) is a standard formula for the relation between
the body velocities of three coordinate frames ([12] Chap. 2,b
pp. 59, Proposition 2.15). Because the camera velocity Vwcbis adequate as a input rather than the mirror velocity Vwo
in this framework, we lead the body velocity of the relative
rigid body motion with the camera velocity.
The body velocity of the mirror frame relative to Σw will
be denoted as
b
−1
−1
= gwm
ġwm = gwm
ġwcgcm
V̂wm
−1
−1
−1 b
= gwm gwcgwc ġwc gcm = gcm
V̂wc gcm .

(13)

From the property concerning the adjoint transformation,b
Vwm
can be transformed intobb
= Ad(gcm−1 V
Vwm) wc .

(14)

Thus, Eq. (12) can be transformed into
b
b
b
Vmo
= −Ad(gmo−1 Ad −1 V)(gcm ) wc + Vwo .

(15)This is the body velocity of the relative rigid body motion
for the panoramic camera. While gcm is known informationbb
from Eq. (3), gmo and gwo , i.e. Vmo
and Vwo, are unknown
information in the visual feedback system. Then, the control
objective is described as follows.
Control Objective: The controlled mobile robot follows
the target robot, i.e., the relative rigid body motion gco is
coincided with the desired one gd .
Because gcm is known a priori, gco can be obtained fromgmo by the composition rule gco = gcm gmo . Thus, weconsider the estimate of gmo for the above control objective.
IV. V ISION -B ASED E STIMATIO N
A. Image Jacobian for Panoramic Camera
Since the measurable information is only the image feature
f from the panoramic camera, we consider a visual motion
observer in order to estimate the relative rigid body motion
gmo from the image feature f. Using the body velocity of therelative rigid body motion (15), we choose estimates ḡmo =ˆb(p̄mo , eξ̄θ̄mo ) and V̄moof the relative rigid body motion andvelocity, respectively asbb−1 Ad −1 VV̄mo
= −Ad(ḡmo)(gcm ) wc + ue .

(16)

T
T T
ωue
] is to be determined in order
The new input ue = [vuebto their actual
to drive the estimated values ḡmo and V̄movalues.
Similarly to (8) and (9), the estimated image featuref¯i (i = 1, · · · , n) is defined asp̄mi = ḡmo poiλα(p̄mi )
x̄mi
f¯i =
2r + α(p̄mi )z̄mi ȳmi

(17)
(18)

where p̄mi := [x̄mi ȳmi z̄mi ]T . f¯(ḡmo ) := [f¯1T · · · f¯nT ]T∈ R2n means the n image points case.
In order to establish the estimation error system, we define
the estimation error between the estimated value ḡmo and theactual relative rigid body motion gmo as−1
gmo
gee := ḡmo

(19)

ˆ

in other words, pee = e−ξ̄θ̄mo (pmo − p̄mo ) and eξ̂θee =ˆe−ξ̄θ̄mo eξ̂θmo . We next define the error vector of the rotation
matrix eξ̂θab as eR (eξ̂θab ) := sk(eξ̂θab )∨ where sk(eξ̂θab )denotes 12 (eξ̂θab − e−ξ̂θab ). Using this notation, the vectorof the estimation error is given by ee := [pTee eTR (eξ̂θee )]T .
From the above, we derive a relation between the actual andthe estimated image feature. Suppose the attitude estimation
error θee is small enough that we can let eξ̂θee  I +sk(eξ̂θee ). Then we have the following relation between theactual feature point pmi and the estimated one p̄mi




ˆ
pee
pmi − p̄mi = eξ̄θ̄mo I −p̂oi. (20)eR (eξ̂θee )Using a first-order Taylor expansion approximation, therelation between the actual image feature and the estimatedone can be expressed as∂fi 
∂fi 
fi − f¯i =
p
=p̄
mi
mi
∂xmi
∂ymi pmi =p̄mi
∂fi 
(pmi − p̄mi ) (21)∂zmi pmi =p̄mi
where the partial differentiations ∂fi /∂xmi , ∂fi /∂ymi and∂fi /∂zmi are represented as Eqs. (22)–(24) at the bottom of
the next page, respectively.
Let us define the image feature error as fe := f(gmo ) −f¯(ḡmo ). Hence, the relation between the actual image featureand the estimated one can be given byfe = J(ḡmo )ee ,2n×6

where J(ḡmo ) : SE(3) → R(25)

is defined asT
J(ḡmo ) := [J1T (ḡmo ) J2T (ḡmo ) · · · Jm(ḡmo )]T(26)
∂fi 
∂fi 
Ji (ḡmo ) :=∂xmi pmi =p̄mi ∂ymi pmi =p̄miˆ∂fi 
× eξ̄θ̄mo I −p̂oi .
p
=p̄
mi
mi
∂zmi
(i = 1, · · · , n)(27)

We assume that the matrix J(ḡmo ) is full column rank for allḡmo ∈ SE(3). Then, the relative rigid body motion can be
uniquely defined by the image feature vector. Because thismay not hold in some cases when n = 3, it is known thatn ≥ 4 is desirable for the full column rank of the image
Jacobian.
The above discussion shows that we can derive the vectorof the estimation error ee from image feature f and theestimated value of the relative rigid body motion ḡmo ,along the trajectories of (29) gives us

ee = J † (ḡmo )fe

Integrating (32) from 0 to T yields

V̇e = pTee eξ̂θee e−ξ̂θee ṗee + eTR (eξ̂θee )eξ̂θee ωeeb= eTe Ad(eξ̂θee ) Vee= uTe (−ee ).

(28)

T

where † denotes the pseudo-inverse. Therefore the estimation
error ee can be exploited in the 3D visual feedback control
law using image feature f obtained from the panoramic
camera.
Remark 1: If we select one and the imaginary unit as aand b numerically, i.e., a = 1, b = i, then the image Jacobian
for the panoramic camera (27) equals to the pinhole’s one
T
T
∂fi
∂fi
λ
λ
[5] as follows: ∂x= zmi[1 0] , ∂y= zmi
[0 1] ,mimi∂fi
= − zλ2 [xmi ymi ]T . Thus our previous work [5] can∂zmi
mibe regarded as a special case of this study, although thepinhole camera has different applications from the panoramic
one in the practical view.

0ue = Ke ee(30)

where βe is a positive scalar.
Proof: Consider the positive definite function1pee 2 + φ(eξ̂θee )2(31)

where φ(eξ̂θab ) := 12 tr(I − eξ̂θab ) is the error function of
the rotation matrix [14]. Evaluating the time derivative of Ve λα(pmi )∂fi
2rλαx (pmi )
xmi
1
+
=
∂xmi
(2r + α(pmi )zmi )2 ymi
2r + α(pmi )zmi 0 λα(pmi )∂fi
2rλαy (pmi )
xmi
0
+
=
2
y
∂ymi
(2r + α(pmi )zmi )2r + α(pmi )zmi 1
mi




∂fi
λα2 (pmi )2rλαz (pmi )
xmi
xmi
−
=
∂zmi
(2r + α(pmi )zmi )2 ymi(2r + α(pmi )zmi )2 ymi
where
αx(pmi ) =2+ a2 pmi 2 )∂α(pmi )−2a2 b2 rxmi zmi pmi  ± ab2 xmi (r 2 zmi=
2∂xmi
pmi 
b 2 z 2 − a2 x 2 − a2 y 2
mi

mi

mi

2
+ a2 pmi 2 )∂α(pmi )−2a2 b2 rymi zmi pmi  ± ab2 ymi (r 2 zmi
αy (pmi ) ==2∂ymi
pmi 
b 2 z 2 − a2 x 2 − a2 y 2
mi

(33)

(34)

where Ke := diag{ke1 , · · · , ke6} is the positive gain matrixof x, y and z axes of the translation and the rotation for theestimation error.
bTheorem 1: If Vwo= 0, then the equilibrium point ee = 0
for the closed-loop system (29) and (34) is asymptotic stable.
Proof: Theorem 1 can be easily proved by Lemma 1,
hence the proof is omitted.
Fig. 4 shows a block diagram of the visual motion observerwith a panoramic camera. By the proposed visual motion
observer, the unmeasurable motion gco will be exploited as
the part of control law. Our proposed visual motion observeris composed just as Luenberger observer for linear systems.
Remark 3: The estimation error vector is configured byavailable information (i.e.,the measurement and the estimate)though it is defined by unavailable one. This is one of themain contributions of this paper.

T

Ve =

V̇ dτ ≥ −V (0) ≥ −βe

Based on the above passivity property of the estimation
error system, we consider the following control law.

Then, we have the following lemma relating the input ue tothe vector form of the estimation error ee .
bLemma 1: If Vwo= 0, then the following inequality holds
for the estimation error system (29).

00C. Visual Motion Observer(29)

uTe (−ee )dt ≥ −βe

TuTe (−ee )dτ =where βe is the positive scalar which only depends on theinitial state of gee .
Remark 2: Let us consider the vector form of the estimation error −ee as its output. Then, Lemma 1 says that theestimation error system (29) is passive from the input ue tothe output −ee . In fact, the body velocity of the relative rigid
body motion (15) has passivity, the estimation error systempreserves its passivity.

B. Passivity of Estimation Error System
In the same way as Eq. (12), the estimation error system
can be represented bybb−1 ue + V
Vee
= −Ad(gee
wo .
)

(32)

mi

mi

2
2
2
b2 r(b2 zmi+ a2 x2mi + a2 ymi)pmi  ∓ ab2 zmi r 2 (x2mi + ymi) + b2 pmi 2∂α(pmi )=αz (pmi ) =2∂zmi
pmi 
b 2 z 2 − a2 x 2 − a2 y 2
mi

mi

mi

(22)
(23)
(24)

b
Vwo
b
Vwc

RRBM-PC

gmo

Ke
ue

where β is a positive scalar.
Proof: Lemma 2 can be proved by using the positive
definite function V = 12 pec2 + φ(eξ̂θec ) + 12 pee 2 +φ(eξ̂θee ), hence the proof is omitted.
Remark 4: Let us take u as the input and −x as its output.
Thus, Lemma 2 implies that the visual motion error system(37) is passive from the input u to the output −x.

Panoramic Camera
Panoramic Camera f
Projection
ee

Image
Jacobianfe +Panoramic Camera Model
RRBM-PC gmo Panoramic Camera f
Model
Projection Model

Visual Motion Observer for Panoramic Camera

Fig. 4.

C. Visual Motion Pose Control and Stability Analysis

Block diagram of visual motion observer.

V. V ISUAL M OTION O BSERVE R - BASED P OSE C ONTROL
A. Pose Control Error System
Let us consider the dual of the estimation error system,
which we call the pose control error system, in order toachieve the control objective. First, we define the posecontrol error as follows:
gec = gd−1 gco ,(35)

which represents the error between the relative rigid body
motion gco and the reference one gd . It should be remarked
that gco can be calculated by using the estimated relative
rigid body motion ḡmo , the estimation error vector ee =[pTee eTR (eξ̂θee ) ] and the known mirror parameter gcm equivalently, although gco can’t be measured directly (see [6] formore details). Using the notation eR (eξ̂θ ), the vector of thepose control error is defined as ec := [pTec eTR (eξ̂θec )]T .
The reference of the relative rigid body motion gd isassumed to be constant in this paper, i.e., ġd = 0 and
b
b
= Vco
. Thus, the pose control error system can behence Vec
represented as


b
b
b
Vec
= −Ad(gec−1(36)) Ad (g−1 ) Vwc + Vwo .
dThis is dual to the estimation error system. Similar to theestimation error system, the pose control error system alsopreserves the passivity property.
B. Passivity of Visual Motion Error System
Combining the estimation error system (29) and the posecontrol one (36), we construct the visual motion observerbased pose control error system (we call the visual motion
error system) as follows: b   
−1
−Ad(gec
0
Vec
I
)
b
=
u+
Vwo
(37)
b
−1
0
−Ad(gee
Vee
I
)

T
b T
where u := (Ad(g−1 ) Vwc) uTe . Let us define the errordvector of the visual motion error system as x := [eTc eTe ]Twhich consists of the pose control error vector ec and theestimation error vector ee . It should be noted that if thevectors of the pose control error and the estimation one are
equal to zero, then the actual relative rigid body motion gco
tends to the reference one gd when x → 0.
Next, we show an important relation between the input
and the output of the visual motion error system.
b
Lemma 2: If Vwo= 0, then the visual motion error system(37) satisfiesTu (−x)dt ≥ −β, ∀T > 0
T

0

(38)

Based on the above passivity property of the visual motion
error system, we consider the following control law.


Kc 0
(39)
u = Kx, K :=0 Ke
where Kc := diag{kc1 , · · · , kc6} is the positive gain matrixof x, y and z axes of the translation and the rotation for thepose control error.
bTheorem 2: If Vwo= 0, then the equilibrium point x = 0
for the closed-loop system (37) and (39) is asymptotic stable.
Proof: Theorem 2 can be easily proved by Lemma 2,
hence the proof is omitted.
Theorem 2 shows Lyapunov stability for the closed-loopsystem. If the camera velocity is decided directly, the control
objective is achieved by using the proposed control law(39). Although the dead angle problem [9] is not considered
explicitly in this framework, our proposed method will
overcome this problem by dealing with a collision avoidanceas in [11].
D. L2 -Gain Performance Analysis
In this subsection, we utilize L2 -gain performance analysis
to evaluate the tracking performance of the control scheme
in the presence of a moving target robot. The motion of thetarget robot is regarded as an external disturbance.
In order to derive a simple and practical gain condition,
we redefine Ke = ke I where ke is a positive scalar.
Theorem 3: Given a positive scalar γ, assume
γ 2 (2ke − 1) + 2(ke − 1)
2 {γ 2 (2ke − 1) − 1}
2
γ +1
ke >
2γ 2

kc,min >

(40)
(41)

where kc,min means the minimum value in Kc . Then theclosed-loop system (37) and (39) has L2 -gain ≤ γ.
Proof: The proof is omitted due to its similarity to theproof of Theorem 3 in [5].
The estimation gain does not have to be restricted to ascalar, although it cause slightly complicated gain conditions
compared with Eqs. (40) and (41). for the reason that Schurcomplement can not be used. In this framework, γ can beconsidered as an indicator of the tracking performance.
VI. S IMULATION AND E XPERIMENTAL R ESULT SBoth the proposed estimation and control methods are
confirmed by the simulation, while the experiment is carriedout by using the proposed visual motion observer only.

4

6

0

-0.02
0

2

time [s]46Fig. 5. Estimation error ee (Gain
A: Dashed, Gain B: Solid).

6

180
0

1
time [s]

2

130

220
2

4

6

0

-0.04
0

2

time [s]46Fig. 6. Pose control error ec (Gain
A: Dashed, Gain B: Solid).

180

0

1
time [s]2

1time [s]

2

150

130

0.1
0
0
-2
2 x 10

170

200

0

peex [m]150

peey [m]

4

0

-0.01
0
0.04

NGecz [rad]22

fx2, fx2 [pixel]

6pecy [m]

4

0

-5
0
0.02

NGeez [rad]

2

170

200

fx4, fx4 [pixel]

-5
0
0.01

peey [m]-40x 10-35

220

NGeez [rad]x 10-30

fx1, fx1 [pixel]

5

fx3, fx3 [pixel]

x 10-30

pecx [m]peex [m]40

1time [s]2Fig. 9. Image features and estimated ones w.r.t. the x-axis.

1

2

1

2

1
time [s]

2

0
-2
-4
0
0.2
0

-0.2
0

Fig. 10. Estimation error ee for thestatic target object with ke = 5.

we can confirm that the estimation error ee tends to zero byusing the proposed visual motion observer.

Fig. 7.

Panoramic camera.

Fig. 8.

Target mobile robot.

A. Simulation Results
In this subsection, we present simulation results for stability and L2 -gain performance analysis in the case of a moving
target robot. The target robot has four feature points andmoves by t = 4.8 [s]. The gains for the control law u (39)were empirically selected as follows:
Gain A) γ = 0.123, Kc = 50I, ke = 100
Gain B) γ = 0.082, Kc = 100I, ke = 300.
The simulation results are presented in Figs. 5 and 6 which
illustrate the estimation error ee and the pose control one ec ,respectively. In these figures, we focus on the errors of thetranslations of x and y and the rotation of z. In Figs. 5 and 6,
the dashed line and the solid line are the errors in the caseof γ = 0.123 and γ = 0.082, respectively.
In the case of the static target robot, i.e., after t = 4.8[s], all errors in Figs. 5 and 6 tend to zero. Therefore,
asymptotic stability can be confirmed through the simulation.
In the presence of the moving target robot as disturbancesby t = 4.8 [s], the tracking performance is improved for thesmaller values of γ from Figs. 5 and 6. Thus the simulation
results show that L2 -gain is adequate for the performancemeasure of the visual motion observer-based pose control.
B. Experimental Results
In this subsection, we describe experimental results withrespect to the proposed visual motion observer with apanoramic camera for a static target object as shown in
Figs. 7 and 8. A panoramic camera consists of a MTV7310 camera and a hyperbolic mirror. The video signals
are acquired by a frame graver board PICOLO DILLIGENT (Euresys) and an image processing software HALCON (MVTech Software GmbH).
The experiment was carried out with an appropriate initial
estimation error. The experimental results are presented in
Figs. 9 and 10. In Fig. 9, the dashed lines and the solid ones
mean the image features and the estimated ones with respect
to the x-axis, i.e., fxi and f¯xi (i = 1, · · · , 4), respectively.
The estimated image features coincide with the actual ones
after t = 1 [s]. From Fig. 10 which illustrates the estimation
error of the translations of x and y and the rotation of z,VII. C ONCLUSIONSThis paper considers the vision-based estimation and control with a panoramic camera based on the passivity. The
main contribution of this paper is to show that the estimation
error system with a panoramic camera has the passivity
which allows us to prove stability in the sense of Lyapunov.
The visual motion error system which consists of the estimation error system and the pose control one preserves thepassivity. Our previous work [5] can be regarded as a special
case of this study.
R EFERENCE S[1] F. Chaumette and S. A. Hutchinson, “Visual Servoing and Visual
Tracking,” In:B. Siciliano and O. Khatib (Eds), Springer Handbookof Robotics, Springer–Verlag, pp. 563–583, 2008.
[2] V. Lippiello, B. Siciliano and L. Villani, “Position-Based Visual
Servoing in Industrial Multirobot Cells Using a Hybrid Camera
Configuration,” IEEE Trans. on Robotics, Vol. 23, No. 1, pp. 73–86,
2007.
[3] N. R. Gans and S. A. Hutchinson, “Stable Visual Servoing ThroughHybrid Switched-System Control,” IEEE Trans. on Robotics, Vol. 23,
No. 3, pp. 530–540, 2007.
[4] G. Hu, W. MacKunis, N. Gans, W. E. Dixon, J. Chen, A. Behal, and D.
Dawson, “Homography-Based Visual Servo Control With Imperfect
Camera Calibration,” IEEE Trans. on Automatic Control, Vol. 54,
No. 6, pp. 1318–1324, 2009.
[5] M. Fujita, H. Kawai and M. W. Spong, “Passivity-based Dynamic
Visual Feedback Control for Three Dimensional Target Tracking:Stability and L2 -gain Performance Analysis,” IEEE Trans. on Control
Systems Technology, Vol. 15, No. 1, pp. 40–52, 2007.
[6] T. Murao, H. Kawai and M. Fujita “Predictive Visual Feedback Control
with Eye-in/to-Hand Configuration via Stabilizing Receding Horizon
Approach,” Proc. of the 17th IFAC World Congress on AutomaticControl, pp. 5341–5346, 2008.
[7] C. Geyer and K. Daniilidis, “A Unifying Theory for Central Panoramic
Systems and Practical Implications,” In:D. Vernon (Ed), Computer
Vision - ECCV 2000, Springer–Verlag, pp. 445–461, 2000.
[8] G. L. Mariottini and D. Prattichizzo, “EGT for Multiple View Geometry and Visual Servoing,” IEEE Robotics & Automation Magazine,
Vol. 12, No. 4, pp. 26–39, 2005.
[9] R. T. Fomena and F. Chaumette, “Improvements on Visual Servoing
From Spherical Targets Using a Spherical Projection Model,” IEEE
Trans. on Robotics, Vol. 25, No. 4, pp. 874–886, 2009.
[10] R. Vidal, O. Shakernia and S.Sastry, “Following the flock,” IEEE
Robotics & Automation Magazine, Vol. 11, No. 4, pp. 14–20, 2004.
[11] M. Fujita, T. Hatanaka, N. Kobayashi, T. Ibuki and M. W. Spong,“Visual Motion Observer-based Pose Synchronization: A Passivity
Approach,” Proc. of the 48th IEEE Conf. on Decision and Control,
pp. 2402-2407, 2009.
[12] R. Murray, Z. Li and S. S. Sastry, A Mathematical Introduction to
Robotic Manipulation, CRC Press, 1994.
[13] D. Forsyth and J. Ponce, Computer Vision - A Modern Approach,Prentice–Hall, 2003.
[14] F. Bullo and A.D. Lewis, Geometric Control of Mechanical Systems,
Springer–Verlag, 2004.