Extrinsic Camera Parameter Recovery
from Multiple Image Sequences Capturedby an Omni-directional Multi-camera System
Tomokazu Sato, Sei Ikeda, and Naokazu Yokoya
Graduate School of Information Science,
Nara Institute of Science and Technology,
8916-5 Takayama, Ikoma, Nara 630-0192, Japan,{tomoka-s,sei-i,yokoya}@is.aist-nara.ac.jp ,WWW home page: http://yokoya.aist-nara.ac.jp/

Abstract. Recently, many types of omni-directional cameras have beendeveloped and attracted much attention in a number of diﬀerent ﬁelds.
Especially, the multi-camera type of omni-directional camera has advantages of high-resolution and almost uniform resolution for any directionof view. In this paper, an extrinsic camera parameter recovery method
for a moving omni-directional multi-camera system (OMS) is proposed.
First, we discuss a perspective n-point (PnP) problem for an OMS, andthen describe a practical method for estimating extrinsic camera parameters from multiple image sequences obtained by an OMS. The proposed
method is based on using the shape-from-motion and the PnP techniques.

1

Introduction

In recent years, many types of omni-directional cameras have been developed[1–6] and have attracted much attention in a number of diﬀerent ﬁelds suchas robot navigation, telepresence and video surveillance. Especially, the omnidirectional multi-camera system (OMS) [4–6] has some advantages in the ﬁeld ofaugmented virtuality because that provides high-resolution and almost uniform
resolution for any direction of view. Generally, an OMS has multiple camera
units that are located radially and are ﬁxed in certain positions in the camera
block of the OMS. Although an OMS has many advantages, the position of an
OMS has been ﬁxed in most of applications, so the camera parameter recoveryof an OMS has not been well discussed in the literature. Application ﬁelds of
OMS would be expanded if absolute position and posture of OMS could berecovered accurately; for example, human navigation, environment virtualizationand 3-D model reconstruction. This paper provides an absolute and accurate
camera parameter recovery method for a moving OMS using both a few featurelandmarks and many natural features.
In a common single camera system, the extrinsic camera parameter reconstruction problem from a single image using n-point feature landmarks of known
3-D and 2-D positions are called the perspective n-point (PnP) problem [7–9].

2

Tomokazu Sato, Sei Ikeda, and Naokazu Yokoya

Generally, this problem can be solved by a least-squares minimization method if
six or more feature landmarks can be observed [10]. Although Chen and Chang[11] have extended this problem for generalized projection image sensors, thePnP problem for a multi-camera system has not been well discussed. Additionally, these PnP approaches cannot be successfully used if feature landmarks are
partially invisible in the image sequence.
On the other hand, for a single camera system, there is another approach torecovering extrinsic camera parameters from motion of image features [12–14].
They are called the shape-from-motion (SFM). Although these techniques arealso attempted for omni-directional camera systems [15–17], they cannot deal
with a large number of images because such methods are sensitive to feature
tracking errors and these errors must be accumulated. The problem of nonuniﬁed center of projection in an OMS has not also been well discussed.
In this paper, a practical, accurate and absolute method for extrinsic camera
parameter recovery of an OMS is proposed. In our method, both the PnP andSFM techniques are used tracking both feature landmarks and natural features.
The assumptions in our method are that intrinsic camera parameters (including
local extrinsic parameters in camera block) are calibrated in advance and are
ﬁxed during capturing an image sequence. A small number of feature landmarks
need to be visible in some frames of input for minimizing accumulative errors.
This paper is structured as follows. First, the PnP problem for an OMS at aﬁxed position is discussed in Section 2. Section 3 describes an extrinsic camera
parameter recovery method for a moving OMS. Experimental results with realscenes and simulation will show the feasibility and accuracy of the proposed
method in Section 4. Finally, Section 5 describes conclusion and future work.

2

PnP problem of OMS

This section describes a method for estimating absolute extrinsic camera parameters of an OMS by solving the PnP problem. In the PnP problem, theextrinsic camera parameters are estimated by using at least six feature landmarks of known 3-D and 2-D positions. Under assumptions of this problem,
intrinsic camera parameters (focal length, lens distortion parameters, center ofdistortion, aspect ratio) and local extrinsic parameters (relative camera positions
and postures) of the camera block are known.
In the following sections, ﬁrst, extrinsic camera parameters of an OMS andprojection errors of feature landmarks are deﬁned. The PnP problem is thensolved to acquire the extrinsic camera parameters by dealing with multiple cameras and multiple landmarks systematically.
2.1

Extrinsic camera parameters and projection errors

In this section, extrinsic camera parameters of an OMS at a ﬁxed position andprojection errors of feature landmarks are deﬁned. Generally, an OMS is composed of multiple cameras and each camera is ﬁxed at a certain place in theExtrinsic Camera Parameter Recovery for an OMS

3Local camera coordinatey
Lk=TkM

Camera kx

Camera 1

Camera 2z
M
World coordinate
Camera block coordinate

Tk

Fig. 1. Coordinate system of an OMS

OMS. As shown in Figure 1, the position and posture of the OMS are determined by the relationship between the camera block coordinate system and theworld coordinate system, and those of each camera are determined by the relationship between the local camera coordinate system and the camera block
coordinate system. In the following, the world coordinate system is transformed
to the camera block coordinate system by a 4 × 4 matrix M, and the camera
block coordinate system is transformed to the local camera coordinate systemof the k-th camera by the local extrinsic camera parameter Tk .
First, 6-DOF(degree of freedom) extrinsic camera parameters (posture: r1 , r2 ,r3 , position: t 1 , t2 , t3 ) from the world coordinate to the camera block coordinate
are deﬁned as follows.
⎛⎞
m11 m12 m13 m14⎜ m21 m22 m23 m24 ⎟
⎟
M=⎜
(1)
⎝ m31 m32 m33 m34 ⎠
0 0 0
1


R(r1 , r2 , r3 ) (t1 , t2 , t3 )T
=
,
(2)
0
1
where R is a 3 × 3 rotation matrix. The local extrinsic camera parameter Tk of
the k-th camera is also deﬁned by a 4 × 4 matrix in the same manner. By using
M and Tk , the extrinsic camera parameter Lk of the k-th camera in the world
coordinate system is expressed as: Lk = Tk M.
In the following expressions, for simplicity, we assume that the focal lengthis 1 and the lens distortion eﬀect is already corrected. The relationship between
the 2-D position (up , vp ) on the k-th camera image and its 3-D position S p =(xp , yp , zp , 1)T of a feature landmark p in the world-coordinate system can be
expressed as:⎛⎞aup⎝ avp ⎠ = Lk Sp = Tk MSp ,(3)awhere, a is a parameter. A computed position (up , vp ) by Eq. (3) and an actuallydetected position (ûp , v̂p ) of the feature landmark p are not always consistent
with each other due to quantization and detecting errors. The sum of squared
distances between (up , vp ) and (ûp , v̂p) for multiple m landmarks has been often

4Tomokazu Sato, Sei Ikeda, and Naokazu Yokoya

used as an error function in camera parameter estimation for a single camera
system [18]. In this paper, the sum of squared errors E is deﬁned as an error
function for an OMS as follows.
n(up − ûp )2 + (vp − v̂p )2 ,

E=

(4)

k=1 p∈Fk

where Fk is a set of visible landmarks from the k-th camera, and n denotes thenumber of camera units in the camera block.
2.2

Solving the PnP problem by minimizing projection errors

This section provides a method for solving the PnP problem of the OMS, which
is based on minimizing the error function E deﬁned in Eq. (4) in the previous
section. By solving the PnP problem, the extrinsic camera parameter M of the
OMS can be acquired.
The problem of computing M with 6-DOF by minimizing E is a non-linear
problem. To avoid local minimum solutions in a non-linear minimization process,a linear method is ﬁrst used for computing an initial estimate of M without the
6-DOF constraint. After that, M is adjusted to 6-DOF and reﬁned by a gradientmethod so as to minimize E globally.
Linear estimation of an initial parameter: To solve the PnP problem linearly, multiple n cameras and totally j feature landmarks are used systematically.
The local extrinsic camera parameter Lk of the k-th camera is re-deﬁned using
row vectors (lxk , lyk , lzk ) as follows.
⎛⎞
lxk
Lk = Tk M = ⎝ ly k ⎠ ,(5)lzk
Eq. (3) is transformed by Eq. (5) as follows.
lxk Sp − ûp lzk Sp = 0 ,lyk Sp − v̂p lzk Sp = 0 .

(6)Eq. (6) can be uniﬁed about j points of feature landmarks, and transformed byusing the parameter vector m = (m11 , · · · , m 14 , m21 , · · · , m 24 , m31 , · · · , m 34 )Tof M and the local extrinsic camera parameter Tk as follows.
Am = s,
⎛

(7)
⎞

⎛

⎞

−s4(k1 )s1(k1 )S1 s2(k1 )S1 s3(k1 )S1
⎜
⎜
⎟
⎟
..
..
..
..
⎜
⎜
⎟
⎟
.
.
.
.
⎜
⎜
⎟
⎟
⎜ −s4(kj ) ⎟⎜ s1(kj )Sj s2(kj )Sj s3(kj )Sj ⎟⎟,s = ⎜⎟
A=⎜⎜ −s8(k1 ) ⎟ ,⎜ s5(k1 )S1 s6(k1 )S1 s7(k1 )S1 ⎟
⎜
⎜
⎟
⎟
⎜
⎜
⎟
⎟
..
..
..
..
⎝
⎝
⎠
⎠
.
.
.
.
s5(kj )Sj s6(kj )Sj s7(kj )Sj−s8(kj )(8)Extrinsic Camera Parameter Recovery for an OMS

s1(ki ) s2(ki ) s3(ki ) s4(ki )s5(ki ) s6(ki ) s7(ki ) s8(ki )=

1 0 ûi 0
0 1 v̂i 0

5


Tki

(i = 1, · · ·, j),(9)where ki is the number associated with the camera from which the feature i is
visible.
In Eq. (7), all the parameters except m are known. If j is six or more, m canbe determined linearly so as to minimize |Am − s| 2 :
m = (AT A)−1 AT s.

(10)

Note that, if distances between local cameras are much smaller than 3-D distribution of feature landmarks, computed values in Eq. (10) becomes often unstable. In this case, m is deﬁned by mij = mij /m34 , and by approximating s
as 0, stable values will be acquired except the scale parameter of M by solvingAm = 0.
Camera parameter adjustment to 6-DOF: 12 parameters of M should be
reduced to 3 position parameters (t1 , t2 , t3 ) and 3 posture parameters (r1 , r2 , r3)
for Euclidean reconstruction. In this research, the position (t1 , t2 , t3 ) of the OMS
is simply decided as (m14 , m24 , m34 ). The posture parameters (r1 , r2, r3 ) of the
OMS can be determined from the rest of 9 parameters of rotation factors R̂ of
M by the singular value decomposition method [19] as follows:
R(r1 , r2 , r3 ) = Udiag(1, 1, det(UV T ))VT ,(11)

where U and V are a left and right singular vector matrices of R̂, respectively.
Non-linear minimization of error function: From the initial estimate of the
6 extrinsic camera parameters (r1 , r2 , r3 , t1 , t2 , t3 ) acquired by the previous step,
The error function E deﬁned in Eq. (4) is minimized using a gradient methodby iterating the following expressions until convergence.
ri ← ri − lriδEδE, ti ← ti − lti(i = 1, 2, 3)
δri
δti

(12)

where (lr1 , lr2 , lr3 , lt1 , lt2 , lt3 ) are scale factors of derivatives, and these values are
decided so as to minimize E in each iteration. By using this method, the extrinsic
camera parameter M of the OMS can be determined by a few iterations so as
to minimize E globally with 6 DOF, because the initial estimates are expected
to be close to the true parameters.

3

SFM of OMS

This section describes a method for estimating extrinsic camera parameters fromomni-directional movies acquired by an OMS. The SFM proposed in this paper
is not an ego-motion but absolute position estimation method which is based onusing both feature landmarks and natural features.
The extrinsic camera parameters of the moving OMS are estimated by solving
the PnP problem in each frame of input sequences using both 3-D and 2-D

6

Tomokazu Sato, Sei Ikeda, and Naokazu Yokoya

positions of feature landmarks and natural features. The feature landmarks areused as detonators for the 3-D position estimation of natural features. The 3D positions of natural features for the PnP are gushed out by tracking them
together. In our method, the six or more feature landmarks should be visible atleast in the ﬁrst and the second frames of an input video sequence to acquire
the initial estimate of extrinsic camera parameters. Finally, projection errors areglobally minimized for all feature landmarks and natural features.
In the following sections, ﬁrst, error functions in an omni-directional movie
are deﬁned. Next the deﬁned errors are minimized to estimate extrinsic camera
parameters of the OMS in the input sequence.
3.1

Deﬁnition of error function

The sum of projection errors E deﬁned in Eq. (4) is extended for a movie input.
The modiﬁed sum of projection errors in the f-th frame (f = 1, · · ·, v) of the
input movie is represented by the following expression.
n

Ef =Wp (uf p − ûf p )2 + (vf p − v̂f p )2 ,(13)

k=1 p∈Fkf

where Wp is a conﬁdence of feature p, and that is computed as an inverse covariance of projection errors of the feature p [20]. Fkf is a set of feature landmarks
and natural features that are visible in the f-th frame image of the k-th camera.
By using Ef , the total error of the input movie is deﬁned as:v

Etotal =Af Ef ,

(14)

f =1

where Af is a weighting coeﬃcient for each frame f that is set as 1 when theframe contains no speciﬁed feature landmarks or else is set to a suﬃciently large
value when the frame contains speciﬁed feature landmarks. In this paper, theerror function Etotal is employed for estimating the extrinsic camera parameters
Mf (f = 1, · · ·, v) and the 3-D positions S p of natural features.
On the other hand, the sum of projection errors about the feature p from thefs-th frame to the fe-th frame is also deﬁned as follows:fe

EFp (fs, fe) =(uf p − ûf p )2 + (vf p − v̂f p )2 .

(15)

f =f s

3.2

Estimation of extrinsic camera parameters from an
omni-directional movie

In this section, ﬁrst, initial parameters of Mf and Sp are estimated by tracking
both feature landmarks and natural features automatically by using a robust
approach. Next, Mf and Sp are then reﬁned so as to minimize Etotal by a nonlinear minimization method. The method described in this section is basicallyan extension of our previous work [20] for the OMS.

Extrinsic Camera Parameter Recovery for an OMS

7Feature tracking and initial estimation: The initial extrinsic parameter of
the OMS in each frame is estimated by using the PnP technique described in
the Section 2. Our method can compute the initial parameters even if the feature landmarks are invisible in most frames of a long input movie, because thesubstitutes of feature landmarks with known 3-D and 2-D positions are gushed
out by tracking natural features. The feature landmarks are used as detonatorsfor obtaining 3-D positions of natural features. By using a huge number of natural features to solve the PnP problem, accurate and stable estimation can beaccomplished. The following paragraphs brieﬂy describe computational steps forthe f-th frame.
(a) Feature tracking in each camera: Feature landmarks are tracked automatically by a standard template matching method until a suﬃcient numberof 3-D positions of natural features are estimated. Natural features are automatically detected and tracked by using Harris corner detector [21] for
limiting feature position candidates on the images. RANSAC approach [22]is also employed for detecting outliers. In this process, these features aretracked within each camera image.
(b) Extrinsic camera parameter estimation: The 3-D and 2-D positionsof feature landmarks and natural features are used for estimating the extrinsic camera parameter Mf . In this step, the error function Ef deﬁned in
Eq. (13) is minimized by the method described in the Section 2.2.
(c) Feature tracking between diﬀerent camera: The features that become
invisible in a certain camera are detected and tracked also in other camera
images by using the extrinsic camera parameter Mf acquired in Step (b).
The 3-D position of natural feature that has already been estimated until
the previous frame is projected to each camera image by Lkf (= Tk Mf ), andthen the visibility of the 3-D position is checked. If interest points detectedby Harris operator exist near by the projected position, the feature is tracked
to the nearest interest point.
(d) 3-D position estimation of natural features: The error function EF p
deﬁned in Eq. (15) is used for estimating a 3-D position S p of the feature
p. For all the natural features tracked in the f-th frame, EFp (fs(p), f) is
minimized and the 3-D position S p is reﬁned in every frame, where fs(p) is
the ﬁrst detected frame of the feature p.
(e) Computing conﬁdences of natural features: In this paper, the distribution of tracking errors of the feature is approximated by a Gaussian probability density function. Then, the conﬁdence Wp of the feature p is computed
as an inverse covariance of projection errors from the fs(p)-th frame to thef-th frame, and reﬁned in every frame [20].
(f ) Addition and deletion of natural features: In order to obtain accurate
estimates of camera parameters, good features should be selected. In thispaper, the set of natural features is automatically updated by checking conditions of features using multiple measures [20].
By iterating the steps above from the ﬁrst frame to the last frame of the input
movie, the initial estimate of the extrinsic camera parameters of the OMS iscomputed.

8

Tomokazu Sato, Sei Ikeda, and Naokazu Yokoya(a) appearance(b) viewing volume

Fig. 2. Omni-directional camera system ”Ladybug”.

Global optimization in video sequence: From the initial parameters ofMf (f = 1, · · ·, v), the error function Etotal deﬁned in Eq. (14) is gradually minimized by using derivatives of parameters. This minimization process is almost
the same as the method in Section 2.2, except for the 3-D positions of the natural features Sp . In this minimization, the 3-D positions S p = (xp , yp , zp , 1) of
natural features are also adjustable parameters and reﬁned by using derivatives
total δEtotal δEtotal( δEδx, δyp , δzp ). The feature conﬁdences Wp computed in the iteratingpprocess in each frame are also used for this error function Etotal .
By iterating this minimization for all the input images until convergence, anaccurate extrinsic camera parameters and 3-D positions of natural features canbe acquired. Local minimum and computational cost problems can be avoided
simply by a standard gradient descent method, because the initial parameters
are expected to be suﬃciently close to the true values.

4

Experiments

In this section, two kinds of experimental results are demonstrated. In the ﬁrst
experiment, the accuracy of extrinsic parameters estimated by the method described in Section 2 is evaluated by computer simulation. The second experimentis concerned with 3-D reconstruction test in real environments.
In all the experiments, Ladybug camera system [23] is used as an OMS.
As shown in Figure 2, Ladybug has radially located six camera units in thecamera block and their positions and postures are ﬁxed. Each camera can acquire
768 × 1024 resolution images at 15 fps, and the multi-camera system can capturea scene covering more than 75% of full spherical view. The intrinsic parametersof the camera block are estimated as shown in Figure 2(b) by using a laser
measurement system and a calibration board [6] in advance. From the result ofcamera calibration, it is known that displacements of adjacent camera units are
40 ± 2mm in the horizontal direction and 46 ± 4mm in the vertical direction.
4.1

Quantitative evaluation of solving PnP problem in simulation

In this section, the eﬀectiveness of the OMS for the PnP problem is quantitatively
evaluated by computer simulation. This simulation is carried out by using aExtrinsic Camera Parameter Recovery for an OMS(a) position error

0.12
0.09

1
2
3
4
5
6

f ca

ks

100

80

ndma
r

90

r of la

60

Numb
e

70

0

40

0.03

mer
a

s

0.06

50

f ca

1
2
3
4
5
6

Nu m
ber
o

ks

100

80

60

ndma
r

90

r of la

70

Numb
e

40

0

50

20

mer
a

s

40

0.15

10
20
30

60

Angle error [deg.]

80

10
20
30

Position error [mm]1000.15-0.18
0.12-0.15
0.09-0.12
0.06-0.09
0.03-0.06
0-0.03

0.18

Nu m
ber
o

100-12080-100
60-80
40-60
20-40
0-20

120

9

(b) angle error

Fig. 3. Errors in camera block position and angle (simulation).

virtual Ladybug at a ﬁxed position and feature landmarks that are randomly
scattered in 50m to 100m range of space from the virtual Ladybug. The featurelandmarks are projected to each camera of the virtual Ladybug, and are detectedwith a Gaussian noise. The Gaussian noise is set so as to have 2.0 pixel standard
deviation in each image. After this process, projected positions are quantitizedon pixels.
In this situation for solving the PnP problem, both the number of landmarks
and the number of cameras of the OMS are changed, and position errors andangle errors by the method described in Section 2 are measured. Figure 3 (a)and (b) show the computed average errors in camera position and angle witha hundred trials. It should be noted that both (a) and (b) exhibit the same
behavior: The average error monotonously decreases when the number of landmarks and cameras are increased. Especially about the number of cameras, it
is conﬁrmed that the use of the OMS is eﬀective for solving the PnP problem
accurately comparing with a single camera system.
4.2

Experiments with real scenes

To demonstrate the validity of the proposed method described in Section 3,
extrinsic camera parameters of moving Ladybug are actually reconstructed andevaluated in an indoor and outdoor environment. For both experiments, someof natural features are used as feature landmarks and their 3-D positions aremeasured by the total station (Leica TCR1105XR). These feature landmarks
are speciﬁed manually on the ﬁrst frame image and the last frame image.
Camera parameter recovery for an indoor scene: In this experiment, an
indoor scene is captured as an image sequence of 450 frames for each camera
by walking in a building as shown in Figure 4. First, the extrinsic camera parameters of Ladybug are reconstructed by the proposed method. On an average,
approximately 440 points of natural features are automatically tracked in each10

Tomokazu Sato, Sei Ikeda, and Naokazu Yokoya

Fig. 4. Sampled frames of input image sequences obtained by six cameras (indoor
scene).

(a) top view(b) side viewFig. 5. Result of extrinsic camera parameter estimation (indoor scene).

set of frames by six cameras. The squared average of re-projection errors of thefeatures is 2.1 pixels. Figure 5 shows the recovered extrinsic camera parametersof the camera 1 of the Ladybug. The curved line in this ﬁgure indicates thecamera path, the quadrilateral pyramids indicate the camera postures drawn at
every 20 frames. The black point clouds show the estimated 3-D positions of thenatural features. The length of the recovered camera path is 29m. As shown inthis ﬁgure, the camera parameters are recovered very smoothly.

Extrinsic Camera Parameter Recovery for an OMS

11Fig. 6. Sampled frames of input image sequences obtained by six cameras (outdoor
scene).

(a) top view(b) side viewFig. 7. Result of extrinsic camera parameter estimation (outdoor scene).

Camera parameter recovery for an outdoor scene: An outdoor scene iscaptured by walking in our campus including several buildings as shown in Figure
6. The image sequence for each camera consists of 500 frames. The distances
between the camera system and objects in this scene are longer than thosein the indoor scene. In this experiment, approximately 530 points of naturalfeatures on an average are automatically tracked in each set of frames by six
cameras. The squared average of re-projection errors of the features is 1.6 pixels.

12

Tomokazu Sato, Sei Ikeda, and Naokazu Yokoya

Figure 7 shows the recovered extrinsic camera parameters of the camera 1 of the
Ladybug. The length of the recovered camera path is also 29m. As same as theindoor environment, the camera parameters are recovered very smoothly.
Quantitative evaluation for real scenes: The recovered camera paths andpostures are evaluated by comparing with the ground truth. The ground truth
is made by solving the PnP problem in every frame. For obtaining the groundtruth, features in the input images are manually tracked throughout the input
sequences and their 3-D positions are measured by the total station.
Figure 8 denotes position errors and posture errors for the indoor data. The
average estimation errors in position and posture of the camera system before
global optimization are 50mm and 0.49degree, respectively. After global optimization, they are reduced to 40mm and 0.12degree, respectively. We can conﬁrm that the accumulation of estimation errors is reduced by global optimization.
As same as for the indoor sequence, Figure 9 illustrate position errors and posture errors about the outdoor scene. In this sequence, the average estimation
error before global optimization is 280mm(position) and 1.10degrees(angle). After the global optimization, they are reduced to 170mm(position) and 0.23degrees(angle), respectively. It is also conﬁrmed that the accumulation of the estimation errors is eﬀectively reduced by global optimization. Note that the average
errors of the outdoor scene are larger than those of the indoor scene, because
the scale of the outdoor scene is several times larger than the scale of the indoor
scene. Although these errors are considered as signiﬁcantly small comparing withthe scale of each scene, more accurate reconstruction could be accomplished by
specifying additional feature landmarks, for example, in the middle frame.

5

Conclusion

This paper has proposed a method for recovering extrinsic camera parameters ofan OMS. In the proposed method, ﬁrst, the PnP problem for an OMS is solved torecover an extrinsic camera parameter using feature landmarks. Next, extrinsic
parameters of the OMS are estimated by using an SFM approach which is basedon tracking both feature landmarks and natural features.
In the experiments, the eﬀectiveness of the use of an OMS for PnP problem
is quantitatively examined by computer simulation. Additionally, the extrinsic
camera parameter recovery with real scenes is successfully demonstrated using
multiple long image sequences captured by a real OMS: Ladybug. In future work,
the recovered camera parameters will be applied for dense 3-D scene reconstruction of outdoor environments.

Extrinsic Camera Parameter Recovery for an OMS
1.4

200
180

before global optimization

160

after global optimization

before global optimization
1.2

Posture error [deg.]

Position error [mm]

13

140
120
100
80
60
40
20
0

after global optimization
1.0
0.8
0.6
0.4
0.2
0.0

1

51

101

151

201

251

301

351

401

1

51

101

151

201

251

301

351

401

Frame

Frame

(a) position error(b) posture error

Fig. 8. Errors in estimated camera path and posture (indoor scene).
2.5

800

Posture error [deg.]

Position error [mm]before global optimization

before global optimization

700

after global optimization

600
500
400
300
200
100
0

after global optimization

2.0

1.5

1.0

0.5

0.0
1

51

101

151

201

251

301

Frame

(a) position error

351

401

451

1

51

101

151

201

251

301

351

401

451

Frame

(b) posture error

Fig. 9. Errors in estimated camera path and posture (outdoor scene).

References
1. K. Miyamoto: “Fish Eye Lens,” Jour. of Optical Society of America, Vol. 54, No.
2, pp. 1060–1061, 1964.
2. K. Yamazawa, Y. Yagi and M. Yachida: “Omnidirectional Imaging with Hyperboloidal Projection,” Proc. Int. Conf. on Intelligent Robots and Systems, Vol. 2,
pp. 1029–1034, 1993.
3. S. K. Nayar: “Catadioptic Omnidirectional Cameras,” Proc. IEEE Int. Conf. on
Computer Vision and Pattern Recognition, pp. 482–488, 1997.
4. J. Shimamura, H. Takemura, N. Yokoya and K. Yamazawa: “Construction and
Presentation of a Virtual Environment Using Panoramic Stereo Images of a Real
Scene and Computer Graphics Models,” Proc. 15th IAPR Int. Conf. on Pattern
Recognition, Vol. IV, pp. 463–467, 2000.
5. H. Tanahashi, K. Yamamoto, C. Wang and Y. Niwa: “Development of a Stereo
Omni-directional Imaging System(SOS),” Proc. IEEE Int. Conf. on Industrial Electronics, Control and Instrumentation, pp. 289–294, 2000.
6. S. Ikeda, T. Sato and N. Yokoya: “High-resolution Panoramic Movie Generationfrom Video Streams Acquired by an Omnidirectional Multi-camera System,” Proc.

14

Tomokazu Sato, Sei Ikeda, and Naokazu Yokoya

IEEE Int. Conf. on Multisensor Fusion and Integration for Intelligent System,
pp. 155–160, 2003.
7. R. Horand, B. Conio and O. Leboullex: “An Analytic Solution for the Perspective
4-Point Problem,” Computer Vision, Graphics, and Image Processing, Vol. 47,
pp. 33–44, 1989.
8. J. S. C. Yuan: “A General Photogrammetric Method for Determining Object Position and Orientation,” IEEE Trans. on Robotics and Automation, Vol. 5, No. 2,
pp. 129–142, 1989.
9. R. Krishnan and H. J. Sommer: “Monocular Pose of a Rigid Body Using Point
Landmarks,” Computer Vision and Image Understanding, Vol. 55, pp. 307–316,
1992.
10. R. Klette, K. Schluns and A. koschan Eds.: Computer Vision: Three-dimensional
Data from Image, Springer, 1998.
11. C. S. Chen and W. Y. Chang: “Pose Estimation for Generalized Imaging Device
via Solving Non-perspective N Point Problem,” Proc. IEEE Int. Conf. on Robotics
and Automation, pp. 2931–2937, 2002.
12. P. Beardsley, A. Zisserman and D. Murray: “Sequential Updating of Projective
and Aﬃne Structure from Motion,” Int. Jour. of Computer Vision, Vol. 23, No. 3,
pp. 235–259, 1997.
13. C. Tomasi and T. Kanade: “Shape and Motion from Image Streams under Orthography: A Factorization Method,” Int. Jour. of Computer Vision, Vol. 9, No.
2, pp. 137–154, 1992.
14. M. Pollefeys, R. Koch, M. Vergauwen, A. A. Deknuydt and L. J. V. Gool: “Threedimentional Scene Reconstruction from Images,” Proc. SPIE, Vol. 3958, pp. 215–
226, 2000.
15. J. Gluckman and S. Nayer: “Ego-motion and Omnidirectional Cameras,” Proc. 6th
Int. Conf. on Computer Vision, pp. 999–1005, 1998.
16. M. Etoh, T. Aoki and K. Hata: “Estimation of Structure and Motion Parameters
for a Roaming Robot that Scans the Space,” Proc. 7th Int. Conf. on Computer
Vision, Vol. I, pp. 579–584, 1999.
17. C. J. Taylor: “VideoPlus,” Proc. IEEE Workshop on Omnidirecitonal Vision, pp. 3–
10, 2000.
18. B. Triggs, P. McLauchlan, R. Hartley and A. Fitzgibbon: “Bundle Adjustment a
Modern Synthesis,” Proc. Int. Workshop on Vision Algorithms, pp. 298–372, 1999.
19. K. Kanatani: Statistical Optimization for Geometric Computation: Theory andPractice, Elsevier Science, 1998.
20. T. Sato, M. Kanbara, N. Yokoya and H. Takemura: “Dense 3-D Reconstruction ofan Outdoor Scene by Hundreds-baseline Stereo Using a Hand-held Video Camera,”
Int. Jour. of Computer Vision, Vol. 47, No. 1-3, pp. 119–129, 2002.
21. C. Harris and M. Stephens: “A Combined Corner and Edge Detector,” Proc. Alvey
Vision Conf., pp. 147–151, 1988.
22. M.A. Fischler and R.C. Bolles: “Random Sample Consensus: A Paradigm for Model
Fitting with Applications to Image Analysis and Automated Cartography,” Communications of the ACM, Vol. 24, No. 6, pp. 381–395, 1981.
23. Point Gray Research Inc.: “Ladybug,”
http://www.ptgrey.com/products/ladybug/index.html.