Calibrating and Centering Quasi-Central Catadioptric Cameras
Miriam Schönbein1 , Tobias Strauß1 and Andreas Geiger2

Abstract— Non-central catadioptric models are able to copewith irregular camera setups and inaccuracies in the manufacturing process but are computationally demanding and thus notsuitable for robotic applications. On the other hand, calibrating
a quasi-central (almost central) system with a central model
introduces errors due to a wrong relationship between theviewing ray orientations and the pixels on the image sensor.
In this paper, we propose a central approximation to quasicentral catadioptric camera systems that is both accurate andefficient. We observe that the distance to points in 3D is typicallylarge compared to deviations from the single viewpoint. Thus,
we first calibrate the system using a state-of-the-art noncentral camera model. Next, we show that by remapping theobservations we are able to match the orientation of the viewing
rays of a much simpler single viewpoint model with the true rayorientations. While our approximation is general and applicable
to all quasi-central camera systems, we focus on one of themost common cases in practice: hypercatadioptric cameras. We
compare our model to a variety of baselines in synthetic andreal localization and motion estimation experiments. We showthat by using the proposed model we are able to achieve nearnon-central accuracy while obtaining speed-ups of more than
three orders of magnitude compared to state-of-the-art noncentral models.

I. I NTRODUCTION
Recently, catadioptric cameras which combine the principles of refraction and reflection in one single optical systems
have gained popularity in robotics. They are able to establish
a 360◦ field of view with a very flexible geometry as theshape of the reflecting surface is a powerful design factor.
Furthermore, efficient central projection models exist thatallow for computing the point of reflection by intersecting
a line with a quadric in case the single viewpoint (SVP)
condition is fulfilled. Also, manufacturing costs are low as
catadioptric cameras merely require combining a classicalperspective camera with a mirror coated surface. Applications include robotic perception [1], [2] as well as driverassistance systems [3], [4], amongst others.
Catadioptric systems are usually designed to closely satisfy the SVP condition, i.e., all light rays are assumed tointersect at a single viewpoint. While leading to simple
projection models [5], [6], the SVP assumption is often
violated in practice [7], [8] (in the following we call suchcameras ’quasi-central’). The reason for this is that off-theshelve catadioptric cameras often use varifocal lenses wherethe viewpoint depends non-linearly on the focus and the1 Miriam Schönbein and Tobias Strauß are with the Institute of Measurement and Control Systems, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany. {strauss,miriam.schoenbein}@kit.edu
2 Andreas Geiger is with Max Planck Institute for Intelligent Systems, Perceiving Systems Department, 72076 Tübingen, Germany.

andreas.geiger@tue.mpg.de

True viewing ray
Viewing rays after centering

Centered
viewpoint
MirrorOptical axis

Catadioptric Mirror and Calibration Targets (<1m)

Object of Interest (>1m away)

Fig. 1. Illustration of Centered Camera Model. This figure illustrates
the true viewing rays emanating from the optical system (bold) as well as
the rays of our centered approximation (dashed). Note that the distance tothe objects of interest is usually very large compared to the deviation of therays from the centered single viewpoint. Thus getting the ray orientationright is more important than considering translational errors.

focal length. Furthermore, precisely aligning the perspective
camera center with the optical axis of the mirror is hard inpractice. While a number of non-central projection models
have been proposed [9], [10] they are slow as they either require non-linear optimization [9] or finding the solution of acomplex root-finding problem [10]. This prevents them frombeing applied whenever real-time algorithms are required andcomputational resources are scarce.
In this paper, we take advantage of the fact that the
distance to points in 3D is often large compared to thedeviations from the SVP as illustrated in Fig. 1. This leads us
to the conclusion that getting the orientation of the viewing
rays right is more important than considering the translational
deviation from the SVP correctly. Unfortunately, calibrating
a quasi-central catadioptric camera using traditional SVP
models [11], [12] introduces a bias in the orientation of theviewing rays due to the fact that the calibration patterns arepresented in the vincinity of the camera.
Instead of using a central camera model for calibration, we
propose to first calibrate the camera using a state-of-the-art
non-central model [10] which we extend by a perspective
camera and an appropriate distortion model in order toobtain the viewing ray orientations. Next, we remap theobservations in a way such that the viewing ray orientationsof a much simpler single viewpoint model coincide withthose of the more accurate non-central model as illustrated
in Fig. 1. This leads to a mapping where points at infinity are
projected to the same pixels as in the non-central model andapproximation accuracy gracefully degrades in the immediate
vicinity of the camera center.
In our experiments we show that our model combines theefficiency of a single viewpoint model with the accuracy of afull (but slow) non-central model in tasks such as localization
or motion estimation. Towards this goal, we construct a
dataset using two catadioptric cameras and 17 landmarks. For

performance evaluation we consider the 3D displacement inchallenging monocular and stereo localization experiments
using triplets of landmarks. We believe that these efforts
are necessary for a fair comparison and to provide novel
insights into the properties of the models and their parameters. Amongst others, we compare against the calibration
toolboxes of Mei et al. [11] and Scaramuzza et al. [12]which are widely used in the robotics community. We also
investigate the approximation accuracy when deviating fromthe single viewpoint in axial and radial direction as well as
in terms of the distance to the points in 3D. Furthermore, weshow that our centered model results in significant speed-ups
which are important for real-time applications as in robotics
or driver assistance systems. Our data and code in form of aneasy-to-use calibration toolbox, including a fully automatic
corner and checkerboard detector, is online available1 .
II. R ELATED W ORK
Recently, a taxonomy for omnidirectional camera calibration methods has been proposed by Puig et al. [7].
They classify calibration methods into five catagories: Linebased calibration [13], [14], [15], 2D pattern calibration [12],[11], 3D pattern calibration [16], self-calibration [9] andpolarization imaging [17]. As 2D patterns are easy to employ
and constrain the problem sufficiently well, we focus onplanar checkerboards as calibration targets in this paper.
The projection models that have been developed foromnidirectional cameras can be grouped into central andnon-central models. The central case has been intensely
researched over the last decades [5], [18], [14], [19], [20],[16], [15]. For instance, Geyer et al. [6] propose the spherecamera model which unifies all central catadioptric models.
To account for the properties of real lenses, Mei et al. [11]add a perspective camera with distortions and provide theirimplementation2 . Another widely used calibration toolbox3
has been developed by Scaramuzza et al. [12] which describes the imaging function using a polynomial.
Unfortunatley, the SVP condition is hard to fulfill inpractice [9] and even small deviations can impact performance when applying the model to accuracy sensitive tasks
such as 3D reconstruction, localization or motion estimation.
Furthermore, permitting more flexible arrangements allows
to compromise image characteristics such as field of view
with spatial resolution [8]. Consequently, a number of models
for non-central catadioptric systems have been proposed.
Early approaches [21], [9], [22], [23] make use of computationally expensive non-linear optimization and require
an initial estimate of the pixel coordinates. Goncalves et al.
[24] increase efficiency by reducing the optimization to a 1D
search problem, but still require ∼ 200 seconds for projecting
10.000 3D points to the image plane.
Recently, Agrawal et al. [10] derived an analytical forward projection (AFP) which requires solving an 8th degree
polynomial equation. While the model has been applied in a
1 http://www.mrt.kit.edu/software/
2 http://www.robots.ox.ac.uk/cmei/Toolbox.html
3 https://sites.google.com/site/scarabotix/ocamcalib-toolbox

Fig. 2. Geometric Projection Model with Parameters. This figure shows
the setup of the full non-central model which we use for obtaining the rayorientations in our centered approximation.

bundle adjustment context with known intrinsics, no analysis
in the context of calibrating the intrinsics and extrinsics
jointly has been carried out yet. In this paper, we combine
the AFP model with a perspective camera including lens distortions and derive an approximation that is more than threeorders of magnitude faster (i.e., five orders of magnitudewith respect to [24]), yet achieves similar accuracy in our
experiments. Our results also reveal which parameters areimportant for modeling axial as well as lateral deviations
from the single viewpoint, insights which have been missing
in existing comparisons [7]. Furthermore, we investigate theimpact of parameter initialization and suggest modifications
to [12], resulting in improved convergence.
III. P ROJECTION M ODEL
We start with a description of the geometric model [10]which we use as base model to obtain the ray orientations
for the proposed centered model. While this specific choiceof base model makes the assumption of a quadric mirror, ourcentered approximation (Section III-B) is more general andrequires estimated light rays as the only input.
A. Geometric Model for Quadric Mirrors
Let us assume a quadric mirror with parameters A, B and C
that can be described as
x2 + y2 + Az2 + Bz −C = 0(1)

Further, let p denote the 3D world point, m a point on the
mirror surface and c the perspective camera center (all ∈ R3 )as illustrated in Fig. 2. Let the indices m, r and c denote themirror, the rotated mirror and the camera coordinate systems,
respectively. Using this notation, our geometric projection
model maps a 3D world point p via the point of reflection
on the mirror m to a pixel (u, v)T in the image. Recently,
Agrawal et al. [10] showed that the point of reflection m
can be obtained analytically from the 3D world point p andthe camera center c. In the following, we briefly repeat theirmost important results and extend the model to include aperspective camera with distortions.
To reduce the order of the resulting polynomial forward
projection equation, the world point pm = (xm , ym , zm )T andthe camera location cm = (rm , sm ,tm )T are rotatedt (sm + ε) −t rm 0
pr = RR pmt (sm + ε) 0 
RR =  t rm
cr = RR cm
001(mr − s)T · [(pr − cr ) × (s − cr )] = 0(2)

The first constraint is obtained by solving (2) for zr andplugging the result into the mirror equation (1). Similarly, asecond constraint can be derived from the law of reflection.
Combining both yields an 8th degree polynomial equation in
zr whose roots can be computed via eigenvalue decomposition of the companion matrix, yielding mr .
We now extend the model of Agrawal et al. [10] by aperspective camera that includes a lens distortion model.
First, we rotate the point of reflection mr back into mirror
coordinates and map it into the camera coordinate system:
mc = (xc , yc , zc )T = RC RT
R mr + tC

with rn =p
xn2 + y2n and projected to the image via 
fu α fu cu  qq=  0 f v cv  · d
1
1
0 0 1

0.02
0.01
0
−0.01
−0.02
−0.03
−0.04
−0.05
−0.05

0
x [m]

0.05

Fig. 3. Centered Viewpoint Estimation. The viewpoint v (red cross) of
the centered model is obtained as the one that minimizes the distance to allreflected rays wr (red). The camera center c and the camera rays wc are
shown in blue.

Order: 1, Residual: 25.6 px

Order: 2, Residual: 8.2 px

Order: 3, Residual: 2.5 px(3)Here, RR is the pre-rotation matrix from above and Rc |tC
denotes the 3D rigid transformation from mirror to camera
coordinates. Let qn = (xn , yn )T = (xc /zc , yc /zc )T be the normalized projection. The distorted point is given by
qd = (1 + k1 rn2 + k2 rn4 + k5 rn6 ) qn

2k3 xn yn + k4 (rn2 + 2xn2 )+k3 (rn2 + 2y2n ) + 2k4 xn yn

0.03

z [m]

T
such that the rotated camera
p cr = (0, sr ,tr ) aligns with2
2
the y/z plane, where t = 1/ rm + (sm + ε) . Note that incontrast to [10] we have introduced a small positive scalar εwhich regularizes RR against the identity matrix and prevents
singularities in case rm and sm are both small.
Let π denote the plane of reflection on which pr , cr andthe (rotated) point of reflection mr = (xr , yr , zr )T are located.
π can be represented by pr , cr as well as the intersection of
π with the z axis of the mirror coordinate system, which is
given as s = (0, 0, zr −Azr −B/2)T . As (mr −s) is orthogonal
to the normal of π, we have

Order: 1, Residual: 39.8 px Order: 2, Residual: 24.3 px Order: 3, Residual: 20.3 px

Fig. 4. Centered Residual Field for re-mapping the observations of thecentered model for polynomials of different order. Top row: SVP scenario.
Bottom: 5 mm deviation from the SVP.

(4)

(5)

where q = (u, v)T denotes a pixel in the image and f , c, α, k
are the intrinsic parameters of the camera which combinedwith A, B and C define the set of all calibration parameters.
B. Centered Model
While the presented non-central projection model is accurate, our experiments reveal that the complex analytic formof the AFP renders the projection slow compared to simple
single viewpoint models, preventing real-time applications,e.g., on mobile devices.
However, we observe that in many situations of practicalrelevance the distance to the objects of interest is large
compared to the deviation from the single viewpoint. Thus,
given a calibrated non-central model we are able to specify
a central approximation which maps observations according
to the image residuals for points at infinity, i.e., in a way
which lets the viewing ray orientations coincide with the true
ones. Note that when applying the model this mapping canbe pre-computed and efficiently applied to the whole input
image or individual feature points, similar to undistortionor rectification maps for perspective cameras. Importantly,
note that our representation is able to represent any SVP
model accurately, i.e., without approximation, as long as the
mapping between the observations is smooth.
As detailed in the following sections, we first compute theoptimal single viewpoint, i.e., the point that comes closest
to all viewing rays as illustrated in Fig. 3. In a second step,
we derive an efficient centered camera model which we use
to remap the image observations (see Fig. 4). Importantly,
the quality of the centered model itself does not depend on
the actual choice of the projection function as all centeredmodels establish the same relationship between (remapped)
pixel coordinates and the corresponding viewing ray orientations as detailed in the appendix. While our exposition isbased on the geometric model from the previous section, theproposed centered model can be utilized as surrogate for any
other projection function as well.
Viewpoint: Let c be the estimate of the true camera position,
obtained by calibration using the non-central model presented in Section III-A. Here, we drop the mirror coordinate
index m for clarity. Denote Ω = {m} the set of all points
on the mirror surface, defined by Eq. 1. For any m ∈ Ω, thereflected ray wr (m) can be computed as
wT
cn
nT n
with camera ray wc and normal n given by
wr = wc − 2n

m−c
wc =
km − ck2

v = argminṽΩn = (x, y, Az + B/2)

5

kṽ − mk22 − ([ṽ − m]T wr (m))2 dm(8)v − m − wr (m) (v − m)T wr (m) dm = 0(9)Ωwhich is linear in v. This integral can be approximated toarbitrary precision by a summation over a discretized set of
surface points {m}. In practice, we found 2, 500 equidistantly
sampled rays to yield a sufficiently good approximation.
The optimal viewpoint v is illustrated in Fig. 3 for a 2D
example. Note that even in the case of deviations from the
SVP model, distant objects are mapped accurately due to thesmall displacement of v from the viewing ray. Importantly,
the orientation of each viewing ray is left unaltered by meansof a residual field applied to the observations which we
describe in the following section.
Centered Camera Model: Given the optimal viewpoint v,
we seek for a simple and efficient camera model that maps aworld point p = (x, y, z)T into image coordinates q = (u, v)T .
Note that here q refers to the remapped image observations
as described in the following.
We propose an angle-based representation
  k


c
cos θ
q(ϕ, θ ; cu , cv , γ) = u + ∑ γi ϕ i
(10)
cv
sin θ
i=0
with polynomial order k, pitch ϕ and yaw angle θ of thelight ray defined by!
yz
ϕ(p) = atan pθ (p) = atan
(11)xx2 + y2Here, (cu , cv )T denotes the location of the image center and
γ = (γ0 , . . . , γk )T are the polynomial coefficients that describe
the relationship between the pitch angle and the distancefrom the image center. The parameters of the model (cu , cv , γ)
are obtained via non-linear least-squares: Using Eq. 6, we
calculate the viewing ray orientations ϕ and θ for each pixel
q0ϕ,θ in the calibrated geometric camera model and minimize
2

cu , cv , γ = argmin ∑ kq0ϕ,θ − q(ϕ, θ ; c̃u , c̃v , γ̃)k2(12)c̃u ,c̃v ,γ̃ ϕ,θ

to obtain the parameters of the centered model cu , cv , γ.
Here, q0ϕ,θ denotes the pixel associated with ϕ and θ .
The image residuals q0 − q after optimization define the

0

0

(7)

The solution to Eq. 8 is given by the integral equation
Z

2
1
0
2

T

Our goal is to find the viewpoint v that is optimal in theleast-squares sense, i.e., that minimizes the squared distanceto the set of all reflected rays {m + λ wr (m) | m ∈ Ω}. Thisrequirement can be formalized as
Z

Z [m]

(6)

−2
X [m]

(a) Camera

−5
−4

−10

Y [m]

(b) Landmarks for Localization Experiments

Fig. 5. Experimental Setup. Fig. (a) shows the omnidirectional camera
we use (VS-C450U). Fig. (b) depicts the landmarks used in our localization
experiments in red, their footpoints (z = 0) in black and the true camera
locations of our stereo setup in blue.

residual displacement field which is applied to the image
observations before using the centered model projection.
Note that the degree of the polynomial in Eq. 10 does not
impact the quality of the approximation, but only affects thesmoothness of the residual displacement field, as illustrated
for polynomials of differing order in Fig. 4.
Experimentally, we found a third order polynomial (k =3) sufficient for providing smooth displacement fields. By
initializing cu , cv to half the image size and γ to zero
the optimization of Eq. 12 always converged to the true
parameters. For a given camera calibration, the displacements
can be densely precomputed. When applying the centeredmodel, all observations are mapped through this field suchthat the centered model in Eq. 10 applies. Note that for points
at infinity (or all central models) the centered model is exact,i.e., it is equivalent to the model it is derived from. For details
we refer the reader to the appendix.
IV. E XPERIMENTAL E VALUATION
We evaluate the proposed centered model aginst a varietyof popular calibration models [12], [11], [6] on real experiments and in simulation. Towards this goal, all projection
models have been integrated into a multi-camera bundleadjustment framework.
A. Localization
Our setup consists of two hypercatadioptric cameras (seeFig. 5(a)) as it could be used in robotics or driver assistanceapplications. In our setup, the camera violates the single
viewpoint assumption by approximately 20 mm in axial and
1 mm in lateral direction. The location of the cameras ismarked in blue in Fig. 5(b). Furthermore, we installed 17
landmarks (red circles in Fig. 5(b)) at various heights (0−2.5m) and distances (2.5 − 10 m), spanning an area of 10 × 20
meters.
For accurate 3D ground truth we made use of a highprecision laser range finder to measure the distances between
all pairwise combinations of cameras and landmarks, aswell as their heights above ground level (black lines in Fig.
5(b)). We initialized the 3D camera and landmark locations
manually and refined them by minimizing all distance andheight errors using non-linear least-squares.

To obtain the calibration parameters for all camera models
we presented 67 checkerboard calibration patterns to thecameras and automatically extracted the corners at subpixel accuracy using [25], which we extended by a tracking stage and modified to better handle catadioptric image
distortions. In particular, we apply the detector of [25] on
two image scales and predict corners non-linearly in theassociation stage. Note that for calibration we only makeuse of the checkerboard calibration patterns for which thepose is unknown (allowing for a simple standard calibration
procedure). The 3D landmark locations are not used forcalibration but only in our localization experiment. Using
the automatically detected checkerboard corners, we jointly
optimize the intrinsic and extrinsic calibration parameters of
the catadioptric stereo camera rig described in the previous
section using the geometric model, the centered model, theunified central model [6] and the models of Mei et al.
[11] (unified+distortions) and Scaramuzza et al. [12]. We
initialize the mirror parameters of the geometric model to the
manufacturer settings which is a reasonable assumption forcommercial systems where these values are usually available.
All other parameters are initialized automatically assuming a
SVP model. The parameters of the competing methods havebeen initialized as proposed by the authors.
We evaluate camera localization performance by minimizing the reprojection errors of the landmarks in the images
using a monocular and a stereo setup. We selected 29 noncollinear landmark triplets as minimum sets for localization.
Our results are shown in Table I. For the geometric model,
we always calibrate the focal length and the principal point
parameters. Distortions (k), perspective camera pose (C)and mirror parameters (M) are optimized when indicated.
Note that single viewpoint models without distortions (firstrow and last row) fail completely in capturing the systemgeometry. Including distortion parameters (e.g., second row,
Mei et al.) improves calibration results. As expected, using
a full non-central model yields the best results, e.g., reduces
stereo localization errors by 23% with respect to Mei et al.
[11] and by 29% with respect to Scaramuzza et al. [12]Importantly, note that our centered model achieves almost thesame performance as the geometric model from which it has
been derived (in this case we used Geometric+C, third row)while being significantly faster as shown in Section IV-E. In
the absence of suitable regularizers, including all parametersof the geometric model degrades calibration quality due tothe complex interplay between the mirror, camera center anddistortion parameters. This overfitting behavior also indicatesthat reprojection errors – which are often employed to judge
calibration performance – are insufficient to assess the true
accuracy of a calibration model.
Furthermore, we observed that the method of Scaramuzza
et al. [12] is sensitive with respect to the initialization
when deviating from the single viewpoint. We found thereason for this to be mainly numerical instabilities which
can be mitigated by normalizing the polynomial coefficients
approprietly. Using this modification and optimizing allcameras and checkerboards jointly as opposed to the two-

MethodGeometric
SVP Model
Geometric
Non SVP
Model

Parameter

k
C
C+k
C+M
C+M+k

Centered Model
Scaramuzza
Improved
et al. [12]Orig.
Mei et al. [11]Geyer et al. [6]Reproj.
Local. Error
Error
Mono Stereo[Pixels] [mm][mm]1.5944 207.23 166.83
0.6241 50.89
45.56
0.5989 42.11
36.21
0.5864 40.04
34.64
0.5977 43.39
38.15
0.5850 89.31
86.43
42.14
36.26
0.6241 49.51
48.08
3.4143 771.93 687.86
0.6229 50.48
44.78
0.6421 127.45 122.17

TABLE I
Calibration and Localization Experiments. T HIS TABLE SHOWS OUR
EXPERIMENTS ON REAL DATA IN TERMS OF THE REPROJECTION ERRORSOF THE CHECKERBOARD CORNERS AFTER CALIBRATION AND THE
LOCALIZATION ERRORS , AVERAGED OVER ALL TRIPLETS .

step procedure suggested in [12] improved calibration results
significantly (’Scaramuzza Improved’ vs. ’Orig’ in Table I)compared to the original work.
B. Motion Estimation
In a second experiment, we mounted the hypercatadioptricstereo setup from Section IV-A onto our driving platform
AnnieWAY [26] which is equipped with a high-precision
GPS/IMU that delivers the ground truth motion [27]. We calibrate both cameras jointly using 70 images of checkerboards
at various locations and orientations. Next, we recorded fourchallenging inner-city and rural sequences.
Feature detection and matching between two consecutive
stereo pairs is performed using BRIEF [28] features in combination with the FAST corner detector [29] which gave thebest matching results in our experiments. Using RANSAC
for robustness against outliers, we estimate the motion
between two consecutive frames by triangulating the 3D
points in the previous frame and minimizing the reprojection
errors with respect to the observations in the current frame.
Note that while more sophisticated structure-from-motion
pipelines [30], [31], [32], [33] could be used, we focus on
the two-frame motion estimation problem here to keep things
simple and avoid side-effects from the respective method. We
expect that improvements in this task directly translate into
improvements when using more advanced algorithms.
For comparing different calibration approaches, we show
endpoint errors after 200 frames (corresponding to a driven
path of up to 300 m depending on the driving speed) for all
frames of each sequence. The results are shown in Fig. 6(d-g) , with averages given in the legend of the error plot.
Fig. 6 (b+c) shows the trajectories for sequence 1 in bird’s
eye view and from the side. Due to the large axial deviationfrom the single viewpoint (20 mm), the centered model isable to significantly reduce drift in z-coordinate direction(corresponding to altitude), while projected on the groundplane, all methods perform similarly. Overall, the proposed
centered calibration model is able to reduce 3D translation
errors by more than a factor of two compared to the baselines

2

10

r = 100 (unwarped)
r = 100 (warped)
r = 200 (unwarped)
r = 200 (warped)
r = 300 (unwarped)
r = 300 (warped)
r = 400 (unwarped)
r = 400 (warped)
r = 500 (unwarped)
r = 500 (warped)
r = 600 (unwarped)
r = 600 (warped)
r = 700 (unwarped)
r = 700 (warped)

1

10

0

Error [px]

10

−1

10

−2

10

−3

10

−4

10

1

10

2

10

3

10
Distance [mm]

4

10

5

10

Fig. 7. Approximation Errors of the Centered Model. This figure shows
the average approximation error of the projection in pixels over the distanceof the 3D point from the camera center for simulations of the camera setup in
Section IV-A. Here, r denotes the distance from the image center. The errorsof the projection without warping the observations are given for reference.

which suffer from inaccuracies in the orientations of theviewing rays.

Running Time
Numeric Non-Central [24] ∼ 185,000.00 ms
Geometric Model*
2,919.98 ms
Scaramuzza et al. [12]913.93 ms
Mei et al. [11]6.58 ms
Centered Model
3.42 ms
Centered Model*
1.79 ms
TABLE IIRUNNING TIMES FOR PROJECTING 10, 000 POINTS IN MATLAB.
M ETHODS MARKED WITH AN ASTERISK (*) ARE WRAPPED IN C++.

real catadioptric camera system described in our experiments
in Section IV-A. Fig. 7 depicts the reprojection errors withrespect to the distance of the 3D point for various radii, i.e.,
distances from the image center. For reference we also plot
the reprojection errors with respect to the original unwarped
observations. This corresponds to applying an SVP model
without distortions to the non-central calibration problem.
Fig. 7 shows that the quality of our calibration model
degrades gracefully: All errors fall below 0.1 pixel at 1 meter
distance and below 0.01 pixel at 10 meters distance, eventhough the SVP has been violated by 20 millimeters.
E. Running Times

C. Single Viewpoint Violation
To further assess the sensitivity of the methods withrespect to deviations from the SVP, we simulate a set of
scenarios by assuming a perfect hypercatadioptric system as
described by the geometric model in Section III-A using the
intrinsic and extrinsic parameters from our real camera. To
emphasize the differences we consider the noise-free casehere. We vary the location of the perspective camera laterally(x-direction) and along the mirror axis (z-direction). For eachsetting, we project the checkerboards and the landmarks onto
the image plane using the geometric model and calibrate thecameras using the previously described models. Next, we
localize the camera as described in the previous section.
Fig. 8 shows the reprojection errors of the checkerboards
after calibration (left) and the localization errors (right) foraxial (top) and lateral (bottom) displacements. Again we
observe that the localization errors for axial displacements
are small when using a model that includes distortions.
On contrast, lateral displacements impact performance muchstronger. However, note that the centered model can handle
both cases much better than the baseline methods. The factthat the reprojection errors of the centered model are larger
can be attributed to the proximity of the calibration patterns
to the camera (0.1 − 1m), compared to the landmarks (>2.5m). Again, small checkerboard reprojection errors are not
an indicator for a well calibrated camera with respect to sometarget application, e.g., localization. In fact, this confirms ourassumption that getting the viewing ray orientation right ismuch more important as small orientation errors propagate
to large translation errors at distance.
D. Approximation Analysis
In this section, we analyse the approximation propertiesof the centered model experimentally. Towards this goal,
we make use of the geometric model parameters from our

As applications (e.g., in robotics) often require real-time
performance, we analyze the time required for projecting
a 3D world point onto the image plane. Whenever the
reprojection error needs to be computed, e.g., during localization, motion estimation or 3D reconstruction, this function
is called (frequently) by the optimization routine. Table IIshows the time required for projecting 10, 000 random 3D
points to the image plane. The running time for the numeric
non-central projection has been taken from [24] and isconsidered approximate. However our results agree with theobservations in [10]. All the remaining running times havebeen computed on an Intel i7 2.67 Ghz machine using a single CPU core. As expected, the numeric optimization in the
non-central model is the slowest, requiring ∼ 185 seconds for
projecting the points. The run-time of the geometric modelis heavily dominated by the computation of the analyticalforward projection [10] (provided to us by the authors) which
involves the computation of the polynomial coefficients andthe evaluation of MATLAB’s roots function for finding
the polynomial roots. The latter is also responsible forthe relatively slow evaluation of Scaramuzza’s projection
function [12]. On contrary, the proposed centered model andthe relatively simple projection model of Mei et al. [11] arevery fast and able to project 10, 000 points in a couple ofmilliseconds only. Compared to the geometric model, ourcentered approach yields a speed-up of more than threeorders of magnitude.
V. C ONCLUSION
In this paper we have proposed to center quasi-central
catadioptric cameras while ensuring that the viewing ray
orientation matches the true one. We showed that the
proposed centered camera model approximates non-central
catadioptric systems sufficiently well as long as the distanceto the observed points is relatively large compared to the

100
GPS
Centered
Mei
Scaramuzza

y [m]

100

50

50

0

0

−50

−50

−100
0

50

100

150
x [m]

200

0

250

(a) Cameras on top of our recording Platform AnnieWAY (b) Sequence 1: Trajectory (bird’s eye view)

5

10

30
3D Error [m]1040Centered (Avg. Error 2.99 m)
Mei (Avg. Error 6.35 m)
Scaramuzza (Avg. Error 7.06 m)

5

50

100

150
x [m]

200

250

(c) Sequence 1: Trajectory (side view)
30

Centered (Avg. Error 7.09 m)
Mei (Avg. Error 15.39 m)
Scaramuzza (Avg. Error 15.57 m)
3D Error [m]15Centered (Avg. Error 3.22 m)
Mei (Avg. Error 4.50 m)
Scaramuzza (Avg. Error 5.67 m)
3D Error [m]3D Error [m]15GPSCentered
Mei
Scaramuzza

20

Centered (Avg. Error 6.89 m)
Mei (Avg. Error 14.46 m)
Scaramuzza (Avg. Error 14.72 m)

20

10

10

0
0

200

400
Frames600800(d) Sequence 1: Translation Error

0
0

200

400
600
Frames800

1000(e) Sequence 2: Translation Error

0
0

1000

2000
Frames

3000

4000(f) Sequence 3: Translation Error

0
0

500

1000
Frames1500

2000(g) Sequence 4: Translation Error

Fig. 6. Motion Estimation Experiment. This figure shows the hypercatadioptric cameras on top of our recording platform which we used in our
experiments (a), the bird’s eye and side view of the estimated and ground truth trajectories of the first (and shortest) sequence (b+c) and the end-point
errors after estimating the vehicle’s motion over 200 frames (d-g) corresponding to 0 − 300 meters for sequences of various length.

deviation from the single viewpoint, which is a reasonableassumption in practice, e.g., for mobile robots. Unlike exact
forward projections for the non-central case, its efficiency
makes the proposed projection model suitable for real-time
applications. We provide our dataset and code in form of aeasy-to-use calibration toolbox.We believe this to be a step
forward towards the practical use of catadioptric cameras
in settings where both, calibration accuracy and runtime
matters.
A PPENDIX
Claim 1: Every central base model can be represented
exactly by the centered model presented in Section III-B.
Proof: Assuming an undistorted perspective camera, every
central projection can be represented in the form   


u
c
cos θ
= u + f (ϕ)
(13)
v
cv
sin θ
where (cu , cv ) denotes the principal point, ϕ and θ are thepitch and yaw angle of the viewing ray and f is an arbitrary
monotonic and smooth function. By rearranging the terms
we see thatq

ϕ = f −1(u − cu )2 + (v − cv )2(14)
Substituting Eq. 14 into Eq. 10 yields a displacement field
which maps the image observations into the coordinates of
the centered model.
Claim 2: The centered model maps points at infinity exactly
for any non-central base model.

Proof: Let p = λ (x, y, z)T denote a 3D world point and t =(tx ,ty ,tz )T an arbitrary finite translation of the viewing raywith pitch angle ϕ and yaw angle θ , i.e.,!

λ y + ty
λ z + tz
ϕ = atan p
θ = atan
λ x + tx
(λ x + tx )2 + (λ y + ty )2For λ → ∞ we obtainϕ → atanzpx2 + y2!
θ → atanyxThus we can represent the viewing ray orientations exactly
using a centered model.

R EFERENCES[1] A. Voigtländer, S. Lange, M. Lauer, and M. A. Riedmiller, “Real-time
3d ball recognition using perspective and catadioptric cameras,” in
ECMR, 2007.
[2] O. Tahri and H. Araujo, “Non-central catadioptric cameras visual
servoing for mobile robots using a radial camera model,” in IROS,
2012.
[3] T. Ehlgen, T. Pajdla, and D. Ammon, “Eliminating blind spots forassisted driving,” vol. 9, no. 4, pp. 657–665, 2008.
[4] D. Scaramuzza and R. Siegwart, “Appearance-guided monocular omnidirectional visual odometry for outdoor ground vehicles,” vol. 24,
no. 5, pp. 1015–1026, 2008.
[5] S. Baker and S. K. Nayar, “A theory of single-viewpoint catadioptric
image formation,” IJCV, vol. 35, no. 2, pp. 1 – 22, 1999.
[6] C. Geyer and K. Daniilidis, “A unifying theory for central panoramicsystems and practical implications,” in ECCV, 2000.
[7] L. Puig, J. Bermúdez, P. Sturm, and J. J. Guerrero, “Calibrationof omnidirectional cameras in practice: A comparison of methods,”
CVIU, vol. 116, no. 1, pp. 120–137, 2012.
[8] R. Swaminathan, M. D. Grossberg, and S. K. Nayar, “Non-single viewpoint catadioptric cameras: Geometry and analysis,” IJCV, vol. 66,
no. 3, pp. 211–229, 2006.

10
0.03
SVP
SVP + kGeometric Model
Scaramuzza et al.
Mei et al.
Centered Model
Geyer et al.

0.02

8

Localization Error [mm]Reprojection Error [Pixel]

0.025

SVP
SVP + kGeometric Model
Scaramuzza et al.
Mei et al.
Centered Model
Geyer et al.

9

0.015

0.01

7
6
5
4
3
2

0.005

1
0
−20

−15

−10

−5
0
5
Distance from SVP [mm]

10

15

0
−20

20

(a) Calibration Reprojection Error (Z-Displacement)

−10

−5
0
5
Distance from SVP [mm]

10

15

20

(b) Localization Error (Z-Displacement)
120

0.5
SVP
SVP + kGeometric Model
Scaramuzza et al.
Mei et al.
Centered Model
Geyer et al.

0.4
0.35

SVP
SVP + kGeometric Model
Scaramuzza et al.
Mei et al.
Centered Model
Geyer et al.

100

Localization Error [mm]0.45Reprojection Error [Pixel]

−15

0.3
0.25
0.2

80

60

40

0.15
0.1

20

0.05
0
−10

−8

−6

−4

−2
0
2
Distance from SVP [mm]

4

6

8

10

(c) Calibration Reprojection Error (X-Displacement)

0
−10

−8

−6

−4

−2
0
2
Distance from SVP [mm]

4

6

8

10

(d) Localization Error (X-Displacement)

Fig. 8. Simulated Displacements from the Single Viewpoint. This figure shows the reprojection errors of the checkerboard corners after calibration(left) and the localization errors from the localization experiment (right) using triplets of points when displacing the camera center axially (top row) andlaterally (bottom row). The geometric model (green) which has been used for generating the data produces zero error and is given for reference only.
[9] B. Micusı́k and T. Pajdla, “Autocalibration & 3d reconstruction with
non-central catadioptric cameras,” in CVPR, 2004.
[10] A. Agrawal, Y. Taguchi, and S. Ramalingam, “Beyond alhazen’s
problem: Analytical projection model for non-central catadioptriccameras with quadric mirrors,” in CVPR, 2011.
[11] C. Mei and P. Rives, “Single view point omnidirectional camera
calibration from planar grids,” in ICRA, 2007.
[12] D. Scaramuzza and A. Martinelli, “A toolbox for easily calibrating
omnidirectional cameras,” in IROS, 2006.
[13] C. Geyer and K. Daniilidis, “Paracatadioptric camera calibration,”
PAMI, vol. 24, no. 5, pp. 687–695, 2002.
[14] J. P. Barreto and H. Araujo, “Geometric properties of central catadioptric line images and their application in calibration,” PAMI, vol. 27,
no. 8, pp. 1327–1333, 2005.
[15] X. Ying and Z. Hu, “Catadioptric camera calibration using geometricinvariants,” PAMI, vol. 26, pp. 1260–1271, 2004.
[16] L. Puig, Y. Bastanlar, P. Sturm, J. Guerrero, and J. Barreto, “Calibration of central catadioptric cameras using a dlt-like approach,” IJCV,
vol. 93, pp. 101–114, 2011.
[17] O. Morel, R. Seulin, and D. Fofi, “Catadioptric camera calibration bypolarization imaging,” in IAPR, 2007.
[18] S. Baker and S. K. Nayar, “Single viewpoint catadioptric cameras,” inPanoramic Vision. Springer, 2001.
[19] J. Barreto, “A unifying geometric representation for central projection
systems,” CVIU, vol. 103, no. 3, pp. 208–217, 2006.
[20] S. Gasparini, P. Sturm, and J. Barreto, “Plane-based calibration ofcentral catadioptric cameras,” in ICCV, 2009.
[21] V. Caglioti, P. Taddei, G. Boracchi, S. Gasparini, and A. Giusti,“Single-image calibration of off-axis catadioptric cameras using lines,”in Omnivis, 2007.

[22] D. Strelow, J. S. Mishler, D. Koes, and S. Singh, “Precise omnidirectional camera calibration,” in CVPR, 2001.
[23] M. Lhuillier, “Automatic scene structure and camera motion using acatadioptric system,” CVIU, vol. 109, no. 2, pp. 186–203, 2008.
[24] N. Goncalves and A. Nogueira, “Projection through quadric mirrors
made faster,” in Omnivis, 2009.
[25] A. Geiger, F. Moosmann, O. Car, and B. Schuster, “Automatic calibration of range and camera sensors using a single shot,” in ICRA,
2012.
[26] A. Geiger, M. Lauer, F. Moosmann, B. Ranft, H. Rapp, C. Stiller, andJ. Ziegler, “Team annieway’s entry to the grand cooperative driving
challenge 2011,” TITS, 2012.
[27] A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics:
The kitti dataset,” IJRR, vol. 32, pp. 1229 – 1235, 2013.
[28] M. Calonder, V. Lepetit, M. Ozuysal, T. Trzcinski, C. Strecha, andP. Fua, “Brief: Computing a local binary descriptor very fast,” PAMI,
vol. 34, no. 7, pp. 1281–1298, 2012.
[29] E. Rosten and T. Drummond, “Machine learning for high-speed cornerdetection,” in ECCV. Springer Berlin Heidelberg, 2006, pp. 430–443.
[30] M. Kaess, H. Johannsson, R. Roberts, V. Ila, J. J. Leonard, andF. Dellaert, “iSAM2: Incremental smoothing and mapping using theBayes tree,” IJRR, vol. 31, pp. 217–236, 2012.
[31] S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski,“Building rome in a day,” in ICCV, 2009.
[32] A. J. Davison, I. D. Reid, N. D. Molton, and O. Stasse, “Monoslam:
Real-time single camera slam,” PAMI, vol. 29, no. 6, pp. 1052–1067,
2007.
[33] P. Alcantarilla, L. Bergasa, and F. Dellaert, “Visual odometry priors
for robust EKF-SLAM,” in ICRA, 2010.