**Two-slit cameras.** Gupta and Hartley showed that the epipolar geometry of linear pushbroom cameras can be modeled by a 4 × 4 fundamental matrix, expressed in lifted point coordinates [197]:

$$ (q_{2,1}q_{2,3} \quad q_{2,1}q_{2,2} \quad q_{2,2}q_{2,3} \quad q_{2,3}^2) F \begin{pmatrix} q_{1,1}q_{1,3} \\ q_{1,1}q_{1,2} \\ q_{1,2}q_{1,3} \\ q_{1,3}^2 \end{pmatrix} = 0, $$

where $q_{j,k}$ denotes the kth homogeneous coordinate of a point in image j. Images of straight lines are hyperbolas and consequently, so are epipolar curves.

Feldman et al. have developed the fundamental matrix for more general two-slit cameras [139]. To be precise, they considered the case where a two-slit image is constructed from images acquired by a perspective camera that translates by a constant speed parallel to the image plane, such that at each acquisition instant t, the vertical pixel column at coordinate $s(t) = \alpha t + \beta$ is taken from the perspective image, where $\alpha$ and $\beta$ can be arbitrary values. The final image is obtained by concatenating these pixel columns. For any pair of values $\alpha$ and $\beta$, this construction corresponds to a two-slit camera. The first slit is the line of optical centers corresponding to the translational motion whereas the second slit depends on the values of $\alpha$ and $\beta$. The two slits, when seen along their common perpendicular, form a right angle.

Feldman et al. showed that two two-slit images constructed from the same original image sequence but with different values $\alpha$ and $\beta$ are subject to an epipolar constraint embodied by a 6 × 6 fundamental matrix, acting on the lifted image coordinates. They also revealed the general conditions under which two two-slit images have an epipolar geometry (see also Section 3.4.2); this is the case if the two cameras share one slit completely or if each slit of one camera intersects both slits of the other camera. In the first case, the stereo system has a pencil of epipolar planes, in the second case, these are replaced by epipolar quadrics.

Note that even if the cameras are not in one of these configurations, there exists a fundamental matrix for two-slit images, of size 6 × 6. It allows, as usual, to compute the epipolar curve (here, a conic) for every
image point. However, epipolar curves in the two images are no more in a one-to-one correspondence: different points on the same epipolar curve in one image, may be associated with different epipolar curves in the other image.

Khan et al. showed that there exists a fundamental matrix between a linear pushbroom panorama and a perspective image and used this for estimating the position of a query perspective image with respect to a set of georeferenced panoramic images [273].

**One-dimensional radial camera model.** Thirthala and Pollefeys showed the existence of quadrifocal tensors for the 1D radial camera model [500]; refer to Section 3.1.9 for some more details.

**The case of pure rotations.** It is well known that two images taken by a pinhole camera undergoing a pure rotation about its optical center are related by a homography. How about other central cameras?

We consider the case of cameras modeled by back-projection matrices of size $3 \times n$, acting on $a$-order lifted image coordinates. For any pair of matching points $\mathbf{q}_1$ and $\mathbf{q}_2$, we must have:

$$R \mathbf{B}_1^i \bar{\mathcal{L}}^a(\mathbf{q}_1) \sim \mathbf{B}_2^i \bar{\mathcal{L}}^a(\mathbf{q}_2),$$

where $R$ is the rotation relating the two images. This constraint can be written as:

$$\{R \mathbf{B}_1^i \bar{\mathcal{L}}^a(\mathbf{q}_1)\} \times \{\mathbf{B}_2^i \bar{\mathcal{L}}^a(\mathbf{q}_2)\} = \mathbf{0}_3,$$

which is equivalent to three bilinear constraints (only two being independent) on the lifted image coordinate vectors:

$$(\bar{\mathcal{L}}^a(\mathbf{q}_2))^T M_j \bar{\mathcal{L}}^a(\mathbf{q}_1) = 0, \quad j = 1 \cdots 3,$$

with three matrices $M_j$ of size $n \times n$ which depend on the rotation matrix $R$ and the two back-projection matrices.

There is no bijective homography in general, unlike in the pinhole case, but it is replaced by the above bilinear constraints, which allow, for any point $\mathbf{q}_1$, to compute all potential matches $\mathbf{q}_2$. In general, there are only finitely many matches. To be precise, since the matching constraints are linear in the lifted coordinates of $\mathbf{q}_2$, there are in general
at most $a^2$ potential matches (two independent constraints on $a$-order
lifted coordinates).

As for the 1D radial camera model and the case of pure rotations,
Thirthala and Pollefeys showed the existence of trifocal tensors [500,
501] (cf. Section 3.1.9).

**The case of planar scenes.** Like for pinhole cameras, the case of a planar scene is very similar to that of a rotating central camera. For example, Draréni et al. derived the expression for plane homographies for linear pushbroom cameras and proposed a calibration approach using planar grids [120]. Fitzgibbon mentioned plane homographies for the division model [145]. Sturm and Barreto showed the existence of plane homographies for all central catadioptric cameras [473].

## 4.3 Images of Lines and the Link between Plumb-line Calibration and Self-calibration of Non-perspective Cameras

As will be discussed in Section 5.2.1, non-perspective camera parame-
ters can in general be fully calibrated from images of straight lines.
There is a strong link between this, epipolar geometry and self-
calibration: from point matches between images, one can compute
the epipolar geometry. Epipolar geometry allows to compute epipolar
curves, which are nothing else than images of straight lines (of camera
rays of the respective other camera). It is thus obvious that epipo-
lar geometry allows for the self-calibration of non-perspective imaging
models, at least partially. One may even directly use any plumb-line
technique for self-calibration: from the epipolar geometry, one may
generate arbitrarily many epipolar curves and use these as input to
plumb-line calibration. This direct link between plumb-line calibra-
tion, epipolar geometry, and self-calibration is implicitly present in all
non-perspective self-calibration approaches, although it has not been
explained before in its full generality. It will be developed further in
Section 5.3.1.

**Line images.** As for the computation of a 3D line’s image, this is
particularly simple for cameras modeled by back-projection matrices.
In the case of non-central cameras and back-projection matrices of size 6 × n, operating on a-order lifted image coordinates $\mathcal{L}^a(\mathbf{q})$, the image of a 3D line $\mathbf{L}$ is computed as follows. An image point $\mathbf{q}$ lies on the line image if its back-projected camera ray intersects the line $\mathbf{L}$, i.e., if (cf. Equation (1.3)):

$$ \underbrace{\mathbf{L}^T \begin{pmatrix} 0_{3 \times 3} & l_{3 \times 3} \\ l_{3 \times 3} & 0_{3 \times 3} \end{pmatrix} \mathbf{B}^l \mathcal{L}^a(\mathbf{q})}_{\mathbf{c}^T} = 0. \qquad (4.4) $$

We immediately see that the image of $\mathbf{L}$ is the curve of order $a$ whose coefficients are given in the vector $\mathbf{c}$. In the case of $a = 2$ for example, the line image is a conic; the six coefficients defining its symmetric matrix representation, are contained in $\mathbf{c}$. An example of a line image in a linear pushbroom panorama is shown in Figure 4.2.

In the case of central cameras, we can obtain the analogous result after computing the full back-projection ray from $\mathbf{B}^f$ (the origin) and $\mathbf{B}^i \sim \mathbf{B}^i \mathcal{L}^a(\mathbf{q})$ (the direction of the ray):

$$ \begin{pmatrix} \mathbf{B}^i \mathcal{L}^a(\mathbf{q}) \\ \mathbf{0} \end{pmatrix} \sim \begin{pmatrix} \mathbf{B}^i \\ \mathbf{0} \end{pmatrix}_{6 \times n} \mathcal{L}^a(\mathbf{q}). $$

By inserting this in Equation (4.4), we compute the line image for central cameras as:

$$ \mathbf{c}^T \sim \mathbf{L}^T \begin{pmatrix} 0_{3 \times 3} & l_{3 \times 3} \\ l_{3 \times 3} & 0_{3 \times 3} \end{pmatrix} \begin{pmatrix} \mathbf{B}^i \\ \mathbf{0} \end{pmatrix} \sim \mathbf{L}^T \begin{pmatrix} \mathbf{0} \\ \mathbf{B}^i \end{pmatrix}. $$

<figure><img src="image_29.png" /><figcaption>Fig. 4.2 The image of a line in a linear pushbroom panorama.</figcaption></figure>
<figure><img src="image_30.png" /><figcaption>Fig. 4.3 Line images for the equiangular camera model. The black curve is the image of any line in the focal plane i.e., the plane containing the optical center and perpendicular to the optical axis. It is naturally a circle and corresponds to the hemispheric part of the field of view. The other curves show the images of lines whose interpretation planes (plane spanned by a line and the optical center) form angles of 70, 50, 30, and 10 degrees with the optical axis, respectively. The limiting case of 0 degrees corresponds to lines that are coplanar with the optical axis and whose images are lines going through the distortion center. Although in general, line images are not algebraic curves, the parts within the hemispheric field of view, i.e., within the black circle, can be relatively well approximated by conics.</figcaption></figure>

Let us denote by $l$ the second sub-vector of length 3 of $\mathbf{L}$. This represents the line at infinity of the plane spanned by the optical center (the origin) and $\mathbf{L}$. The line image can then also be written as:

$$ \mathbf{c}^{\mathrm{T}} \sim l^{\mathrm{T}} \mathbf{B}^{i}. $$

**Non-polynomial back-projection.** In this case, line images cannot be computed by matrix-vector products. Still, it suffices to plug into Equation (4.4) the expression of back-projected image points to get an equation defining the line image, allowing for example to plot it. For example, Figure 4.3 shows line images for the equiangular camera model; the figure suggests that although they are not algebraic curves, e.g., are not conics, they can be well approximated by conics within a hemispheric field of view.
This section aims at providing an overview of different calibration approaches, mainly with respect to the type of images or other information that are used for calibration, with an emphasis on approaches developed for omnidirectional cameras. The section is a far from exhaustive treatment of the topic of calibration. A few other references are as follows. An excellent overview of photogrammetric calibration approaches and their historical development, together with the development of distortion models, is the paper [95] by Clarke and Fryer, see also a paper by Remondino and Fraser [426] and a book section by Boland [55]. Among the many books covering the topic, we may cite the Manual of Photogrammetry, its fourth edition of 1980 [451] and the more recent fifth edition [336]. The article [430] by Roelofs is an interesting extended account of the state of the art on calibration up to 1951. Other relevant books are [19, 195, 213, 287] to cite just a few.

## 5.1 Calibration Using Calibration Grids

### 5.1.1 Three-Dimensional Grids

This is probably the most well-known calibration approach and will not be detailed much further. Let us just note that in order to calibrate omnidirectional cameras, several researchers used hollow
3D calibration objects painted with markers or patterns on the inside, such that cameras are put inside the object for calibration, see, e.g., [23, 61, 83, 413, 455, 506, 519, 552]. Usually, the object was used to gather 3D-to-2D matches which could then be used like in traditional calibration approaches. Ying and Hu used two hemispherical calibration objects with different radius to calibrate a model akin to the two-plane one (cf. Section 3.1.3) [552].

Among the many other approaches, let us mention one by Shih et al. who proposed a method for calibrating one radial distortion coefficient as well as the perspective camera parameters, via the solution of an 8 × 8 eigenvalue problem [448]. This is very similar to recent approaches for the simultaneous estimation of multi-view geometry (e.g., the fundamental matrix) and a distortion model (e.g., the division model), for example the approaches of Fitzgibbon and Claus [98, 145] and Mičušik and Pajdla [344, 346, 347, 349]. Shih et al. also gave a similar calibration method for the usage of planar grids.

Bastanlar et al. developed a calibration approach for central cataractic cameras that is akin to the DLT approach for pinhole cameras [40]: computation of a projection matrix from 3D-to-2D matches, followed by the extraction of intrinsic parameters. In the case of cataractic cameras, the projection matrix is somewhat different from that of a pinhole camera: as shown by Sturm and Barreto [473], it is possible to formulate a projection matrix but which does not directly map 3D points to 2D points, the reason being the existence of two theoretical image points (cf. Section 3.1.5). Instead, the projection matrix maps 3D points to a degenerate dual conic, representing these two image points. Besides this difference, DLT-like calibration is also possible, as exploited by Bastanlar et al. [40]. Gasparini et al. and Barreto et al. developed similar procedures for using planar grids [36, 159].

Huang et al. proposed a calibration approach for non-central panoramic images and compared it experimentally to a line-based and a self-calibration method [239].

### 5.1.2 Planar Grids

Many approaches have been developed for using planar grids in camera calibration, due to their ease of manufacture, storage, and use.
Planar or near-planar objects, e.g., crosses, are routinely used in photogrammetry, usually via a (self-calibrating) bundle adjustment procedure. Computer vision researchers proposed “closed-form” type solutions [475, 567], which are suboptimal and should be followed by a bundle adjustment (cf. Section 6.4).

In this section, we concentrate on plane-based calibration approaches developed for omnidirectional cameras or other cameras with general distortion profiles. There are many approaches for global camera models, which usually apply similar procedures: an optional initialization procedure, followed by a bundle adjustment. We will thus not describe them in any more detail and refer the reader to the references on camera models in Section 3.1. As for the local models, the calibration procedures designed for them are mostly already outlined in Section 3.2. In the following, we give a few details on plane-based approaches for discrete camera models, for two variants thereof — ray-based models and discrete samples of the distortion curve.

**Ray-based calibration.** Several similar approaches for ray-based or raxel-based calibration were proposed, e.g., by Gremban et al. [189], Champleboux et al. [85], and Grossberg and Nayar [191]. The principle is simple: two or more images of planar calibration grids are acquired, with a known motion of the camera or grid between the acquisitions. A dense matching between the grid in each position, and the camera image, is carried out. Hence, for each pixel, one can compute its camera ray by fitting a 3D line to the set of matched grid points, which can be put in the same coordinate frame since the motion between image acquisitions is known. This way, camera rays can be computed for each individual pixel (or some other spatial sampling of the image area, be it sub-pixel or super-pixel-wise).

Debaecker et al. used a similar procedure to calibrate their cone-based camera model from multiple images of planar grids [108]. These methods can be considered as dense versions of the two-plane approach, cf. Section 3.1.3.

This type of approach was generalized by Sturm and Ramalingam in order to allow an image acquisition without having to know the camera/grid motion [477]. While the approach based on the knowledge of camera/grid motion can in theory work with two images, a minimum
of three images is required in the case of unknown motion. Sturm and Ramalingam showed that from matches for three images, the unknown camera/grid motion can be recovered via the computation of particular trifocal tensors. Once the motion is known, the above approach can be readily applied to compute the camera rays and thus to finalize the calibration. In [477], the minimal case of three images was considered; this was extended toward using multiple images in [418], allowing for an easier calibration of omnidirectional cameras. Sturm and Ramalingam developed several variants of their approach, for using planar as well as 3D calibration grids, as well as for calibrating fully non-central [477], central [477], or axial cameras [420]. In [416], they gave a minimal solution for the calibration of a central camera using matches for four pixels and three grids.

Gonçalves and Araújo used this approach to first compute pixel-ray matches, to which they then fitted a catadioptric model [180].

Dunne et al. used a structured-light type approach where an LCD screen is used as planar calibration grid, allowing to achieve dense matches between the grid and the image through the display of a sequence of particular patterns [125]. Matches between the grid and one of the input images are used to define a warping from the image to the grid. The warping, when applied to that input image, will by definition lead to a perspectively correct image of the grid — the grid itself. This warping thus eliminates all non-perspective distortions. After applying it to other images of grids (they do not need to be of the same grid), one may use any calibration approach for perspective cameras to calibrate the perspective part of the camera projection, e.g., the methods of Zhang or Sturm and Maybank for planar grids [475, 567]. This approach combines the ideas of warp-based approaches (cf. Section 3.2.2) and ray-based calibration.

**Discrete sampling of the distortion curve.** In the 1950s, Hallert developed formulas for non-linearly optimizing either intrinsic or extrinsic parameters of a camera looking at a planar grid, supposing the respective other set of parameters is known [201]. He also included the computation of a radial distortion model in his procedure. The model he used is of the discrete sampling type (cf. Section 3.3.2): he
used a calibration grid of 5 × 5 regularly distributed points and seemed
to assume that the grid can be positioned such that the center point
coincides with the distortion center. Then, he computed the radial dis-
tance in the undistorted image, for every circle centered in the grid's
center point and going through other points in the grid. From the com-
puted values, he seemed to interpolate the complete distortion function,
between distorted and undistorted radial distances. This is probably
one of the first examples where radial distortion is determined by com-
puting samples of the distortion function, as also done for example by
Hartley and Kang [209] and Tardif et al. [491], see below.

Hartley and Kang proposed a set of methods for calibrating cameras
with general radial distortion [209]. Their key observation is that point
matches between a planar calibration grid and an image with what-
ever radial distortion are subject to a constraint akin to the epipolar
constraint and that the epipole in the image plane is nothing else than
the center of radial distortion. The latter may thus be computed by
fitting a fundamental matrix to the point matches and extracting the
epipole from it (one may also use multiple images and fundamental
matrices to get a better estimate). It was shown that this can also be
done, in a similar fashion, when using a 3D calibration grid or by self-
calibration, i.e., from images of an unknown scene (see Section 5.3.1).
Hartley and Kang then used the assumptions of a radially symmetric
camera and of a monotonic distortion curve, to compute the latter,
from multiple images of grids. They showed how to compute a discrete
sampling of the distortion curve as well as how to fit parametric models
thereof.

Hartley and Kang's approach was studied experimentally by
Barreto et al. for the task of calibrating endoscopic cameras using pla-
nar grids [37]. It was found to be superior to using the division or a
classical model for radial distortion.

Tardif et al. proposed two approaches for plane-based calibra-
tion of a general radially symmetric radial distortion model (cf. Sec-
tion 3.3.2) [491]. The approach is similar in spirit to Hartley and Kang's
although the method for finding the distortion center is less elegant.
The basic versions of these approaches work for central cameras; an
extension for non-central ones was also proposed.
### 5.1.3 One-Dimensional Grids: Sticks or Circular Grids Positioned in Radial Planes

Adorni et al. proposed a calibration method for the task of back-projecting an omnidirectional image onto a fronto-parallel ground plane [3]. They made the assumption of a radially symmetric camera and acquired an image of a linear calibration object lying on the ground plane, carefully positioned such that the camera's optical axis “pierces” the object at a known position. For each marker on the calibration object, the distance to the optical axis as well as the distance of the image point from the assumed distortion center is recorded in a look-up table. By interpolating the information contained in this look-up table and using the assumption of radial symmetry, each image point can be mapped to the point on the ground plane that it sees, and vice versa, i.e., back-projecting the image to the ground plane is possible. A similar approach was proposed earlier by Southwell et al. which instead of markers on the calibration object used one with a fractal Grey code pattern printed on it [455].

As for classical cameras, i.e., non-omnidirectional ones, using sticks with markers is probably the *de facto* standard approach for calibrating commercial multi-camera systems for motion capture, although many alternative approaches exist, e.g., [33, 480]. How to calibrate a single camera using a 1D object, was shown by Zhang [568].

### 5.1.4 Calibration Grids at Infinity — Collimators, Stellar Calibration, Vanishing Points

A classical photogrammetric calibration technique exploits arrays of collimators or devices with a movable collimator [95, 202, 268, 492]. The incidence angle of each collimator with the lens to be calibrated is measured, e.g., using theodolites or a goniometric apparatus. One can interpret such an array as a calibration grid lying at infinity: instead of providing matches between image positions and 3D points, they directly provide directions of camera rays for image positions. If a planar array is used (collimator beams are coplanar) and positioned such that that plane contains the distortion center, one can directly sample the radial distortion function from the observations, or fit any radial
distortion model to them. In the case of a non-planar array, tangen-
tial distortions may be measured as well. More generally speaking, one
may fit any global or local central camera model to data obtained from
collimators.

Another classical method termed stellar calibration consisted in
exploiting the well-known angular position of stars to calibrate cameras
(care had to be taken though to take into account, e.g., atmospheric
refractions) [151]. This may again be seen as using a calibration grid
lying at infinity.

Finally, we may mention calibration techniques exploiting vanish-
ing points arising from scene structure, e.g., associated with parallel
edges of buildings. Knowing the angles between the associated direc-
tions in 3D gives essentially the same input as collimators and images of
stars, although usually with (much) lower accuracy. Also, the number
of vanishing points in typical scenes is low, often not higher than three.
Thus, this kind of observation could typically not be used to measure
non-perspective distortions. However, from three vanishing points with
known angles, up to three intrinsic parameters of the pinhole model can
be estimated. Methods doing so are for example those by Gracie [185],
Caprile and Torre [81], and Echigo [126].

Hughes et al. showed that vanishing points arising from sets of paral-
lel lines may be used to calibrate the distortion center of radially sym-
metric distortion models [241]. They did this for the division model
(which was used as an approximation to the equiangular model), by
showing the following properties. Lines are imaged to circles and a set
of parallel lines are imaged to circles which intersect in two vanishing
points; these correspond to the two mathematical images under the
division model, of the point at infinity associated with the set of lines.
The line joining the two vanishing points also contains the center of
radial distortion. Thus, when observing at least two different sets of
parallel lines, the distortion center can be computed in a straightfor-
ward manner. It seems that this approach may be used for any radially
symmetric distortion model. In [240], Hughes et al. extended the pre-
vious approach by estimating another intrinsic parameter besides the
distortion center, namely the proportionality factor *k* of the equiangu-
lar camera model (cf. Section 3.1.7).
**5.2 Using Images of Individual Geometric Primitives**

By *individual* primitives, we mean that their relative position is not
known.

**5.2.1 Lines — The Plumb-Line Approach**

Brown introduced the so-called plumb-line approach which allows to calibrate non-perspective distortions from images of straight lines [64]. Since then, many such approaches have been proposed for different camera models, see below. Line-based calibration approaches are attractive since they allow to separate perspective from non-perspective parts of the imaging model and since their algebraic complexity is typically lower than that of full calibration formulations, allowing for easier closed-form type solutions. Further, since lines are omnipresent in urban or interior scenes or in man-made objects, they allow for a calibration in many applications where calibration grids cannot be employed.

**Classical polynomial distortion model.** Kang proposed a plumb-line method where snakes that are constrained by the radial distortion model are used to fit to images of straight lines, thereby estimating the distortion coefficients [263].

Alvarez et al. [14] used a classical distortion model and a plumb-line
type method, based on observations of sets of collinear points. They
defined a cost function which measures how close undistorted points
are from being collinear. That measure is not based on the Euclidean
distance between points and lines, i.e., is an algebraic distance. How-
ever, Alvarez et al. showed that it is possible to find the distortion
coefficients that are globally optimal under that measure, using a stan-
dard resultant-based technique. This is possible for up to two distortion
coefficients but hints on how to possibly handle more than two are also
given in [14].

Many other plumb-line approaches were proposed for the classical as
well as other distortion models, e.g., [127, 240, 241, 272, 337, 408, 427,
485, 503, 510, 520, 521, 553, 575]. Ricolfe-Viala and Sánchez-Salmerón
combined a plumb-line constraint with a constraint expressing the
invariance of the cross-ratio of collinear points, to calibrate a classi-
cal distortion model from images of planar grids [428].

**Central panoramas.** Smadja et al. proposed a plumb-line calibra-
tion method for circular central panoramas, acquired by a 1D camera
rotating about its optical center [452].

**Catadioptric cameras.** As for catadioptric cameras, Geyer and Daniilidis introduced a plumb-line method [164, 165, 167]. Based on their unified model (see Section 3.1.5), they showed that from two respectively three images of lines, one may fully calibrate a central hyper-catadioptric respectively para-catadioptric camera. They gave an algorithm for the para-catadioptric case; Barreto and Araújo developed an algorithm for hyper-catadioptric cameras [28, 31] as well as an improved algorithm for para-catadioptric ones that is able to fully compute the mirror shape [30, 32]. Ying and Hu, Vandeportaele et al., and Wu et al. gave similar algorithms [518, 537, 551]. Previously, Geyer and Daniilidis also proposed a method for calibrating para-catadioptric cameras from images of sets of parallel lines [163].

It is noteworthy that central catadioptric cameras can be fully cali-
brated from just images of lines, contrary to perspective cameras, with
or without the classical radial or tangential distortion models. In the
latter case, images of straight lines only allow to calibrate the cam-
era up to an arbitrary perspective projection, whereas for catadioptric
cameras, a full calibration is possible. One explanation for this is that
catadioptric cameras cannot be modeled by a perspective projection fol-
lowed by a distortion in the image plane. Rather, the distortion already
happens in the first step of the projection (in the unified model of Geyer
and Daniilidis, when mapping 3D points onto a sphere). Every camera
model for which this can be said may be fully calibrated from images
of lines even if their relative position is unknown.

**Fish-eye models.** Devernay and Faugeras presented a plumb-line
method in [113] and applied it to different distortion models: the clas-
sical backward model used by Brown [64], but also the models proposed
by Basu and Licardie and the field-of-view model (see Section 3.1.7).
**Polynomial and rational polynomial models.** Claus and Fitzgib- bon proposed two plumb-line methods for the bi-quadratic rational function camera model (see Section 3.1.8) [97]. They are both based on the fact that with that model, lines are imaged as conics. The first method starts with estimating conics from extracted line images and recovers distortion parameters via the factorization of a measure- ment matrix consisting of the matrices of all image conics. The second method is of the bundle adjustment type; it estimates the distor- tion model’s coefficients using as cost function the Sampson distance between image points and line images (conics). Barreto as well as Strand and Hayman proposed similar plumb-line methods for the 1-parameter division model, for which line images are circles [26, 466].

**Discrete camera models.** Tardif et al. showed how to calibrate a general radially symmetric radial distortion model with a plumb-line approach [489]. Their method is applicable to both, a parametric radial distortion curve and a discrete sampling of it.

**General remarks.** Whenever the back-projection operation can be written using polynomials, it is easy to set up “closed-form” solutions for plumb-line calibration. Consider the case of back-projection matri- ces for central cameras, of size 3 × n, for any n. As we have seen in Section 4.3, the image of a line L can be written as:

$$ \mathbf{c}^{\mathrm{T}} \sim \mathbf{I}^{\mathrm{T}}\mathbf{B}^{i}, $$

where l is the line at infinity of the plane spanned by the optical center and the line L. If we have m line images, we can set up a system like:

$$ \begin{pmatrix} \mathbf{c}_1^T \\ \vdots \\ \mathbf{c}_m^T \end{pmatrix} \sim \begin{pmatrix} \mathbf{l}_1^T \\ \vdots \\ \mathbf{l}_m^T \end{pmatrix} \mathbf{B}^i. $$

It is easy to see that the back-projection matrix can be recovered by factorizating the matrix on the left, like in factorization-based methods for structure-from-motion [508]. It can only be determined up to the pre-multiplication with an arbitrary 3 × 3 matrix, which
corresponds to a projective transformation of the directions of back-
projected points — the perspective part of the imaging process. In
other words, the non-perspective distortions are fully recovered. Once
this is done, one may apply any calibration or self-calibration method
designed for perspective cameras, to make the calibration complete.

This approach can work with a minimum of three line images,
although this is not recommended in practice; in that case, the matrix
on the left of the above equation can directly be adopted as solution
for B'. If prior knowledge on the camera is available, i.e., if constraints
on B' are available, fewer line images may be sufficient in theory.

One may apply the same approach to non-central cameras, with
more general back-projection matrices, e.g., of size 6 × n. In that case
however, it must be taken into account that often, they will be of rank
lower than 6, a fact that has to be taken into consideration when esti-
mating them.

Finally, as always, the best approach for calibration is to optimize
a meaningful geometrical/statistical error criterion, possibly after
initializing with any of the above approaches, which usually comes
down to a non-linear bundle adjustment like optimization process.
For example, one may simultaneously estimate the coefficients of
undistorted lines and of camera model parameters where the cost
function is the sum of Euclidean distances between undistorted lines
and points. It will in general be better even to minimize distances in
the original, distorted image, i.e., distances between original image
points and line images, where the latter are parameterized by camera
model parameters and parameters for line images (two for central
cameras, four for non-central ones).

5.2.2 Spheres

Algorithms using images of spheres for the calibration of perspective
cameras were proposed for example by Penna [398], Daucher et al. [104],
Teramoto and Xu [498], and Agrawal and Davis [6], to name a few.

Stevenson and Fleck used images of spheres to calibrate a local
camera model, see Section 3.2.3 [463].

Ying and Hu showed how to use sphere images for calibrating central
catadioptric cameras [551]. In these cameras, spheres are mapped to
conics, like in perspective images (to be precise, the mathematical
image of a sphere is the union of two conics, but in true cameras only
one of them is usually visible). Ying and Hu showed that from three
or more extracted conics, the catadioptric camera can be calibrated.
Ying and Zha analyzed the relationships between sphere images and
the image of the absolute conic and proposed a linear calibration algo-
rithm based on these [554, 555, 556].

5.2.3 Circles

Herbert proposed a calibration procedure where instead of point
matches or images of lines, projections of circles painted on the inside
of a hemisphere are used [219]. Coefficients of his distortion models
(see Section 3.1.7) are then estimated by measuring and comparing the
imaged areas of such circles.

Many algorithms using circles for calibrating pinhole cameras have
been proposed in the literature, see [91, 199, 257, 476] to name just a
few.

5.3 Self-calibration

Self-calibration has attracted great interest in the computer vision com-
munity in the last close to 20 years. The possibility of camera self-
calibration was already well known in photogrammetry, although it
was usually identified with the non-linear simultaneous optimization
of intrinsic and extrinsic camera parameters and scene structure (self-
calibrating bundle adjustment), whereas computer vision researchers
aimed at developing initialization approaches.

Most computer vision self-calibration works concern pinhole
cameras; a good general reference is the book by Hartley and Zisser-
man [213]. In the following, we describe approaches for non-perspective
cameras.

5.3.1 Self-calibration from Image Matches

**General observations.** As explained in Section 4.3, there is a strong  
link between plumb-line calibration (calibration from line images) and  
**conics**
self-calibration. From image matches, epipolar geometry can be recovered; epipolar curves can be generated and since they are images of straight lines (camera rays) they can be used as input to any plumbline approach. Instead of this, mainly conceptual, idea of explicitly generating line images from a computed epipolar geometry, one may directly use fundamental matrices for self-calibration.

In the following, we only consider the self-calibration of the nonperspective part of a central camera model. By this we mean that it suffices to recover its 3 × n back-projection matrix up to the pre-multiplication by any (invertible) 3 × 3 matrix M. The effect of M is a projective transformation of the directions of back-projected camera rays, i.e., a perspective part of the camera model.

In the following, we consider the case of a fundamental matrix between two images taken by the same central camera. Let B be its 3 × n back-projection matrix (we omit the superscript i here for ease of notation). The fundamental matrix is then given by (cf. Equation (4.3)):

$$F \sim B^T[t]_x R B.$$

Consider rows of F; they are linear combinations of the rows of B, with coefficients of these combinations given by the matrix $(B^T[t]_x R)$. Similarly, the columns of F are linear combinations of the columns of $B^T$, i.e., of the rows of B. This suggests the idea of determining the rows of B by computing a basis of the space of row and column vectors of F. To do so, let us now look at the singular value decomposition (SVD) of F:

$$F \sim U \Sigma V^T,$$

where all matrices are of size n × n. Since F is of rank 2 in general, the SVD can be reduced by only considering the first two columns of U, the first two rows of $V^T$ and the two non-zero singular values in Σ:

$$F \sim \bar{U}_{n \times 2} \bar{\Sigma}_{2 \times 2} \bar{V}_{2 \times n}^T.$$

The columns of $\bar{U}$ as well as the rows of $\bar{V}^T$ are linearly dependent on the rows of B. Let us stack them together in a 4 × n matrix and
compute its SVD:

$$ \begin{pmatrix} \bar{U}^T \\ \bar{V}^T \end{pmatrix}_{4 \times n} = W_{4 \times n} \Gamma_{n \times n} X_{n \times n}^T. $$

In the absence of noise, the above matrix is of rank 3 in general, since all its rows are linearly dependent on the three rows of B. The three rows of X^T corresponding to the non-zero singular values in Γ can now be directly adopted as the rows of the back-projection matrix B. It is clear that the rows of B are only defined up to linear combination of the rows of X^T, but this does not matter: as stated above, B needs only be recovered up to a projective transformation M, and this means nothing else than a linear combination of its rows.

A more compact alternative procedure would proceed as follows. Stack F on top of F^T and compute the SVD of this $2n \times n$ matrix:

$$ \begin{pmatrix} F \\ F^T \end{pmatrix} = U \Sigma V^T. $$

Using similar arguments as above, the matrix on the left is of rank 3 in general and the three rows of V^T corresponding to non-zero singular values in Σ can directly be chosen as rows of B, which concludes this self-calibration algorithm.

A similar procedure was proposed by Claus and Fitzgibbon for the case of 3 × 6 back-projection matrices [98]. The self-calibration approach for para-catadioptric cameras by Geyer and Daniilidis (cf. Section 5.3.1) follows the same principle [166, 170]. In their case, the back-projection matrix could be fully determined (up to a rotation instead of a projective transformation), due to using constraints on the coefficients of B. More generally speaking, there are two ways of obtaining more than the self-calibration of “just” the non-perspective part of B. (i) Use constraints on the entries of B. (ii) In any case, once B is estimated up to a projective transformation, we are in the situation of an uncalibrated perspective camera. From the fundamental matrix between two perspective images, one may estimate up to two intrinsic parameters [206, 474]; usually one just computes the focal length. Hence, in many cases it should be possible to obtain a full self-calibration of non-perspective cameras from a single fundamental matrix.
From the above, it is straightforward to extend self-calibration to
the use of multiple fundamental matrices for a better stability and/or
to the self-calibration of two or more cameras with different parame-
ters/models. Without giving a formal proof, it should be possible for
example, to self-calibrate three entirely different cameras (possibly with
different sizes of back-projection matrices), from the three pairwise fun-
damental matrices. This possibility was demonstrated by Geyer and
Daniilidis for para-catadioptric cameras [166, 170].

In the following, we review a few self-calibration approaches pro-
posed in the literature for different types of non-perspective camera
models.

**Classical distortion model.** Zhang proposed an approach for the simultaneous estimation of one radial distortion coefficient and the perspective fundamental matrix, by non-linear optimization of the epipolar constraint [566]. Based on his experimental results, Zhang concluded that the distortion coefficients can only be estimated reliably if matching points can be extracted accurately *and* if the actual distortion is significant (cf. the discussion in Section 3.6). In other cases, the estimation is ill-conditioned or in other words, taking into account distortion coefficients besides the fundamental matrix constitutes an overparameterization. One may further add that it seems to be required that image matches be well distributed across the image planes, as is always required when estimating radial distortion, which is most observable toward the borders of images.

In [461], Stein proposed a virtually identical method, as well as an
extension to three views. Here, trifocal transfer errors are used to define
a cost function which is optimized simultaneously over the distortion
and trifocal tensor coefficients.

**Non-central panoramas.** Huang et al. proposed a self-calibration method for non-central panoramic images acquired by rotating 1D cameras [239]. In their approach, it is required that images be obtained by rotating about the same axis.

**Catadioptric cameras.** Gluckman and Nayar showed how to determine the focal length of the camera in a single-lens catadioptric stereo
system using one camera and two planar mirrors [174, 175]. Since this
setup is identical to a system of two perspective cameras, perspec-
tive self-calibration approaches can be readily applied. Gluckman and
Nayar used point matches between the two views and the fundamental
matrix estimated from them. Their reported results were not perfect.
We believe that this is because of a generic degeneracy: whenever the
two mirrors are arranged in a rotationally symmetric position with
respect to the camera's optical axis, the two virtual cameras (the real
camera reflected in the two mirrors) are in a degenerate relative pose
for focal length computation [474]. In situations close to this, results
may be expected to be inaccurate.

The first self-calibration approach for non-perspective catadioptric
systems is probably due to Kang, who considered the central para-
catadioptric case [262]. Based on an expression of the epipolar geom-
etry as a function of the relative motion between two views and the
relevant intrinsic parameters, he proposed a cost function consisting of
the sum of distances between image points and epipolar circles. This
cost function was minimized non-linearly.

Geyer and Daniilidis showed how to self-calibrate para-catadioptric
cameras from point matches, via the estimation of the 4 × 4 funda-
mental matrix [166, 170]. Their key observation was that the null-
space of the fundamental matrix contains the vector representation
of the image of the absolute conic (a circle in their case). They showed
that the fundamental matrix is of rank 2; if both images are taken
by the same camera, then the left and right null-spaces may be inter-
sected to find the image of the absolute conic and thus to calibrate
the camera. Geyer and Daniilidis also showed that if the images are
taken with different para-catadioptric cameras, then their intrinsic
parameters can be found from the fundamental matrices between three
images. These observations can be generalized, cf. the above paragraph
“General observations”.

Sturm showed that a similar concept can be used for the self-
calibration of para-catadioptric cameras from homographies associated
with scene planes [471].

In [344, 347, 349], Mičušik and Pajdla proposed an approach for
the estimation of the epipolar geometry of non-central catadioptric
cameras, demonstrated for the examples of a spherical mirror (always non-central) and misaligned para- and hyper-catadioptric systems. The approach proceeds in several steps. First, an appropriate central approximation to the full non-central back-projection is derived. This approximate model is then linearized such that its coefficient(s) can be estimated simultaneously with the fundamental matrix, by solving a quadratic eigenvalue problem similarly as in Fitzgibbon's approach for the division model (see below). This allows to initialize and then optimize the complete, non-central model.

**Polynomial and rational polynomial models.** Fitzgibbon showed how to simultaneously estimate radial distortion and perspective multi-view relations such as the fundamental matrix, plane homographies, or trifocal tensors, by solving quadratic or cubic eigenvalue problems [145]. The approach is based on the division model (see Section 3.1.8). This was extended by Mičušik and Pajdla [344, 346, 349] (see previous paragraph), Barreto and Daniilidis [33, 34], Claus and Fitzgibbon [98], and Steele and Jaynes [458].

Claus and Fitzgibbon showed how to estimate the coefficients of the full bi-quadratic rational function model for central cameras (i.e., 3 × 6 back-projection matrices) from point matches between two views [98]. They also analyzed the nature of epipolar curves (conics) and showed how to calibrate the model from one image of a planar calibration grid.

Mičušik and Pajdla used a division-type model as an extension to the equiangular fisheye model (see Section 3.1.7) and showed, analogously to Fitzgibbon's approach, how to simultaneously estimate its two parameters and the fundamental matrix [344, 346, 349]. If both parameters of their model are used, 15 point matches are required; for a one-parameter version 9 matches are sufficient. They used an analog approach to self-calibrate a one-parameter model of para-catadioptric cameras, leading to a quartic instead of a quadratic eigenvalue problem [344, 348, 349].

Barreto and Daniilidis showed that even if two images with different distortion coefficients for the division model are considered, they are subject to a 4 × 4 fundamental matrix [34]. They gave an algorithm for linearly estimating the distortion coefficients and the epipolar
geometry, from a minimum of 15 point matches. The same factoriza-
tion approach is used in [33] to calibrate the nodes of a multi-camera
network, including radial distortion, intrinsic and extrinsic parameters.

Li and Hartley proposed a different method for estimating distortion coefficients of the division model from point matches in two views [311, 312]. They decoupled the estimation of distortion coefficients from that of the fundamental matrix and showed how to estimate one or two coefficients of the division model from 9 point matches.

Kúkelová and Pajdla proposed a minimal method for the self-calibration problem with one distortion coefficient, using 8 point matches [292]. In [72, 291, 293], Kúkelová, Byröd et al. gave minimal methods for the following two problems. (i) Estimating the fundamental matrix together with two different coefficients for the division model, from 9 point matches (a non-minimal linear solution for this problem was first given by Barreto and Daniilidis [34], see above). (ii) Estimating the epipolar geometry and one distortion coefficient (identical for both images) in the case of partially calibrated images (known principal point and square pixels); a minimal method for estimating the distortion coefficient and the essential matrix (as opposed to the fundamental matrix) was given.

Steele and Jaynes extended Fitzgibbon's approach in two ways [458]. First, they solved a more general quadratic eigenvalue problem, allowing to obtain less ambiguous and more accurate results in the case of more than the minimum of 9 point matches. Second, their approach may be used to simultaneously estimate a global distortion coefficient and fundamental matrices for multiple image pairs, thus increasing the accuracy of each of these estimates.

Kannala et al. proposed a bundle adjustment type approach for the
self-calibration of a unified central catadioptric model (Section 3.1.5)
and the polynomial model of Equation (3.7), from matches between
two images [267].

**Discrete camera models.** Thirthala and Pollefeys showed how to self-calibrate the 1D radial camera model (see Section 3.1.9) from four images, based on estimating the associated quadrifocal tensor [500]. Their approach is stratified, starting with a projective reconstruction
and converting it to a Euclidean one. At that stage, the radial dis-
tortions of the camera are not yet calibrated, just the mapping from
radial planes to radial lines. To get a full calibration, Thirthala and
Pollefeys used the assumption of radial symmetry. They either fit a
division model to the reconstruction [501] or a discrete camera model.

In addition to their approaches for grid-based calibration of general
radial distortion models (see Section 5.1.2), Hartley and Kang also
showed that self-calibration is possible [209]. They observed however
that results are too unstable in the presence of even small amounts of
noise.

Tardif et al. proposed an approach for the self-calibration of a gen-
eral radially symmetric radial distortion model (cf. Section 3.3.2) from
images of a planar scene with unknown structure [489]. A more robust
extension which also works for images with different radial distortions
was presented in [490]. A factorization-based method for the same prob-
lem was given by Ramalingam et al. [417].

**5.3.2 Special or Known Motions**

Like for perspective cameras, using special types of camera motion
or actually knowing the camera motion usually allows to simplify the
self-calibration problem. This enables less complex algebraic problem
formulations, allows to work with fewer images, and often gives more
accurate results. As for perspective cameras [207], the most used motion
is pure rotation about the optical center; an approximate rotation is suf-
ficient if the scene is sufficiently far away. In the following, we describe
several approaches, first ones based on special types of motion (but
with unknown motion parameters), then ones using knowledge of the
motion parameters.

**Classical distortion model.** Stein proposed a method based on pure rotational camera motion [460]. Matching (undistorted) image points are related by a special homography. Distortion coefficients and that homography are estimated simultaneously, by non-linear optimization, using a reprojection error type cost function.

Sawhney and Kumar showed the importance of estimating
non-perspective distortion for generating mosaics based on images
acquired with a rotating camera [435]. They proposed a direct method (i.e., based on image intensities rather than feature matches) for simultaneously estimating distortions and image alignment transformations. In their work, they only considered a single term of the classical radial distortion model, but it is obvious that other distortion models may be estimated from images taken under pure rotation.

**Fish-eye models.** Xiong and Turkowski performed the self-calibration of a fisheye camera from pure rotations [539]. The camera model they used is the extension of the equiangular model described in Section 3.1.7. The calibration is performed by non-linear minimization, where the distortion center/principal point is initialized as the center of the image's bounding ellipse.

**Discrete camera models.** Thirthala and Pollefeys showed how to self-calibrate the 1D radial camera model (see Section 3.1.9) from three images taken under pure rotational motion [500, 501]. To be precise, in the non-central version of the 1D radial model, pure rotation would rather correspond to camera poses such that all optical axes intersect in a single point. A full self-calibration algorithm was given for central cameras.

Tardif et al. showed how to self-calibrate a general radially symmetric radial distortion model using either pure rotational or pure translational camera motion [489]. They used both a parametric radial distortion curve and a discrete sampling thereof.

The probably most general self-calibration problem considered to date, besides the work of Grossmann et al. (see Section 5.3.3), concerns the ray-based self-calibration of a general central camera. In the following works, the only assumption made is that of continuity — neighboring image points have neighboring camera rays.

Ramalingam et al. showed that from pure translational motions, the self-calibration of camera rays is possible up to a projective transformation of their directions [419]. They stated that a complete self-calibration is possible from two rotational motions and gave an algorithm for full self-calibration from one translational and two rotational motions. In [421], they proposed an algorithm that only requires two rotational motions.
Nistér et al. and Grossmann et al. [193] gave several similar results, mainly for the case of infinitesimal rotations or translations [380, 193]. Their main result is an algorithm for self-calibration from rotational optical flow. They issued the conjecture that two infinitesimal rotations about different axes are sufficient for a complete self-calibration. This was theoretically proven by Espuny and Burgos Gil who also gave a practical algorithm [130, 131].

**Known motions.** Stevenson and Fleck developed an approach using a robot arm and a light source to calibrate a camera [462]. The camera is mounted on the robot arm which then rotates the camera by known angles (the rotation should be about the camera's optical center, although this is not further discussed in [462]). The image of the fixed light source is extracted in each frame. The approach calibrates a radially symmetric central model, as well as optical parameters such as light fall-off across the image plane. The gist of the proposed approach is as follows. An initial estimate of the distortion center is assumed; the camera is rotated such that the light source is seen at the distortion center. Then, the camera is successively rotated in one direction by some small angle increment. The observed image positions of the light source, together with the known rotation angles, directly allow to construct a discrete sampling of the distortion function. Stevenson and Fleck also described how to correct the initial estimate of the distortion center.

Similar active approaches were also proposed by Du and Brady [122] and Kojima et al. [282] and seem to be in use for the calibration of cameras for space and planetary exploration, see e.g., [334].

Swaminathan et al. proposed an approach for calibrating non-central catadioptric cameras, using image matches and knowledge of the camera motion [484]. The approach is based on a parametric model for the caustic of catadioptric cameras; its coefficients are non-linearly optimized in a bundle adjustment like manner.

### 5.3.3 Image Statistics

Farid and Popescu showed in [135] that distortion coefficients, as well as coefficients of non-linear intensity transformations, may be estimated
from single images without the use of calibration objects. This is based
on a frequency analysis of the image and on the assumption that non-
linear transformations (geometric or intensity distortions) introduce
specific higher-order correlations in the frequency domain. In some
sense, this approach is an example of using natural image statistics, here
for camera calibration. A similar approach was developed by Yu [561].

Grossmann et al. showed that a highly discrete camera model,
consisting of “isolated” camera rays (see Section 3.3.3), can be self-
calibrated just by taking images of unknown scenes, based on simple
assumptions about their appearance [192, 194]. Roughly speaking, it
is assumed that the smaller the angle between two camera rays, the
more likely it is that the associated pixels produce similar greylevels;
this is in some sense, an assumption on the statistics of natural images.
When acquiring a sufficiently large set of images, observed differences
between greylevels of different pixels, allow to infer the topology of the
camera rays. Based on a simple off-line training, one may even recover
the angles between rays and finally, their directions in a common coor-
dinate system.

**5.4 Special Approaches Dedicated to Catadioptric Systems**

In this section we only cover methods that were not explained in any
of the above sections. When using catadioptric cameras, one rarely
does not know anything at all about the shape of the mirror, unless
it is taken “off-the-shelf” or if the producer of a catadioptric camera
does not disclose any information about the mirror’s shape. In case the
mirror shape is known, calibration can be considered as estimating the
relative pose between camera and mirror, and possibly the camera’s
intrinsic parameters. Otherwise, several possibilities exist, as explained
below.

**Known mirror shape.** One of the two (self-)calibration methods proposed in [262] by Kang is based on the extraction of the mirror’s upper circular rim in the image. From this, some or all intrinsic parameters can be computed. This idea was used in several other works, for example [110].
Strelow et al. proposed a calibration method using 3D calibration grids [467]. They assumed that the perspective camera looking at the mirror is already calibrated. The calibration of the catadioptric sensor then comes down to estimating relative mirror-to-camera pose. Strelow et al. estimated this together with the pose of the calibration grid, by non-linearly optimizing the reprojection error of calibration points. Lhuillier showed how to optimize the mirror-to-camera pose and the 3D position of points observed and matches in several images, via bundle adjustment [308].

Aliaga used a parabolic mirror, looked at by a perspective camera [12]. Hence, the catadioptric system is non-central. A calibration procedure was proposed that estimates the focal length of the perspective camera and its position relative to the mirror (the orientation seems to be assumed to be perfect, i.e., the optical axis to be parallel to the mirror axis), as well as a radial distortion coefficient.

Similar but more restrictive approaches exist that require knowledge of the mirror shape and the camera's intrinsic parameters, i.e., calibration comes down to estimating the relative mirror-to-camera pose. Mashita et al. [332] supposed that the perspective camera is calibrated and that the mirror's circular upper boundary can be extracted in the image. Then, classical algorithms for estimating the pose of a circle in 3D can be applied, e.g., [114], after which the pose of the entire mirror, relative to the camera, is known. This effectively gives the calibration of the entire catadioptric system. This is similar to the approach by Kang described above, which was developed for central para-catadioptric cameras, whereas the one by Mashita et al. is for perspective cameras and central as well as non-central setups.

Another such method was proposed by Fabrizio et al. where besides the upper boundary of the mirror, a second circular section of it must be observable in the image, e.g., the footprint of a black needle glued on the mirror of some prototypes of catadioptric systems [133]. From the images of these two circles, the mirror pose is computed along with some intrinsic camera parameters.

**Unknown mirror shape.** Several possibilities exist to perform calibration in this case, some of which are explained in the following.
If it is known that the catadioptric camera is one of the central
types (cf. Section 2.3.1), then one may for example use any of the
unified models (cf. Section 3.1.5) to calibrate it.

A more general approach was proposed by Caglioti et al. for the case
where all that is assumed is that the mirror is a surface of revolution
[78]. The perspective camera looking at the mirror can be positioned
arbitrarily and the mirror shape can be arbitrary (besides being a sur-
face of revolution), hence the system is non-central in general. The
proposed calibration approach requires to extract the silhouette of the
mirror in the image as well as the image of at least one straight line in
the scene. This input allows to calibrate intrinsic camera parameters
(focal length and principal point), the shape of the mirror (its profile)
and the relative pose of mirror and camera.

It is of course possible to cut the calibration process in two steps,
where in a first step one reconstructs the mirror shape using any of
the many methods for specular surface reconstruction available in the
literature, most of which are based on analyzing images of patterns
reflected in the mirror (a representative list of references is out of scope
for this manuscript). An alternative to these classical approaches is to
use polarization imaging, as proposed for example by Morel et al. [352].

Naturally, another possibility is to calibrate the catadioptric system
using any of the global, local, or discrete models described in the pre-
vious section. If the mirror is of arbitrary shape, only local or discrete
models might work.
The goal of this section is to briefly describe the main building blocks of structure-from-motion in a simple way, that is independent of the camera model used. This is to highlight the underlying (simple) principles, allowing to establish links between seemingly different approaches that abound in the literature and to easily adapt these principles to new camera models.

The first three sections are dedicated to the main building blocks for calibrated cameras: pose and motion estimation, and triangulation of points. These will be explained for central and non-central cameras. The fourth section aims at highlighting the central role of bundle adjustment in all of structure-from-motion and calibration. The fifth section lists a few works on 3D scene modeling from omnidirectional cameras, from single or multiple views. In the last section, the related issues of distortion correction and stereo rectification are discussed; they are not really structure-from-motion topics, but necessary for 3D scene modeling.

Overall, we believe that the most important ingredients of structure-
from-motion systems are: algorithms working with the minimum
required amount of data, to be embedded in robust estimation schemes
such as RANSAC, and a (robust) bundle adjustment, allowing to get
optimal results for all of the individual problems. Other important con-
tributions, not discussed further in this section, are methods determin-
ing globally optimal solutions of structure-from-motion problems, see
e.g., [260] and references therein. Besides the minimal methods, other
algorithms that give suboptimal results may be useful, e.g., to get a
better initial estimate from the set of inliers of a model determined
using a minimal method. Finally, this section only covers some main
structure-from-motion problems; many special cases exist, using lines
or other primitives than the points considered in the following, con-
cerning dynamic scenes or objects, etc.

## 6.1 Pose Estimation

Computing the pose relative to a calibrated perspective camera, of an
object with known structure, is one of the most classical problems in
photogrammetry (where it is called “resection”) and computer vision,
see, e.g., a review paper by Haralick et al. [205]. Pose, i.e., object posi-
tion and orientation, is usually computed using point or line correspon-
dences between the object and the image, although other possibilities
exist, for example computing the pose of a spherical object from its
silhouette in the image [114]. The minimum number of required point
correspondences is three, in which case there may be up to four pos-
sible solutions [230]. Any of the numerous algorithms that were pro-
posed for the perspective camera model can be easily adapted to any
other central camera model, since instead of working with the original
2D image coordinates, each algorithm uses, explicitly or implicitly, the
directions of the back-projected camera rays. Hence, these algorithms
can be used for any central camera as soon as it is calibrated. There
exist different formulations for the pose estimation problem. Here, we
use one that is directly suited for central as well as non-central cam-
eras. Minimal solutions for the non-central pose estimation problem
were proposed independently by Chen and Chang [88], Nistér [378],
Nistér and Stewénius [379], and Ramalingam and Sturm [414, 415].
See also papers by Fabrizio and Devars [132] and Schweighofer and
Pinz [442] about the n-point pose problem for central and non-central
cameras.
**General problem formulation.** The problem can be formulated as follows. Instead of explicitly estimating the rotation and translation of the object model, we estimate the depth of object points relative to the camera, i.e., their position along the known camera rays, associated with the matched image points. Given the position of 3 points, the rotation and translation for the entire object can then be computed easily (via absolute pose estimation, see for example [233]). We use the finite and infinite parts of back-projection, cf. the introduction of Section 3: a camera ray is defined by a finite point $\mathbf{B}^f$ (for central cameras, the optical center) and the direction $\mathbf{B}^i$. A point on the ray is then parameterized by a scalar $\lambda$:

$$ \mathbf{Q} \sim \mathbf{B}^f + \lambda \begin{pmatrix} \mathbf{B}^i \\ 0 \end{pmatrix}. $$

With three object points $\mathbf{Q}_j$, parameterized by $\lambda_j, j = 1 \cdots 3$, we can set up three constraint equations: the Euclidean distances between these points must be identical to the prior known values. The squared distance between two points $\mathbf{Q}_j$ and $\mathbf{Q}_k$ is quadratic in $\lambda_j$ and $\lambda_k$. From the three pairs of points, we thus get a total of three quadratic equations in three unknowns. One may use any appropriate numerical method for solving such a system; it is simple for example to convert the equation system into a degree-8 polynomial in one of the unknowns and then to use any numerical root finding method to get the up to eight real solutions. This should naturally be embedded in a robust estimation scheme, such as RANSAC, where each of the potential solutions is scored against the other point matches.

**Central cameras.** The usual special case of central cameras allows for a slight simplification. As finite point $\mathbf{B}^f$, the origin may be adopted for all camera rays; our points can then be parameterized as:

$$ \mathbf{Q}_j \sim \begin{pmatrix} \lambda_j \mathbf{B}_j^i \\ 1 \end{pmatrix}. $$

If we swap the sign for all three $\lambda_j$, the distances between the $\mathbf{Q}_j$ are not affected. Hence, for each solution of our three constraint equations, there exists a mirror solution with all signs swapped. This allows to
reduce the complexity of the estimation problem: instead of solving the
equivalent of a degree-8 polynomial, a degree-4 polynomial is sufficient.

**Other pose problems.** Josephson and Byröd gave a minimal method for estimating camera pose, together with the focal length and one radial distortion coefficient (division model), from four matches [259].

A special case is that of planar pose, where the unknown object pose consists of a translation in one plane (usually, a horizontal ground plane) and a rotation about the normal of that plane. Minimal solutions requiring 2 point matches only exist [12, 393]. Gourichon et al. proposed a bio-inspired method for planar pose estimation [184].

**6.2 Motion Estimation**

**Generalities.** Motion or egomotion estimation, or relative orienta-
tion, is another classical problem. The goal is to estimate the motion
of a camera or the relative pose of calibrated cameras, from matches
between images but without using information on the scene structure.

The first truly minimal solution for central cameras was developed
by Nistér [377]: an approach for estimating all possible essential matri-
ces [324] from 5 point matches (up to 10 theoretical solutions).

As for non-central cameras, the concept of essential matrix was
introduced by Pless [402], cf. Section 4.1. It can be estimated by solving
a linear equation system, established from 17 or more point matches;
extracting the motion parameters from the essential matrix is then
relatively straightforward. Note that if the cameras are not “fully non-
central”, e.g., are of the axial type, then such a linear estimation pro-
cess may be underconstrained, as pointed out by Mouragnon et al.
[357]. Other degeneracies were studied by Kim and Kanade [279]. Kim
et al. proposed two motion estimation algorithms for the case where the
non-central system consists of two or more central cameras with non-
overlapping fields of view, i.e., where points are only tracked by indi-
vidual cameras but where no handover between cameras occurs [277].
Finally, a minimal solution for the general non-central case, from 6
point matches, was given by Stewénius et al. [464].
Contrary to motion estimation from central cameras, the scale of
the translation can also be estimated with non-central cameras. How-
ever, this is only accurate if the trade-off of several factors is favorable
(cf. the general discussion in Section 3.6): the cameras should be suf-
ficiently non-central, compared to the amount of noise, the distance of
the scene from the cameras, etc. The other way round: if a camera is
not sufficiently non-central, then it is advisable to model it as a central
system and use traditional motion estimation, cf. also a study by Kim
et al. [278].

In the following, we briefly cite a few works on motion estimation
for omnidirectional cameras.

**Works on motion estimation for omnidirectional cameras.**

Yagi et al. considered the motion estimation with a central hyper-
catadioptric camera [543]. The robot-mounted camera is supposed to
move on a horizontal ground plane; the topic of [543] is to estimate
small perturbances to perfect planar motion (so-called rolling), from
optical flow.

McMillan and Bishop estimated the motion between two calibrated
cylindrical images [338]. This is done by non-linear minimization of
the distances between back-projection rays associated with match-
ing image points. Kang and Szeliski [264] used the classical 8-point
method for linearly estimating the essential matrix in the same case.
Other approaches for motion estimation that are based on computing
the essential matrix, explicitly or implicitly from directions of back-
projected image points, are [86, 479]. Svoboda discussed the issue of
properly normalizing coordinates of image matches to ensure a good
conditioning [391, 479], along the lines of Hartley's approach [208].
Bazin et al. proposed a motion estimation method for catadioptric cam-
eras that decouples the estimation of rotation and translation param-
eters [45]. The method is designed for man-made environments and
requires the detection of sets of parallel lines.

Gluckmann and Nayar estimated motion from the optical flow [173].
In order to use a wide field of view and to use the same method for
motion estimation for different imaging geometries, they mapped the
flow field to a sphere centered in the optical center, using camera
calibration information. Then, extensions of classical algorithms for
motion estimation from optical flow, from planar to spherical viewing
surfaces, can be readily applied, independently of the actual camera's
imaging geometry. We also refer to the seminal work on determining
camera motion from the flow field on the viewing sphere, by Nelson
and Aloimonos [373], where they also clearly explained why wide fields
of view are superior to narrow ones for motion estimation. See also the
papers [141] by Fermüller and Aloimonos who studied the same issue
while drawing analogies with the evolution of biological eyes, [304] by
Lee et al., and [375] by Neumann et al. who proposed a mathematical
formulation for designing non-central cameras that are optimal for
motion estimation. Shakernia et al. proposed a similar approach to
that of Gluckman and Nayar, but instead of back-projecting optical
flow to a sphere, they used another curved surface which is implicitly
defined by the lifting operation carried out when back-projecting
image points [446].

In [468], Strelow and Singh described how to combine image point
matches in omnidirectional cameras with inertial sensor readings, for
structure and motion estimation. A first approach assumes a calibrated
camera. A second approach works for uncalibrated cameras — only the
assumption of a radial projection function is required (cf. Section 3.1.9).
This is highly related to work by Thirthala and Pollefeys, see Sec-
tions 3.1.9 and 5.3.1, and to the RAC (radial alignment constraint) by
Tsai [515].

Shakernia et al. formulated motion estimation under the assumption
of small displacements between images [447].

Lim et al. introduced a so-called antipodal-epipolar constraint that
allows to decouple the estimation of rotation and translation for cam-
eras with larger than hemispheric fields of view [318]. This result is
based on the notion of antipodal image points: points whose camera
rays are aligned but which correspond to opposite viewing directions.
If two antipodal points in one camera can both be matched in the other
camera, then a specific epipolar constraint becomes available. Lim et al.
showed how to independently estimate translation and/or rotation from
two or more pairs of such antipodal point matches. In [317], they showed
a similar decoupling approach whose inputs are optical flows measured
at antipodal point pairs.

**6.3 Triangulation**

**Points.** Reconstructing a 3D point from matches in two or more cal-
ibrated images (called intersection in photogrammetry) is of course an
essential module in structure-from-motion. Like for the other problems,
an optimal solution requires in general a bundle adjustment or a global
optimization approach. For the special cases of two or three perspec-
tive images, “direct” optimal solutions were proposed by Hartley and
Sturm [211] and Stewénius et al. respectively [465].

Suboptimal but generic solutions are to estimate the 3D point that
minimizes some distance measure relative to the camera rays associated
with the input image matches, for example the Euclidean distance [478]
or an angle-based measure, see Section 6.4.

**Lines.** The triangulation of lines or line segments is useful, especially in 3D modeling of man-made objects. Whereas for points, it does not matter if they are reconstructed from central or non-central cameras (a single ray is used per camera), there is a difference in the case of lines. As for central cameras, whatever type they are, a line image can be back-projected to an *interpretation plane* in 3D — the plane spanned by the optical center and the original 3D line. From two images, the line can thus be triangulated by computing the intersection of the two interpretation planes.

As for non-central cameras, they allow in general the triangulation
of 3D lines from single images. The reason is that the back-projection
is no longer a plane but some other surface and in general, the original
3D line is the only straight line on that surface. Another way of formu-
lating this is by considering image points lying on a line image, and the
associated camera rays. It is well known that for four lines in general
position, there exist two lines incident to all of them. If more than four
camera rays associated with a line image are considered, the original
3D line can be found as the only one incident to all of them, using
the approach of Teller and Hohmeyer [494]. Of course, if the camera
is of the axial type, all camera rays are incident with the camera axis; this can be taken care off by computing all candidate 3D lines (two in general) and by discarding the camera axis. There may be ambiguous solutions for the 3D line, depending on the camera used and the position of the original line.

Algorithms and in-depth theoretical studies for line triangulation from non-central images are given by Caglioti and Gasparini [75, 76, 77] as well as Lanman et al. [299]. Morita et al. described an early method for reconstructing 3D lines from two fisheye images [353]. Although not described as such, the method seems equivalent to intersecting interpretation planes. Like other researchers, for example Vasseur and Mouaddib [519], Morita et al. used the fact that 3D lines in spherical images, appear as great circles. Yamazawa et al. described a Hough-transform-based approach for detecting line segments in omnidirectional images and demonstrated the 3D reconstruction of lines from image triplets [545]. Another approach for 3D line (segment) triangulation for omnidirectional images is for example [242].

**6.4 Bundle Adjustment**

As mentioned earlier, bundle adjustment is imperative for obtaining an accurate solution of any structure-from-motion, calibration, or other similar problem, an exception being problems for which global optimizers exist. Bundle adjustment means in principle nothing else than a simultaneous non-linear optimization of all unknowns; in addition, it implies that a meaningful cost function is used, see below. Other important issues are the exploitation of the problem structure at hand to obtain efficient algorithms; this includes the well-known sparsity of the normal equations solved at each iteration of the usually employed non-linear least-squares methods but also the question of how to order data in the case of image sequences with multiple overlaps. More on bundle adjustment can be found in the overview paper by Triggs et al. [512].

Bundle adjustment is most often a non-real-time process. Recently, approaches that may be qualified as hybrids between bundle adjustment and simultaneous localization and mapping (SLAM) [107] were proposed, by Mouragnon et al. [356, 358] and Engels et al. [129]. SLAM,
mostly used in robotics, solves in principle exactly the same problem as
structure-from-motion: estimating camera motion and scene structure.
The main difference is the real-time requirement in robotics applica-
tions, which lead to sequential solutions for cameras and other sensors
(SLAM), whereas bundle adjustment and structure-from-motion sys-
tems were often developed without having to satisfy this requirement.
The hybrid approaches of Mouragnon et al. and Engels et al. allow for
real-time performance by following an overall sequential scheme, while
continuously performing a complete non-linear optimization on as long
subsequences as possible with the available computing power.

As mentioned above, an important aspect of bundle adjustment is
the choice of cost function. As generally agreed, the “best” cost function
is based on the reprojection error, i.e., the distance between measured
image points and those predicted by the unknowns (scene structure,
camera intrinsics or extrinsics). To be more precise, the cost function
should be defined based on a statistical model of the noise in the input
data, whenever this is available. If the input, as usual, are extracted
image points then one should use an uncertainty estimate of the inter-
est point extractor; under the default assumption of i.i.d. Gaussian
noise this automatically leads to the standard cost function defined as
the sum of squared distances between measured and predicted image
points. Ideally, actual uncertainty estimates of image point positions
should be used.

Minimizing reprojection errors requires carrying out forward projec-
tions of 3D points; as described elsewhere in this manuscript, for some
cameras back-projection is easier to formulate. Further, if already cali-
brated cameras are considered, then one may develop a generic bundle
adjustment working with camera rays instead of image points as basic
input. Such procedures were for example proposed by Ramalingam
et al. [414, 415], Schweighofer et al. [443], and Lhuillier [309, 310].

Ramalingam et al. used as cost function, the distance between
camera rays and 3D points, which is simple to use but subopti-
mal. Schweighofer et al. gave a provably convergent algorithm for
estimating camera motion and 3D structure, using the same cost
function [443]. As an alternative solution for non-central cameras,
Ramalingam et al. proposed to partition the set of camera rays into
clusters of approximately convergent rays, which are then treated as
individual pinhole cameras inside a classical reprojection error-based
bundle adjustment. Such an idea was also suggested though apparently
not implemented, by Martins et al. [331].

Lhuillier proposed and justified the use of an angle-based cost func-
tion [309, 310]. To be precise, he suggested to use the squared tangent
of the angle between the back-projected camera ray (computed from
estimated camera parameters) and the line spanned by the optical cen-
ter and the estimated 3D point. He also discussed how to minimize the
reprojection error for catadioptric cameras.

Kannala et al. used a different angle-based cost function for bundle
adjustment (sum of squared sines) [267], inspired by Oliensis [385].

**6.5 Three-Dimensional Scene Modeling**

In this section, we cite a few approaches for complete 3D scene modeling
from omnidirectional images. Most of them rely on a combination of
the above structure-from-motion techniques and binocular or multi-
view dense stereo methods.

Kato et al. used a shape-from-silhouette approach with a fisheye
camera to reconstruct a moving object, whose motion was supposed to
be determinable [270].

Kang and Szeliski used mosaics acquired by a calibrated perspec-
tive camera and expressed in cylindrical coordinates [264]. After motion
estimation and bundle adjustment based on interest points, they carried
out a dense multi-baseline stereo algorithm. A similar system, active-
vision based, was proposed by Ishiguro et al. [247]. Teller and his group
used georeferenced omnidirectional images for city modeling [493].
Bunschoten and Kröse [68, 69], Doubek and Svoboda [119], Mičušik
et al. [345], and Havlena et al. [214] presented other multi-baseline
stereo approaches for central omnidirectional images.

One of the most advanced approaches, demonstrated on long image
sequences, is that of Lhuillier [310].

There also exist approaches for 3D modeling from single omni-
directional images. Sturm proposed a method for para-catadioptric
cameras, exploiting user-provided geometric scene constraints such as
perpendicularity or parallelism of lines and coplanarity of points [470]. The method consists of two steps, first a calibration step, then the actual 3D modeling. It is important to underline that the second step is not specific to omnidirectional images, but could be applied to any calibrated central image: the available geometric constraints are applied on the back-projected camera rays and it does not matter what the underlying imaging geometry is.

Another single-view 3D modeling approach for omnidirectional images is due to Chen and Ip [93]. The same comment as above applies, i.e., the actual 3D modeling method is valid for any central camera and not specific to omnidirectional ones. Chen and Ip used a rectangular object in fronto-parallel position, to calibrate a radial lens distortion model and used that object also as reference plane for the 3D model.

## 6.6 Distortion Correction and Rectification

By distortion correction we understand here the generation of perspectively correct images from distorted ones. This is highly related to the problems of texture mapping and stereo rectification. Let us first consider texture mapping from one image: given a calibrated image and some surface, the goal is to “paint” the surface by back-projecting the image and its intensities onto it. Geometrically, this could be done by considering one pixel after the other; for each pixel, we determine the intersection of its camera ray and the surface and apply the pixel’s intensities at that point. It is well known that such a process may lead to holes in the paint and other artifacts and that it is recommended to proceed in the opposite direction. To do so, let us consider a representation of the texture map as a 2D digital image. In addition, we need the mapping between positions in the texture map and points on the 3D surface. The texture map is then computed similar to the above but the other way round: for each pixel of it, we compute the corresponding point on the surface and project it into the input image. The intensities applied to the texture map pixel are then computed using some interpolation scheme, from the intensities in the input image in the vicinity of the computed image position. This is a standard procedure and can be applied to any type of surface and camera.
The application of this procedure to distortion correction is straight-
forward: it suffices to choose a plane as 3D surface for this texture
mapping. The result will be a perspectively correct image, whenever
the input image was taken by a central camera (see Figure 6.1). For
non-central cameras though, generation of perspectively correct images
is only possible in general if the scene geometry is known (one can then
texture map the scene and render it from any desired viewpoint with a

<figure><img src="image_31.png" /><figcaption>Fig. 6.1 Examples for distortion correction. Left and middle: A fisheye image and distortion corrected sections of it. Right: A cubemap generated from a fisheye image.</figcaption></figure>
virtual perspective camera). Swaminathan et al. proposed an approach to obtain good approximations of perspective views if the scene structure is unknown but if simple priors on it are available [483]. A similar approach, for misaligned hyper-catadioptric cameras, was proposed by Jeng and Tsai [256]. Ding and Yu proposed an interactive approach for minimizing distortions in multi-perspective images that can be modeled using the general linear camera model (GLC, see Section 3.4.4) [116]. This problem is an instance of the more general image-based rendering problem [181, 307].

As a sidenote, let us mention an optical procedure for generating perspective images from a fisheye image, explained and demonstrated in 1925 by Beck [48]: he placed the negative of the fisheye image in the camera and reversed the action of the latter, by illuminating the negative and placing a photoplate in front of the camera. The photoplate then gets imprinted with a perspective photograph corresponding to the part of the fisheye's field of view covered by the plate.

Using planar 3D surfaces for distortion correction has its limits, since a hemispheric field of view would require an infinitely large texture map to hold the entire distortion-corrected input image. An alternative is to use cubemaps, i.e., where partial perspective view are rendered onto the insides of a virtual cube, see Figure 6.1 and for example the paper [265] by Kangni and Laganière. Cubemaps and other useful projections for environment mapping were well explained by Greene [187]. See also the work [564] by Zelnik-Manor et al. for various other suggestions for projections used to warp spherical images to flat ones.

The above texture mapping scheme can be applied for other tasks than the correction of non-perspective distortions. A typical example is to warp the usual circular omnidirectional images, which are hard to interpret by humans, into panoramic images; in that case, the 3D surface is the inside of a cylinder and the texture map corresponds to the unrolled cylinder. Besides this, the generation of the texture map is done exactly the same way as above.

Another application is stereo rectification. This consists in warping an image pair such that the warped images have an epipolar geometry where epipolar curves correspond to scanlines, i.e., where they are straight lines and such that in addition corresponding epipolar lines
have the same vertical image coordinate. This enables efficient imple-
mentations of stereo matching algorithms. As for perspective images,
rectification can be achieved by applying appropriate homographies to
each image (there exist infinitely many solutions for these homogra-
phies; different methods differ in the choice of a particular solution),
see for example [21, 65, 176, 212, 325, 451].

Geyer and Daniilidis proposed a stereo rectification method for
para-catadioptric cameras, which applies conformal warping transfor-
mations [169]. Banno and Kanade showed a rectification method for
spherical images [24]. Heller and Pajdla proposed a stereographic rec-
tification procedure for omnidirectional stereo pairs [217]. Instead of
mapping epipolar curves to scanlines, they suggested to map them to
circles; this is based on the observation that this way of rectifying min-
imizes, in some sense, the distortions between original and rectified
images.

In 3D, one may picture stereo rectification as follows. Rectified
images correspond to perspective cameras which are oriented identically
(i.e., their optical axes are parallel and the image planes are rotated
the same way) have the same focal length and whose baseline is par-
allel to the horizontal image direction. Given an input stereo pair, one
may transform it into a rectified geometry, as follows. First, compute
the baseline and choose any planar surface that is parallel to the base-
line, at some finite distance from it. That surface will play the role of
image plane for the rectified images. Next, for each camera, compute
the rotation that would make the optical axis orthogonal to that plane
and such that the image plane's horizontal axis becomes parallel to the
baseline. Finally, one has to zoom in or out one or both of the views
by appropriate factors that lead to equal focal lengths for both images.
Rotating a camera in 3D and changing its focal length is equivalent to
applying an appropriate projective transformation on its 2D image, the
homography used in rectification algorithms.

This principle can be applied to any central camera, even non-
perspective ones. The warping transformations are no longer homo-
graphies in general but this does not matter; all that is needed is the
forward projection for the cameras, the determination of the rectified
image plane, and the application of the above distortion correction
method.

One problem with this approach is that a planar image surface is not
adequate for handling wide field of view cameras, as explained above.
A second main problem is that the case where an epipole lies inside
the image cannot be handled; it would again require an infinitely large
rectified image.

For a fully general solution, one thus has to use other projection
surfaces than planes. Roy et al. used a cylindrical surface, whose axis is
orthogonal to the baseline [431]. The final rectified images are obtained
by projecting the input images on the cylinder and unrolling it. If a
parametric projection model is available for both cameras, then the
mapping from rectified to original images can be written in closed form.

<figure><img src="image_32.png" /><figcaption>Fig. 6.2 Example of the rectification procedure of Pollefeys et al. On the left, two input images (the epipoles are marked as white dots), on the right the two rectified images. Courtesy of Marc Pollefeys [404].</figcaption></figure>
Abraham and Förstner showed such a rectification procedure for fisheye
cameras [1].

Pollefeys et al. proposed a yet more general approach, which is appli-
cable even without a closed-form projection model and which exploits
the fact that in true cameras, the search for matches can usually be
restricted to half-epipolar lines [404]. Essentially, the approach gen-
erates rectified images by straightening individual epipolar curves, as
follows (cf. Figure 6.2). The set of pairs of epipolar curves are browsed.
For each pair, a new row is added to each rectified image. Roughly
speaking, the intensities of pixels along each epipolar curve are just
plotted one after the other on that new row of pixels. It does not mat-
ter if or if not the position of the plotted pixel in the rectified image is
in any simple parametric relationship to the pixel in the input image;
all that matters is that all intensities found along an epipolar curve
are reproduced on a single row in the rectified image. This approach is
highly general and is even applicable to non-central cameras, as long
as they satisfy the standard stereo geometry, cf. Section 3.4.2. A con-
ceptual difference to the above approaches is that in general, it may
not correspond to a projection on (or, texture mapping of) some 3D
surface.
We have described some image acquisition technologies and associated
camera models, with an emphasis on omnidirectional cameras. Princi-
ples and existing works on basic issues for geometric computer vision,
such as epipolar geometry, calibration, and structure-from-motion, were
reviewed. One of the goals in writing this monograph was to give a sur-
vey of the literature and to highlight links between existing works and
underlying principles. While aiming at a certain level of exhaustiveness
in the coverage of the literature, such a survey can never be complete;
we apologize for any significant omission. The reader is invited to notify
the authors of works that should be cited; an addendum to this mono-
graph shall be maintained on the first author's website.

Let us briefly summarize what we consider some of the main
issues we were trying to convey. We concentrated on models for back-
projection, in order to stress that these enable a simple and system-
atic formulation of epipolar geometry, self-calibration, pose and motion
estimation, and the projection of 3D lines. For distortion correction
and bundle adjustment however, forward projection should ideally be
used. An important issue we discussed concerns the choice of camera
model, for example if or if not a non-central model is appropriate in
some application. Our conclusion is that there is no single and simple
answer, but that this choice depends on multiple factors such as the
"true" camera geometry, the amount of noise in data, the distribution
of calibration points, and so forth.

Finally, we note that this monograph is restricted to geometric
aspects of omnidirectional vision. Equally if not more important, are
optical and photometric characteristics; these are well explained and
studied for example by Okatani and Deguchi [384] and in various papers
by Nayar et al.
Acknowledgments

P. Sturm is grateful to the Alexander-von-Humboldt Foundation for a
Research Fellowship supporting his sabbatical at TU Munich, during
which the largest part of this manuscript was written. He also acknowl-
edges support by the French ANR project CAVIAR which supported
part of the research carried out that eventually lead to the idea of writ-
ing this monograph. J. Barreto and P. Sturm are thankful for their sup-
port by the French-Portuguese collaboration program PHC Pessoa. The
authors wish to thank Tomáš Pajdla for comments on the manuscript
and Aseem Agarwala, Frédéric Devernay, Sujit Kuthirummal, Marc
Pollefeys, Ryusuke Sagawa, and Steve Seitz for kindly allowing us to
use images of their work.
[1] S. Abraham and W. Förstner, “Fish-eye-stereo calibration and epipolar rectification,” *ISPRS Journal of Photogrammetry and Remote Sensing*, vol. 59, no. 5, pp. 278–288, 2005.

[2] E. Adelson and J. Wang, “Single lens stereo with a plenoptic camera,” *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 14, pp. 99–106, February 1992.

[3] G. Adorni, M. Mordonini, S. Cagnoni, and A. Sgorbissa, “Omnidirectional stereo systems for robot navigation,” in *Proceedings of the Workshop on Omnidirectional Vision and Camera Networks*, Madison, Wisconsin, USA, 2003.

[4] A. Agarwala, M. Agrawala, M. Cohen, D. Salesin, and R. Szeliski, “Photographing long scenes with multi-viewpoint panoramas,” in *Proceedings of SIGGRAPH*, Boston, USA, pp. 853–861, 2006.

[5] A. Agrawal, Y. Taguchi, and S. Ramalingam, “Analytical forward projection for axial non-central dioptric & catadioptric cameras,” in *Proceedings of the 11th European Conference on Computer Vision, Heraklion, Greece*, pp. 129–143, 2010.

[6] M. Agrawal and L. Davis, “Camera calibration using spheres: A semi-definite programming approach,” in *Proceedings of the 9th IEEE International Conference on Computer Vision, Nice, France*, pp. 782–791, 2003.

[7] M. Ahmed and A. Farag, “A neural approach to zoom-lens camera calibration from data with outliers,” *Image and Vision Computing*, vol. 20, no. 9–10, pp. 619–630, 2002.

[8] M. Ahmed and A. Farag, “Nonmetric calibration of camera lens distortion: Differential methods and robust estimation,” *IEEE Transactions on Image Processing*, vol. 14, pp. 1215–1230, August 2005.
[9] M. Ahmed, E. Hemayed, and A. Farag, “Neurocalibration: A neural network that can tell camera calibration parameters,” in *Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Greece*, pp. 463–468, 1999.

[10] O. Ait-Aider, N. Andreff, J.-M. Lavest, and P. Martinet, “Simultaneous object pose and velocity computation using a single view from a rolling shutter camera,” in *Proceedings of the 9th European Conference on Computer Vision, Graz, Austria*, (H. Bischof and A. Leonardis, eds.), pp. 56–68, Springer-Verlag, 2006.

[11] S. Al-Ajlouni and C. Fraser, “Zoom-dependent calibration for consumer grade-cameras,” in *Proceedings of the ISPRS Commission V Symposium on Image Engineering and Vision Metrology, Dresden, Germany*, pp. 20–25, September 2006.

[12] D. Aliaga, “Accurate catadioptric calibration for real-time pose estimation in room-size environments,” in *Proceedings of the 8th IEEE International Conference on Computer Vision, Vancouver, Canada*, pp. 127–134, 2001.

[13] J. Aloimonos, “Perspective approximations,” *Image and Vision Computing*, vol. 8, pp. 179–192, August 1990.

[14] L. Alvarez, L. Gómez, and J. Sendra, “An algebraic approach to lens distortion by line rectification,” *Journal of Mathematical Imaging and Vision*, vol. 35, pp. 36–50, September 2009.

[15] J. Arnspang, H. Nielsen, M. Christensen, and K. Henriksen, “Using mirror cameras for estimating depth,” in *Proceedings of the 6th International Conference on Computer Analysis of Images and Patterns, Prague, Czech Republic*, pp. 711–716, 1995.

[16] N. Asada, A. Amano, and M. Baba, “Photometric calibration of zoom lens systems,” in *Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria*, pp. 186–190, IEEE Computer Society Press, August 1996.

[17] K. Asari, S. Kumar, and D. Radhakrishnan, “A new approach for nonlinear distortion correction in endoscopic images based on least squares estimation,” *IEEE Transactions on Medical Imaging*, vol. 18, pp. 345–354, April 1999.

[18] C. Aschenbrenner, “Neue Geräte und Methoden für die photogrammetrische Erschließung unerforschter Gebiete,” *Bildmessung und Luftbildwesen*, vol. 4, no. 1, pp. 30–38, 1929.

[19] K. Atkinson, ed., *Close Range Photogrammetry and Machine Vision*. Whittles Publishing, 1996.

[20] O. Avni, T. Baum, G. Katzir, and E. Rivlin, “Recovery of 3D animal motions using cameras and mirrors,” *Machine Vision and Applications*, vol. 21, no. 6, pp. 879–888, 2010.

[21] N. Ayache, *Stereovision and Sensor Fusion*. Cambridge, MA, USA: The MIT Press, 1990.

[22] S. Baker and S. Nayar, “A theory of single-viewpoint catadioptric image formation,” *International Journal of Computer Vision*, vol. 35, no. 2, pp. 1–22, 1999.
[23] H. Bakstein and T. Pajdla, "Panoramic mosaicing with a 180° field of view lens," in *Proceedings of the Workshop on Omnidirectional Vision, Copenhagen, Denmark*, pp. 60–68, 2002.

[24] A. Banno and K. Ikeuchi, "Omnidirectional texturing based on robust 3D registration through Euclidean reconstruction from two spherical images," *Computer Vision and Image Understanding*, vol. 114, no. 4, pp. 491–499, 2010.

[25] J. Barreto, "General central projection systems: Modeling, calibration and visual servoing," PhD thesis, Department of Electrical and Computer Engineering, University of Coimbra, Portugal, September 2003.

[26] J. Barreto, "A unifying geometric representation for central projection systems," *Computer Vision and Image Understanding*, vol. 103, no. 3, pp. 208–217, 2006.

[27] J. Barreto, "Unifying image plane liftings for central catadioptric and dioptric cameras," in *Imaging Beyond the Pinhole Camera*, (K. Daniilidis and R. Klette, eds.), Springer-Verlag, August 2006.

[28] J. Barreto and H. Araújo, "Issues on the geometry of central catadioptric image formation," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA*, pp. 422–427, 2001.

[29] J. Barreto and H. Araújo, "Geometric properties of central catadioptric line images," in *Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark*, pp. 237–251, 2002.

[30] J. Barreto and H. Araújo, "Paracatadioptric camera calibration using lines," in *Proceedings of the 9th IEEE International Conference on Computer Vision, Nice, France*, pp. 1359–1365, 2003.

[31] J. Barreto and H. Araújo, "Geometric properties of central catadioptric line images and their application in calibration," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 27, no. 8, pp. 1327–1333, 2005.

[32] J. Barreto and H. Araújo, "Fitting conics to paracatadioptric projections of lines," *Computer Vision and Image Understanding*, vol. 103, no. 3, pp. 151–165, 2006.

[33] J. Barreto and K. Daniilidis, "Wide area multiple camera calibration and estimation of radial distortion," in *Proceedings of the 5th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Prague, Czech Republic*, 2004.

[34] J. Barreto and K. Daniilidis, "Fundamental matrix for cameras with radial distortion," in *Proceedings of the 10th IEEE International Conference on Computer Vision, Beijing, China*, pp. 625–632, 2005.

[35] J. Barreto and K. Daniilidis, "Epipolar geometry of central projection systems using veronese maps," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA*, pp. 1258–1265, 2006.

[36] J. Barreto, J. Roquette, P. Sturm, and F. Fonseca, "Automatic camera calibration applied to medical endoscopy," in *Proceedings of the 20th British Machine Vision Conference, London, England*, 2009.

[37] J. Barreto, R. Swaminathan, and J. Roquette, "Non parametric distortion correction in endoscopic medical images," in *Proceedings of 3DTV-CON Conference on The True Vision, Capture, Transmission and Display of 3D Video, Kos, Greece*, 2007.
[38] M. Barth and C. Barrows, "A fast panoramic imaging system and intelligent imaging technique for mobile robots," in *Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems*, Osaka, Japan, pp. 626-633, 1996.

[39] O. Baschin, "Eine einfache Methode der stereophotogrammetrischen Küstenvermessung," *Petermanns Mitteilungen*, 1908.

[40] Y. Bastanlar, L. Puig, P. Sturm, J. Guerrero, and J. Barreto, "DLT-like calibration of central catadioptric cameras," in *Proceedings of the 8th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras*, Marseille, France, October 2008.

[41] A. Basu and S. Licardie, "Alternative models for fish-eye lenses," *Pattern Recognition Letters*, vol. 16, pp. 433-441, April 1995.

[42] J. Batista, H. Araújo, and A. de Almeida, "Iterative multi-step explicit camera calibration," in *Proceedings of the 6th IEEE International Conference on Computer Vision, Bombay, India*, pp. 709-714, January 1998.

[43] G. Batog, X. Goaoc, and J. Ponce, "Admissible linear map models of linear cameras," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, USA*, 2010.

[44] E. Bayro-Corrochano and C. López-Franco, "Omnidirectional vision: Unified model using conformal geometry," in *Proceedings of the 8th European Conference on Computer Vision, Prague, Czech Republic*, pp. 536-548, 2004.

[45] J. Bazin, C. Demonceaux, P. Vasseur, and I. Kweon, "Motion estimation by decoupling rotation and translation in catadioptric vision," *Computer Vision and Image Understanding*, vol. 114, no. 2, pp. 254-273, 2010.

[46] S. Beauchemin and R. Bajcsy, "Modelling and removing radial and tangential distortions in spherical lenses," in *Proceedings of the 10th International Workshop on Theoretical Foundations of Computer Vision, Dagstuhl Castle, Germany*, (R. Klette, T. Huang, and G. Gimel'farb, eds.), pp. 1-21, Springer-Verlag, March 2000.

[47] S. Beauchemin, R. Bajcsy, and G. Givaty, "A unified procedure for calibrating intrinsic parameters of fish-eye lenses," in *Proceedings of Vision interface, Trois-Rivières, Canada*, pp. 272-279, 1999.

[48] C. Beck, "Apparatus to photograph the whole sky," *Journal of Scientific Instruments*, vol. 2, no. 4, pp. 135-139, 1925.

[49] R. Benosman, E. Deforas, and J. Devars, "A new catadioptric sensor for the panoramic vision of mobile robots," in *Proceedings of the IEEE Workshop on Omnidirectional Vision, Hilton Head Island, South Carolina*, 2000.

[50] R. Benosman and S. Kang, "A brief historical perspective on panorama," in *Panoramic Vision: Sensors, Theory, and Applications*, (R. Benosman and S. Kang, eds.), pp. 5-20, Springer-Verlag, 2001.

[51] R. Benosman and S. Kang, eds., *Panoramic Vision*. Springer-Verlag, 2001.

[52] R. Benosman, T. Manière, and J. Devars, "Multidirectional stereovision sensor, calibration and scenes reconstruction," in *Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria*, pp. 161-165, 1996.

[53] H. Beyer, "Accurate calibration of CCD cameras," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Urbana-Champaign, Illinois, USA*, pp. 96-101, 1992.
[54] S. Bogner, "An introduction to panospheric imaging," in *Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Vancouver, Canada*, pp. 3099-3106, 1995.

[55] J. Boland, "Cameras and sensing systems," in *Manual of Photogrammetry, Fifth Edition*, (C. McGlone, ed.), ch. 8, pp. 581-676, Falls Church, Virginia, USA: American Society of Photogrammetry and Remote Sensing, 2004.

[56] R. Bolles, H. Baker, and D. Marimont, "Epipolar-plane image analysis: An approach to determining structure from motion," *International Journal of Computer Vision*, vol. 1, pp. 7-55, 1987.

[57] R. Boutteau, X. Savatier, J.-Y. Ertaud, and B. Mazari, "An omnidirectional stereoscopic system for mobile robot navigation," in *Proceedings of the IEEE International Workshop on Robotic and Sensors Environment, Ottawa, Canada*, pp. 138-143, 2008.

[58] A. Bouwers and J. van der Sande, "A new camera of extremely high luminosity and resolution based on the concentric optical mirror system," in *Proceedings of the VII ISPRS-Congress, Washington, USA*, pp. 246/1-246/3, 1952.

[59] P. Brand, R. Mohr, and P. Bobet, "Distorsions optiques: Correction dans un modèle projectif," Technical Report 1933, LIFIA-IMAG-INRIA Rhône-Alpes, Grenoble, France, June 1993.

[60] P. Brand, R. Mohr, and P. Bobet, "Distorsion optique: Correction dans un modèle projectif," in *Actes du 9ème Congrès AFCET de Reconnaissance des Formes et Intelligence Artificielle, Paris, France*, pp. 87-98, Paris, January 1994.

[61] E. Brassart, L. Delahoche, C. Cauchois, C. Drocourt, C. Pegard, and M. Mouaddib, "Experimental results got with the omnidirectional vision sensor: SYCLOP," in *Proceedings of the IEEE Workshop on Omnidirectional Vision, Hilton Head Island, South Carolina*, pp. 145-152, 2000.

[62] C. Bräuer-Burchardt and K. Voss, "A new algorithm to correct fish-eye and strong wide-angle-lens-distortion from single images," in *Proceedings of the IEEE International Conference on Image Processing, Thessaloniki, Greece*, pp. 225-228, 2001.

[63] D. Brown, "Decentering distortion of lenses," *Photogrammetric Engineering*, vol. 32, pp. 444-462, May 1966.

[64] D. Brown, "Close-range camera calibration," *Photogrammetric Engineering*, vol. 37, no. 8, pp. 855-866, 1971.

[65] L. Brown, "A survey of image registration techniques," *ACM Computing Surveys*, vol. 24, pp. 325-376, December 1992.

[66] A. Bruckstein and T. Richardson, "Omniview cameras with curved surface mirrors," in *Proceedings of the IEEE Workshop on Omnidirectional Vision, Hilton Head Island, South Carolina*, pp. 79-84, 2000.

[67] T. Buchanan, "The twisted cubic and camera calibration," *Computer Vision, Graphics and Image Processing*, vol. 42, pp. 130-132, April 1988.

[68] R. Bunschoten and B. Kröse, "3D scene reconstruction from cylindrical panoramic images," *Robotics and Autonomous Systems*, vol. 41, no. 2-3, pp. 111-118, 2002.

[69] R. Bunschoten and B. Kröse, "Robust scene reconstruction from an omnidirectional vision system," *IEEE Transactions on Robotics and Automation*, vol. 19, no. 2, pp. 351-357, 2002.
[70] A. Burner, "Zoom lens calibration for wind tunnel measurements," in *Proceedings of the SPIE Conference on Videometrics IV, Philadelphia, Pennsylvania, USA*, (S. El-Hakim, ed.), pp. 19-33, SPIE — Society of Photo-Optical Instrumentation Engineers, October 1995.

[71] P. Burns, "The history of the discovery of cinematography," http://www.precinemahistory.net.

[72] M. Byröd, Z. Kúkelová, K. Josephson, T. Pajdla, and K. Åström, "Fast and robust numerical solutions to minimal problems for cameras with radial distortion," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, USA*, 2008.

[73] E. Cabral, J. de Souza Junior, and M. Hunold, "Omnidirectional stereo vision with a hyperbolic double lobed mirror," in *Proceedings of the 15th International Conference on Pattern Recognition, Cambridge, UK*, pp. 1-4, 2004.

[74] C. Cafforio and F. Rocca, "Precise stereopsis with a single video camera," in *Proceedings of the 3rd European Signal Processing Conference (EUSIPCO): Theories and Application, The Hague, Netherlands*, pp. 641-644, 1986.

[75] V. Caglioti and S. Gasparini, "Localization of straight lines from single images using off-axis catadioptric cameras," in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Beijing, China*, 2005.

[76] V. Caglioti and S. Gasparini, "On the localization of straight lines in 3D space from single 2D images," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA*, pp. 1129-1134, 2005.

[77] V. Caglioti and S. Gasparini, ""How many planar viewing surfaces are there in noncentral catadioptric cameras?" Towards singe-image localization of space lines," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA*, pp. 1266-1273, 2006.

[78] V. Caglioti, P. Taddei, G. Boracchi, S. Gasparini, and A. Giusti, "Single-image calibration of off-axis catadioptric cameras using lines," in *Proceedings of the 7th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Rio de Janeiro, Brazil*, 2007.

[79] S. Cagnoni, M. Mordonini, L. Mussi, and G. Adorni, "Hybrid stereo sensor with omnidirectional vision capabilities: Overview and calibration procedures," in *Proceedings of the 14th International Conference on Image Analysis and Processing, Modena, Italy*, pp. 99-104, 2007.

[80] Z. Cao, S. Oh, and E. Hall, "Dynamic omnidirectional vision for mobile robots," *Journal of Robotic Systems*, vol. 3, no. 1, pp. 5-17, 1986.

[81] B. Caprile and V. Torre, "Using vanishing points for camera calibration," *International Journal of Computer Vision*, vol. 4, pp. 127-140, 1990.

[82] C. Cauchois, E. Brassart, L. Delahoche, and A. Clerentin, "3D localization with conical vision," in *Proceedings of the Workshop on Omnidirectional Vision and Camera Networks, Madison, Wisconsin, USA*, 2003.

[83] C. Cauchois, E. Brassart, C. Drocourt, and P. Vasseur, "Calibration of the omnidirectional vision sensor: SYCLOP," in *Proceedings of the IEEE International Conference on Robotics and Automation, Detroit, Michigan, USA*, pp. 1287-1292, May 1999.
[84] J. Chahl and M. Srinivasan, "Reflective surfaces for panoramic imaging," *Applied Optics*, vol. 36, no. 31, pp. 8275-8285, 1997.

[85] G. Champleboux, S. Lavallée, P. Sautot, and P. Cinquin, "Accurate calibration of cameras and range imaging sensors: The NPBS method," in *Proceedings of the IEEE International Conference on Robotics and Automation, Nice, France*, pp. 1552-1558, May 1992.

[86] P. Chang and M. Hébert, "Omni-directional structure from motion," in *Proceedings of the IEEE Workshop on Omnidirectional Vision, Hilton Head Island, South Carolina*, pp. 127-133, 2000.

[87] A. Charriou and S. Valette, "Appareil photographique multiple pour études sur la prise de vues aériennes," in *Proceedings of the V ISPRS-Congress, Rome, Italy*, pp. 154-157, 1938.

[88] C.-S. Chen and W.-Y. Chang, "On pose recovery for generalized visual sensors," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 26, pp. 848-861, July 2004.

[89] N.-Y. Chen, "Visually estimating workpiece pose in a robot hand using the feature points method," PhD thesis, University of Rhode Island, Kingston, 1979.

[90] N.-Y. Chen, J. Birk, and R. Kelley, "Estimating workpiece pose using the feature points method," *IEEE Transactions on Automatic Control*, vol. 25, pp. 1027-1041, December 1980.

[91] Q. Chen, H. Wu, and T. Wada, "Camera calibration with two arbitrary coplanar circles," in *Proceedings of the 8th European Conference on Computer Vision, Prague, Czech Republic*, pp. 521-532, 2004.

[92] S. Chen, "Quicktime VR — an image-based approach to virtual environment navigation," in *Proceedings of SIGGRAPH, Los Angeles, USA*, pp. 29-38, 1995.

[93] Y. Chen and H. Ip, "Single view metrology of wide-angle lens images," *The Visual Computer*, vol. 22, no. 7, pp. 445-455, 2006.

[94] J. Clark and G. Follin, "A simple "equal area" calibration for fisheye photography," *Agricultural and Forest Meteorology*, vol. 44, pp. 19-25, 1988.

[95] T. Clarke and J. Fryer, "The development of camera calibration methods and models," *Photogrammetric Record*, vol. 91, no. 16, pp. 51-66, 1998.

[96] T. Clarke, X. Wang, and J. Fryer, "The principal point and CCD cameras," *Photogrammetric Record*, vol. 92, no. 16, pp. 293-312, 1998.

[97] D. Claus and A. Fitzgibbon, "A plumbline constraint for the rational function lens distortion model," in *Proceedings of the 16th British Machine Vision Conference, Oxford, England*, pp. 99-108, 2005.

[98] D. Claus and A. Fitzgibbon, "A rational function lens distortion model for general cameras," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA*, pp. 213-219, 2005.

[99] T. Conroy and J. Moore, "Resolution invariant surfaces for panoramic vision systems," in *Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Greece*, pp. 392-397, 1999.

[100] K. Cornelis, M. Pollefeys, and L. van Gool, "Lens distortion recovery for accurate sequential structure and motion recovery," in *Proceedings of*
the 7th European Conference on Computer Vision, Copenhagen, Denmark,
pp. 186–200, 2002.

[101] N. Cornille, “Accurate 3D shape and displacement measurement using a scanning electron microscope,” PhD thesis, University of South Carolina and Institut National des Sciences Appliquées de Toulouse, June 2005.

[102] J. Crowley, P. Bobet, and C. Schmid, “Maintaining stereo calibration by tracking image points,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA*, pp. 483–488, June 1993.

[103] K. Daniilidis, “The page of omnidirectional vision,” http://www.cis.upenn.edu/~kostas/omni.html.

[104] N. Daucher, M. Dhome, and J.-T. Lapresté, “Camera calibration from sphere images,” in *Proceedings of the 3rd European Conference on Computer Vision, Stockholm, Sweden*, pp. 449–454, 1994.

[105] A. Davidhazy, “Camera for conical peripheral and panoramic photography,” *SIGGRAPH Course, 2007*.

[106] C. Davis and T.-H. Ho, “Using geometrical constraints for fisheye camera calibration,” in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Beijing, China*, 2005.

[107] A. Davison, I. Reid, N. Molton, and O. Stasse, “MonoSLAM: Real-time single camera SLAM,” *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 29, no. 6, pp. 1052–1067, 2007.

[108] T. Debaecker, R. Benosman, and S. Ieng, “Cone-pixels camera models,” in *Proceedings of the 8th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Marseille, France*, 2008.

[109] C. Delherm, J.-M. Lavest, M. Dhome, and J.-T. Lapresté, “Dense reconstruction by zooming,” in *Proceedings of the 4th European Conference on Computer Vision, Cambridge, England*, (B. Buxton and R. Cipolla, eds.), pp. 427–438, Springer-Verlag, 1996.

[110] X.-M. Deng, F.-C. Wu, and Y.-H. Wu, “An easy calibration method for central catadioptric cameras,” *Acta Automatica Sinica*, vol. 33, no. 8, pp. 801–808, 2007.

[111] S. Derrien and K. Konolige, “Approximating a single viewpoint in panoramic imaging devices,” in *Proceedings of the IEEE International Conference on Robotics and Automation*, pp. 3931–3938, 2000.

[112] R. Descartes, *Discours de la méthode pour bien conduire sa raison, et chercher la vérité dans les sciences*. Ian Maire, Leyden, 1637.

[113] F. Devernay and O. Faugeras, “Straight lines have to be straight,” *Machine Vision and Applications*, vol. 13, pp. 14–24, August 2001.

[114] M. Dhome, J. Lapreste, G. Rives, and M. Richetin, “Spatial localization of modelled objects of revolution in monocular perspective vision,” in *Proceedings of the 1st European Conference on Computer Vision, Antibes, France*, pp. 475–488, 1990.

[115] H. Dietz, “Fisheye digital imaging for under twenty dollars,” Technical Report, University of Kentucky, April 2006.

[116] Y. Ding and J. Yu, “Multiperspective distortion correction using collineations,” in *Proceedings of the Asian Conference on Computer Vision, Tokyo, Japan*, pp. 95–105, 2007.
[117] E. Doležal, "Photogrammetrische Lösung des Wolkenproblems aus einem Standpunkt unter Verwendung der Reflexe," *Sitzungsberichte Kaiserliche Akademie der Wissenschaften, mathematisch-naturwissenschaftliche Klasse, Abteilung IIa*, vol. 111, pp. 788-813, 1902.

[118] F. Dornaika and J. Elder, "Image registration for foveated omnidirectional sensing," in *Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark*, pp. 606-620, 2002.

[119] P. Doubek and T. Svoboda, "Reliable 3D reconstruction from a few cataractic images," in *Proceedings of the Workshop on Omnidirectional Vision, Copenhagen, Denmark*, 2002.

[120] J. Draréni, P. Sturm, and S. Roy, "Plane-based calibration for linear cameras," in *Proceedings of the 8th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Marseille, France*, October 2008.

[121] D. Drucker and P. Locke, "A natural classification of curves and surfaces with reflection properties," *Mathematics Magazine*, vol. 69, no. 4, pp. 249-256, 1996.

[122] F. Du and M. Brady, "Self-calibration of the intrinsic parameters of cameras for active vision systems," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA*, pp. 477-482, IEEE Computer Society Press, 1993.

[123] A. Dunne, J. Mallon, and P. Whelan, "A comparison of new generic camera calibration with the standard parametric approach," in *Proceedings of IAPR Conference on Machine Vision Applications, Tokyo, Japan*, pp. 114-117, 2007.

[124] A. Dunne, J. Mallon, and P. Whelan, "Efficient generic calibration method for general cameras with single centre of projection," in *Proceedings of the 11th IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil*, 2007.

[125] A. Dunne, J. Mallon, and P. Whelan, "Efficient generic calibration method for general cameras with single centre of projection," *Computer Vision and Image Understanding*, vol. 114, no. 2, pp. 220-233, 2010.

[126] T. Echigo, "A camera calibration technique using three sets of parallel lines," *Machine Vision and Applications*, vol. 3, no. 3, pp. 159-167, 1990.

[127] M. El-Melegy and A. Farag, "Nonmetric lens distortion calibration: Closed-form solutions, robust estimation and model selection," in *Proceedings of the 9th IEEE International Conference on Computer Vision, Nice, France*, pp. 554-559, 2003.

[128] R. Enciso, T. Viéville, and A. Zisserman, "An affine solution to euclidean calibration for a zoom lens," in *Proceedings of the ALCATECH Workshop, Denmark*, pp. 21-27, July 1996.

[129] C. Engels, H. Stewénius, and D. Nistér, "Bundle adjustment rules," in *Proceedings of ISPRS Symposium on Photogrammetric Computer Vision, Bonn, Germany*, 2006.

[130] F. Espuny, "A closed-form solution for the generic self-calibration of central cameras from two rotational flows," in *Proceedings of the International Conference on Computer Vision Theory and Applications, Barcelona, Spain*, pp. 26-31, 2007.
[131] F. Espuny and J. Burgos Gil, “Generic self-calibration of central cameras from two “real” rotational flows,” in *Proceedings of the 8th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras*, Mar-seille, France, 2008.

[132] J. Fabrizio and J. Devars, “An analytical solution to the perspective-n-point problem for common planar camera and for catadioptric sensor,” *International Journal of Image and Graphics*, vol. 8, pp. 135–155, January 2008.

[133] J. Fabrizio, J. Tarel, and R. Benosman, “Calibration of panoramic catadioptric sensors made easier,” in *Proceedings of the Workshop on Omnidirectional Vision, Copenhagen, Denmark*, pp. 45–52, 2002.

[134] W. Faig, “Calibration of close-range photogrammetric systems: Mathematical formulation,” *Photogrammetric Engineering & Remote Sensing*, vol. 41, pp. 1479–1486, December 1975.

[135] H. Farid and A. Popescu, “Blind removal of image non-linearities,” in *Proceedings of the 8th IEEE International Conference on Computer Vision, Vancouver, Canada*, pp. 76–81, 2001.

[136] O. Fassig, “A revolving cloud camera,” *Monthly Weather Review*, vol. 43, no. 6, pp. 274–275, 1915.

[137] O. Faugeras, Q.-T. Luong, and T. Papadopoulo, *The Geometry of Multiple Images*. MIT Press, March 2001.

[138] O. Faugeras and B. Mourrain, “On the geometry and algebra of the point and line correspondences between *n* images,” in *Proceedings of the 5th IEEE International Conference on Computer Vision, Cambridge, Massachusetts, USA*, pp. 951–956, June 1995.

[139] D. Feldman, T. Pajdla, and D. Weinshall, “On the epipolar geometry of the crossed-slits projection,” in *Proceedings of the 9th IEEE International Conference on Computer Vision, Nice, France*, pp. 988–995, 2003.

[140] R. Fergus, A. Torralba, and W. Freeman, “Random lens imaging,” *MIT-CSAIL-TR-2006-058*, Massachusetts Institute of Technology, September 2006.

[141] C. Fermüller and Y. Aloimonos, “Geometry of eye design: Biology and technology,” in *Proceedings of the 10th International Workshop on Theoretical Foundations of Computer Vision, Dagstuhl Castle, Germany*, pp. 22–38, Springer-Verlag, 2001.

[142] R. Feynman, R. Leighton, and M. Sands, *The Feynman Lectures on Physics*, Vol. 1, Mainly Mechanics, Radiation, and Heat. Addison-Wesley, 1963.

[143] M. Fiala and A. Basu, “Feature extraction and calibration for stereo reconstruction using non-svp optics in a panoramic stereo-vision sensor,” in *Proceedings of the Workshop on Omnidirectional Vision, Copenhagen, Denmark*, 2002.

[144] S. Finsterwalder, “Die geometrischen Grundlagen der Photogrammetrie,” *Jahresbericht Deutscher Mathematik*, vol. 6, pp. 1–44, 1899.

[145] A. Fitzgibbon, “Simultaneous linear estimation of multiple view geometry and lens distortion,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA*, pp. 125–132, 2001.

[146] M. Fleck, “Perspective projection: The wrong imaging model,” Technical Report TR 95–01, Department of Computer Science, University of Iowa, Iowa City, IA 52242, USA, 1995.
[147] W. Förstner, B. Wrobel, F. Paderes, R. Craig, C. Fraser, and J. Dolloff, "Analytical photogrammetric operations," in *Manual of Photogrammetry, Fifth Edition*, (C. McGlone, ed.), ch. 11, pp. 763-948, Falls Church, Virginia, USA: American Society of Photogrammetry and Remote Sensing, 2004.

[148] O. Frank, R. Katz, C. Tisse, and H. Durrant-Whyte, "Camera calibration for miniature, low-cost, wide-angle imaging system," in *Proceedings of the 18th British Machine Vision Conference, Warwick, England*, 2007.

[149] C. Fraser, "Digital camera self-calibration," *ISPRS Journal of Photogrammetry and Remote Sensing*, vol. 52, no. 4, pp. 149-159, 1997.

[150] C. Fraser and S. Al-Ajlouni, "Zoom-dependent calibration for consumer grade-cameras," *Photogrammetric Engineering & Remote Sensing*, vol. 72, pp. 1017-1026, September 2006.

[151] L. Fritz and H. Schmid, "Stellar calibration of the orbigon lens," *Photogrammetric Engineering*, vol. 40, no. 1, pp. 101-115, 1974.

[152] J. Fryer and D. Brown, "Lens distortion for close-range photogrammetry," *Photogrammetric Engineering & Remote Sensing*, vol. 52, pp. 51-58, January 1986.

[153] S. Gächter, T. Pajdla, and B. Mičušík, "Mirror design for an omnidirectional camera with a space variant imager," in *Proceedings of the Workshop on Omnidirectional Vision Applied to Robotic Orientation and Nondestructive Testing, Budapest, Hungary*, 2001.

[154] C. Gao and N. Ahuja, "Single camera stereo using planar parallel plate," in *Proceedings of the 15th International Conference on Pattern Recognition, Cambridge, UK*, pp. 108-111, 2004.

[155] C. Gao, H. Hua, and N. Ahuja, "A hemispherical imaging camera," *Computer Vision and Image Understanding*, vol. 114, no. 2, pp. 168-178, 2010.

[156] J. Gaspar, "Omnidirectional vision for mobile robot navigation," PhD thesis, Universidade Técnica de Lisboa, Portugal, December 2002.

[157] J. Gaspar, C. Deccó, J. Okamoto Jr, and J. Santos-Víctor, "Constant resolution omnidirectional cameras," in *Proceedings of the Workshop on Omnidirectional Vision, Copenhagen, Denmark*, pp. 27-34, 2002.

[158] S. Gasparini and P. Sturm, "Multi-view matching tensors from lines for general camera models," in *Tensors in Image Processing and Computer Vision*, (S. Aja-Fernández, R. de Luis García, D. Tao, and X. Li, eds.), Springer-Verlag, 2009.

[159] S. Gasparini, P. Sturm, and J. Barreto, "Plane-based calibration of central catadioptric cameras," in *Proceedings of the 12th IEEE International Conference on Computer Vision, Kyoto, Japan*, 2009.

[160] M. Gasser, "Mehrfachkammer für Aufnahmen aus Luftfahrzeugen," Patent No. 469,413, Reichspatentamt, Germany, 1926.

[161] D. Gennery, "Generalized camera calibration including fish-eye lenses," *International Journal of Computer Vision*, vol. 68, pp. 239-266, July 2006.

[162] T. Georgiev, C. Intwala, and D. Babacan, "Light-field capture by multiplexing in the frequency domain," Technical Report, Adobe Systems Incorporated, April 2007.
[163] C. Geyer and K. Daniilidis, "Catadioptric camera calibration," in *Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Greece*, pp. 398-404, 1999.

[164] C. Geyer and K. Daniilidis, "A unifying theory of central panoramic systems and practical applications," in *Proceedings of the 6th European Conference on Computer Vision, Dublin, Ireland*, (D. Vernon, ed.), pp. 445-461, Springer-Verlag, June 2000.

[165] C. Geyer and K. Daniilidis, "Catadioptric projective geometry," *International Journal of Computer Vision*, vol. 45, no. 3, pp. 223-243, 2001.

[166] C. Geyer and K. Daniilidis, "Structure and motion from uncalibrated catadioptric views," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA*, pp. 279-286, 2001.

[167] C. Geyer and K. Daniilidis, "Paracatadioptric camera calibration," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 24, pp. 687-695, May 2002.

[168] C. Geyer and K. Daniilidis, "Properties of the catadioptric fundamental matrix," in *Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark*, (A. Heyden, G. Sparr, M. Nielsen, and P. Johansen, eds.), pp. 140-154, 2002.

[169] C. Geyer and K. Daniilidis, "Conformal rectification of an omnidirectional stereo pair," in *Proceedings of the Workshop on Omnidirectional Vision and Camera Networks, Madison, Wisconsin, USA*, 2003.

[170] C. Geyer and K. Daniilidis, "Mirrors in motion: Epipolar geometry and motion estimation," in *Proceedings of the 9th IEEE International Conference on Computer Vision, Nice, France*, pp. 766-773, 2003.

[171] C. Geyer, M. Meingast, and S. Sastry, "Geometric models of rolling-shutter cameras," in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Beijing, China*, 2005.

[172] C. Geyer and H. Stewénius, "A nine-point algorithm for estimating paracatadioptric fundamental matrices," in *Proceedings of the 11th IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil*, pp. 1-8, 2007.

[173] J. Gluckman and S. Nayar, "Ego-motion and omnidirectional cameras," in *Proceedings of the 6th IEEE International Conference on Computer Vision, Bombay, India*, pp. 999-1005, January 1998.

[174] J. Gluckman and S. Nayar, "Planar catadioptric stereo: geometry and calibration," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado, USA*, pp. 22-28, 1999.

[175] J. Gluckman and S. Nayar, "Catadioptric stereo using planar mirrors," *International Journal of Computer Vision*, vol. 44, pp. 65-79, August 2001.

[176] J. Gluckman and S. Nayar, "Rectifying transformations that minimize resampling effects," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA*, pp. 111-117, 2001.

[177] J. Gluckman and S. Nayar, "Rectified catadioptric stereo sensors," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 24, pp. 224-236, February 2002.
[178] S. Godber, R. Petty, M. Robinson, and J. Evans, "Panoramic line-scan imaging system for teleoperator control," in *Proceedings of SPIE, Stereoscopic Displays and Virtual Reality Systems*, pp. 247-257, 1994.

[179] N. Gonçalvez and H. Araújo, "Projection model, 3D reconstruction and rigid motion estimation from non-central catadioptric images," in *Proceedings of the Second International Symposium on 3D Data Processing, Visualization and Transmission, Chapel Hill, USA*, pp. 325-332, 2004.

[180] N. Gonçalvez and H. Araújo, "Estimating parameters of noncentral catadioptric systems using bundle adjustment," *Computer Vision and Image Understanding*, vol. 113, pp. 11-28, January 2009.

[181] S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, "The lumigraph," in *Proceedings of SIGGRAPH, New Orleans, LA*, pp. 43-54, 1996.

[182] A. Goshtasby, "Correction of image deformation from lens distortion using Bezier patches," *Computer Vision, Graphics and Image Processing*, vol. 47, pp. 385-394, 1989.

[183] A. Goshtasby and W. Gruver, "Design of a single-lens stereo camera system," *Pattern Recognition*, vol. 26, no. 6, pp. 923-938, 1993.

[184] S. Gourichon, J. Meyer, S. Ieng, L. Smadja, and R. Benosman, "Estimating ego-motion using a panoramic sensor: Comparison between a bio-inspired and a camera-calibrated method," in *Proceedings of the AISB Symposium on Biologically Inspired Vision, Theory and Application*, pp. 91-101, 2003.

[185] G. Gracie, "Analytical photogrammetry applied to single terrestrial photograph mensuration," in *Proceedings of the XIth International Congress of Photogrammetry, Lausanne, Switzerland*, July 1968.

[186] W. Green, P. Jepsen, J. Kreznar, R. Ruiz, A. Schwartz, and J. Seidman, "Removal of instrument signature from mariner 9 television images of mars," *Applied Optics*, vol. 14, no. 1, pp. 105-114, 1975.

[187] N. Greene, "Environment mapping and other applications of world projections," *IEEE Transactions on Computer Graphics and Applications*, vol. 6, no. 11, pp. 21-29, 1986.

[188] P. Greguss, "The tube peeper: A new concept in endoscopy," *Optics & Laser Technology*, vol. 17, no. 1, pp. 41-45, 1985.

[189] K. Gremban, C. Thorpe, and T. Kanade, "Geometric camera calibration using systems of linear equations," in *Proceedings of the IEEE International Conference on Robotics and Automation, Philadelphia, Pennsylvania, USA*, pp. 562-567, IEEE Computer Society Press, 1988.

[190] W. Grosky and L. Tamburino, "A unified approach to the linear camera calibration problem," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 12, pp. 663-671, July 1990.

[191] M. Grossberg and S. Nayar, "The raxel imaging model and ray-based calibration," *International Journal of Computer Vision*, vol. 61, no. 2, pp. 119-137, 2005.

[192] E. Grossmann, J. Gaspar, and F. Orabona, "Discrete camera calibration from pixel streams," *Computer Vision and Image Understanding*, vol. 114, no. 2, pp. 198-209, 2010.

[193] E. Grossmann, E.-J. Lee, P. Hislop, D. Nistér, and H. Stewénius, "Are two rotational flows sufficient to calibrate a smooth non-parametric sensor?" in
*Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, New York, USA, pp. 1222–1229, 2006.

[194] E. Grossmann, F. Orabona, and J. Gaspar, "Discrete camera calibration from the information distance between pixel streams," in *Proceedings of the 7th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras*, Rio de Janeiro, Brazil, 2007.

[195] A. Gruen and T. Huang, eds., *Calibration and Orientation of Cameras in Computer Vision*. Springer-Verlag, 2001.

[196] X. Gu, S. Gortler, and M. Cohen, "Polyhedral geometry and the two-plane parameterization," in *Proceedings of the Eurographics Workshop on Rendering Techniques*, St. Etienne, France, pp. 1–12, 1997.

[197] R. Gupta and R. Hartley, "Linear pushbroom cameras," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 19, pp. 963–975, September 1997.

[198] R. Gupta and R. Hartley, "Camera estimation for orbiting pushbroom imaging systems," unpublished, 2007.

[199] P. Gurdjos, P. Sturm, and Y. Wu, "Euclidean structure from $n \ge$ 2 parallel circles: Theory and algorithms," in *Proceedings of the 9th European Conference on Computer Vision, Graz, Austria*, (H. Bischof and A. Leonardis, eds.), pp. 238–252, May 2006.

[200] P. Hall, J. Collomosse, Y. Song, P. Shen, and C. Li, "RTcams: A new perspective on non-photorealistic rendering from photographs," *IEEE Transactions on Visualization and Graphics*, vol. 13, no. 5, pp. 966–979, 2007.

[201] B. Hallert, "A new method for the determination of the distortion and the inner orientation of cameras and projectors," *Photogrammetria*, vol. 11, pp. 107–115, 1954–1955.

[202] B. Hallert, "The method of least squares applied to multicollimator camera calibration," *Photogrammetric Engineering*, vol. 29, no. 5, pp. 836–840, 1963.

[203] J. Han and K. Perlin, "Measuring bidirectional texture reflectance with a kaleidoscope," *ACM Transactions on Graphics*, vol. 22, no. 3, pp. 741–748, 2003.

[204] H. Haneishi, Y. Yagihashi, and Y. Miyake, "A new method for distortion correction of electronic endoscope images," *IEEE Transactions on Medical Imaging*, vol. 14, pp. 548–555, September 1995.

[205] R. Haralick, C. Lee, K. Ottenberg, and M. Nolle, "Review and analysis of solutions of the three point perspective pose estimation problem," *International Journal of Computer Vision*, vol. 13, no. 3, pp. 331–356, 1994.

[206] R. Hartley, "Estimation of relative camera positions for uncalibrated cameras," in *Proceedings of the 2nd European Conference on Computer Vision, Santa Margherita Ligure, Italy*, (G. Sandini, ed.), pp. 579–587, Springer-Verlag, 1992.

[207] R. Hartley, "An algorithm for self calibration from several views," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, Seattle, Washington, USA, pp. 908–912, 1994.

[208] R. Hartley, "In defence of the 8-point algorithm," in *Proceedings of the 5th IEEE International Conference on Computer Vision, Cambridge, Massachusetts*, USA, pp. 1064–1070, June 1995.
[209] R. Hartley and S. Kang, “Parameter-free radial distortion correction with center of distortion estimation,” *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 29, no. 8, pp. 1309–1321, 2007.

[210] R. Hartley and T. Saxena, “The cubic rational polynomial camera model,” in *Proceedings of the DARPA Image Understanding Workshop, New Orleans, Louisiana, USA*, pp. 649–653, 1997.

[211] R. Hartley and P. Sturm, “Triangulation,” *Computer Vision and Image Understanding*, vol. 68, no. 2, pp. 146–157, 1997.

[212] R. Hartley and A. Zisserman, *Multiple View Geometry in Computer Vision*. Cambridge University Press, June 2000.

[213] R. Hartley and A. Zisserman, *Multiple View Geometry in Computer Vision*. Cambridge University Press, 2nd Edition, March 2004.

[214] J. Havlena, A. Torii, J. Knopp, and T. Pajdla, “Randomized structure from motion based on atomic 3D models from camera triplets,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA*, pp. 2874–2881, 2009.

[215] J. Heikkilä, “Geometric camera calibration using circular control points,” *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no. 10, pp. 1066–1077, 2000.

[216] J. Heikkilä and O. Silvén, “Calibration procedure for short focal length off-the-shelf CCD cameras,” in *Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria*, pp. 166–170, IEEE Computer Society Press, August 1996.

[217] J. Heller and T. Pajdla, “Stereographic rectification of omnidirectional stereo pairs,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA*, pp. 1414–1421, 2009.

[218] H.-J. Hellmeier, “Fisheye-Objektive in der Nahbereichsphotogrammetrie — Theoretische und praktische Untersuchungen,” PhD thesis, Technische Universität Braunschweig, Germany, 1983.

[219] T. Herbert, “Calibration of fisheye lenses by inversion of area projections,” *Applied Optics*, vol. 25, no. 12, pp. 1875–1876, 1986.

[220] R. Hicks, “The page of catadioptric sensor design,” http://www.math.drexel.edu/ahicks/design/.

[221] R. Hicks, “Designing a mirror to realize a given projection,” *Journal of the Optical Society of America A*, vol. 22, no. 2, pp. 323–330, 2005.

[222] R. Hicks and R. Bajcsy, “Catadioptric sensors that approximate wide-angle perspective projections,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, USA*, pp. 545–551, 2000.

[223] R. Hicks and R. Bajcsy, “Reflective surfaces as computational sensors,” *Image and Vision Computing*, vol. 19, no. 11, pp. 773–777, 2001.

[224] R. Hicks, V. Nasis, and T. Kurzweg, “Programmable imaging with two-axis micromirrors,” *Optics Letters*, vol. 32, no. 9, pp. 1066–1068, 2007.

[225] R. Hicks and R. Perline, “Equi-areal catadioptric sensors,” in *Proceedings of the Workshop on Omnidirectional Vision, Copenhagen, Denmark*, pp. 13–18, 2002.
[226] R. Hill, "A lens for whole sky photographs," *Quarterly Journal of the Royal Meteorological Society*, vol. 50, no. 211, pp. 227-235, 1924.

[227] historiccamera.com, "Illustrated history of photography," http://www.historiccamera.com/history1/photo_history300.html.

[228] S. Hiura, A. Mohan, and R. Raskar, "Krill-eye: Superposition compound eye for wide-angle imaging via GRIN lenses," in *Proceedings of the 9th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras*, Kyoto, Japan, 2009.

[229] O. Holmes, "The stereoscope and the stereograph," *The Atlantic Monthly*, vol. 3, pp. 738-749, June 1859.

[230] R. Holt and A. Netravali, "Camera calibration problem: Some new results," *Computer Vision, Graphics and Image Processing: Image Understanding*, vol. 54, pp. 368-383, November 1991.

[231] J. Hong, X. Tan, B. Pinette, R. Weiss, and E. Riseman, "Image-based homing," in *Proceedings of the IEEE International Conference on Robotics and Automation, Sacramento, California, USA*, pp. 620-625, April 1991.

[232] R. Horaud, F. Dornaika, B. Lamiroy, and S. Christy, "Object pose: The link between weak perspective, paraperspective and full perspective," *International Journal of Computer Vision*, vol. 22, pp. 173-189, March 1997.

[233] B. Horn, H. Hilden, and S. Negahdaripour, "Closed-form solution of absolute orientation using orthonormal matrices," *Journal of the Optical Society of America A*, vol. 5, pp. 1127-1135, July 1988.

[234] H. Hua and N. Ahuja, "A high-resolution panoramic camera," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA*, pp. 960-967, 2001.

[235] H. Hua, N. Ahuja, and C. Gao, "Design analysis of a high-resolution panoramic camera using conventional imagers and a mirror pyramid," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 29, no. 2, pp. 356-361, 2007.

[236] F. Huang, "Epipolar geometry in concentric panoramas," Technical Report CTU-CMP-2000-07, Center for Machine Perception, Czech Technical University, Prague, 2000.

[237] F. Huang, R. Klette, and Y.-H. Xie, "Sensor pose estimation from multi-center cylindrical panoramas," in *Proceedings of the Third Pacific Rim Symposium on Advances in Image and Video Technology, Tokyo, Japan*, pp. 48-59, 2008.

[238] F. Huang, S. Wei, and R. Klette, "Epipolar geometry in polycentric panoramas," in *Proceedings of the 10th International Workshop on Theoretical Foundations of Computer Vision, Dagstuhl Castle, Germany*, (R. Klette, T. Huang, and G. Gimel'farb, eds.), pp. 39-50, Springer-Verlag, 2000.

[239] F. Huang, S.-K. Wei, and R. Klette, "Comparative studies of line-based panoramic camera calibration," in *Proceedings of the Workshop on Omnidirectional Vision and Camera Networks, Madison, Wisconsin, USA*, 2003.

[240] C. Hughes, P. Denny, M. Glavin, and E. Jones, "Equidistant fish-eye calibration and rectification by vanishing point extraction," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 32, no. 12, pp. 2289-2296, 2010.
[241] C. Hughes, R. McFeely, P. Denny, M. Glavin, and E. Jones, "Equidistant ($f\theta$) fish-eye perspective with application in distortion centre estimation," *Image and Vision Computing*, vol. 28, no. 3, pp. 538-551, 2010.

[242] Y. Hwang, J. Lee, and H. Hong, "Omnidirectional camera calibration and 3D reconstruction by contour matching," in *Proceedings of the Second International Symposium on Visual Computing*, Lake Tahoe, USA, pp. 881-890, 2006.

[243] N. Ichimura and S. K. Nayar, "A framework for 3D pushbroom imaging," Technical Report CUCS-002-03, Department of Computer Science, Columbia University, 2003.

[244] M. Inaba, T. Hara, and H. Inoue, "A stereo viewer based on a single camera with view-control mechanisms," in *Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems*, Tokyo, Japan, pp. 1857-1864, 1993.

[245] A. Inoue, K. Yamamoto, N. Mizoue, and Y. Kawahara, "Calibrating view angle and lens distortion of the Nikon fish-eye converter FC-E8," *Journal for Forestry Research*, vol. 9, pp. 177-181, 2004.

[246] H. Ishiguro, M. Yamamoto, and S. Tsuji, "Omni-directional stereo for making global map," in *Proceedings of the 3rd IEEE International Conference on Computer Vision, Osaka, Japan*, pp. 540-547, 1990.

[247] H. Ishiguro, M. Yamamoto, and S. Tsuji, "Omni-directional stereo," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 14, no. 2, pp. 257-262, 1992.

[248] F. Ives, "Parallax stereogram and process of making same, U.S. Patent 725,567," 1903.

[249] H. Ives, "A camera for making parallax panoramagrams," *Journal of the Optical Society of America*, vol. 17, pp. 435-437, 1928.

[250] U. Iwerks, "Panoramic motion picture camera arrangement, U.S. Patent 3,118, 340," 1964.

[251] A. Izaguirre, P. Pu, and J. Summers, "A new development in camera calibration — calibrating a pair of mobile cameras," in *Proceedings of the IEEE International Conference on Robotics and Automation*, Saint Louis, Michigan, USA, pp. 74-79, 1985.

[252] M. Jackowski, A. Goshtasby, S. Bines, D. Roseman, and C. Yu, "Correcting the geometry and color of digital images," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 19, pp. 1152-1158, October 1997.

[253] B. Jähne, *Digitale Bildverarbeitung*. Springer-Verlag, 1st Edition, 1989.

[254] B. Jähne, *Digital Image Processing: Concepts, Algorithms, and Scientific Applications*. Springer-Verlag, 1st Edition, 1991.

[255] G. Jang, S. Kim, and I. Kweon, "Single camera catadioptric stereo system," in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras*, Beijing, China, 2005.

[256] S. Jeng and W. Tsai, "Analytic image unwarping by a systematic calibration method for omni-directional cameras with hyperbolic-shaped mirrors," *Image and Vision Computing*, vol. 26, pp. 690-701, May 2008.
[257] G. Jiang, H.-T. Tsui, L. Quan, and A. Zisserman, "Single axis geometry by fitting conics," in *Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark*, pp. 537-550, 2002.

[258] A. Jones, P. Debevec, M. Bolas, and I. McDowall, "Concave surround optics for rapid multi-view imaging," in *Proceedings of the 25th Army Science Conference, Orlando, USA*, 2006.

[259] K. Josephson and M. Byröd, "Pose estimation with radial distortion and unknown focal length," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA*, 2009.

[260] F. Kahl, S. Agarwal, M. Chandraker, D. Kriegman, and S. Belongie, "Practical global optimization for multiview geometry," *International Journal of Computer Vision*, vol. 79, no. 3, pp. 271-284, 2008.

[261] S. Kaneko and T. Honda, "Calculation of polyhedral objects using direct and mirror images," *Journal of the Japan Society for Precision Engineering*, vol. 52, no. 1, pp. 149-155, 1986.

[262] S. Kang, "Catadioptric self-calibration," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, USA*, pp. 201-207, 2000.

[263] S. Kang, "Radial distortion snakes," *IEICE Transactions on Information and Systems*, vol. E84-D, no. 12, pp. 1603-1611, 2001.

[264] S. Kang and R. Szeliski, "3-D scene data recovery using omnidirectional multi-baseline stereo," *International Journal of Computer Vision*, vol. 25, no. 2, pp. 167-183, 1997.

[265] F. Kangni and R. Laganière, "Epipolar geometry for the rectification of cubic panoramas," in *Proceedings of the 3rd Canadian Conference on Computer and Robot Vision, Québec City, Canada*, 2006.

[266] J. Kannala and S. Brandt, "A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 28, no. 8, pp. 1335-1340, 2006.

[267] J. Kannala, S. Brandt, and J. Heikkilä, "Self-calibration of central cameras by minimizing angular error," in *Proceedings of the International Conference on Computer Vision Theory and Applications, Funchal, Portugal*, 2008.

[268] U.-P. Käppeler, M. Höferlin, and P. Levi, "3D object localization via stereo vision using an omnidirectional and a perspective camera," in *Proceedings of the 2nd Workshop on Omnidirectional Robot Vision, Anchorage, Alaska*, pp. 7-12, 2010.

[269] R. Karren, "Camera calibration by the multicollimator method," *Photogrammetric Engineering*, vol. 34, no. 7, pp. 706-719, 1968.

[270] K. Kato, T. Nakanishi, A. Shio, and K. Ishii, "Structure from image sequences captured through a monocular extra-wide angle lens," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, Washington, USA*, pp. 919-924, 1994.

[271] T. Kawanishi, K. Yamazawa, H. Iwasa, H. Takemura, and N. Yokoya, "Generation of high-resolution stereo panoramic images by omnidirectional imaging sensor using hexagonal pyramidal mirrors," in *Proceedings of the 14th International Conference on Pattern Recognition, Brisbane, Australia*, pp. 485-489, 1998.
[272] M. Kedzierski and A. Fryskowska, "Precise method of fisheye lens calibration," in *Proceedings of the ISPRS-Congress, Beijing, China*, pp. 765-768, 2008.

[273] S. Khan, F. Rafi, and M. Shah, "Where was the picture taken: Image localization in route panoramas using epipolar geometry," in *Proceedings of the IEEE International Conference on Multimedia and Expo, Toronto, Canada*, pp. 249-252, 2006.

[274] E. Kilpelä, "Compensation of systematic errors of image and model coordinates," *International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, vol. XXIII, no. B9, pp. 407-427, 1980.

[275] E. Kilpelä, "Compensation of systematic errors of image and model coordinates," *Photogrammetria*, vol. 37, no. 1, pp. 15-44, 1981.

[276] E. Kilpelä, J. Heikkilä, and K. Inkilä, "Compensation of systematic errors in bundle adjustment," *Photogrammetria*, vol. 37, no. 1, pp. 1-13, 1981.

[277] J.-H. Kim, H. Li, and R. Hartley, "Motion estimation for nonoverlapping multicamera rigs: Linear algebraic and $l_\infty$ geometric solutions," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 32, no. 6, pp. 1044-1059, 2010.

[278] J.-S. Kim, M. Hwangbo, and T. Kanade, "Spherical approximation for multiple cameras in motion estimation: Its applicability and advantages," *Computer Vision and Image Understanding*, vol. 114, no. 10, pp. 1068-1083, 2010.

[279] J.-S. Kim and T. Kanade, "Degeneracy of the linear seventeen-point algorithm for generalized essential matrix," *Journal of Mathematical Imaging and Vision*, vol. 37, no. 1, pp. 40-48, 2010.

[280] W. Kim and H. Cho, "Learning-based constitutive parameters estimation in an image sensing system with multiple mirrors," *Pattern Recognition*, vol. 33, no. 7, pp. 1199-1217, 2000.

[281] R. Klette, G. Gimel'farb, S. Wei, F. Huang, K. Scheibe, M. Scheele, A. Börner, and R. Reulke, "On design and applications of cylindrical panoramas," in *Proceedings of the 10th International Conference on Computer Analysis of Images and Patterns, Groningen, The Netherlands*, pp. 1-8, Springer-Verlag, 2003.

[282] Y. Kojima, R. Sagawa, T. Echigo, and Y. Yagi, "Calibration and performance evaluation of omnidirectional sensor with compound spherical mirrors," in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Beijing, China*, 2005.

[283] O. Kölbl, "Analytische Verzeichnungsdarstellung bei der vollständigen Kalibrierung," *Bildmessung und Luftbildwesen*, vol. 39, no. 4, pp. 169-176, 1971.

[284] K. Kondo, Y. Mukaigawa, T. Suzuki, and Y. Yagi, "Evaluation of HBP mirror system for remote surveillance," in *Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China*, pp. 3454-3461, 2006.

[285] K. Kondo, Y. Mukaigawa, and Y. Yagi, "Free-form mirror design inspired by photometric stereo," in *Proceedings of the 8th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Marseille, France*, 2008.
[286] K. Kondo, Y. Yagi, and M. Yachida, "Non-isotropic omnidirectional imaging system for an autonomous mobile robot," in *Proceedings of the IEEE International Conference on Robotics and Automation, Barcelona, Spain*, pp. 1228-1233, 2005.

[287] K. Kraus, *Photogrammetry: Geometry from Images and Laser Scans*. Walter de Gruyter, Berlin, 2nd Edition, 2007.

[288] A. Krishnan and N. Ahuja, "Panoramic image acquisition," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, California, USA*, pp. 379–384, 1996.

[289] G. Krishnan and S. Nayar, "Cata-fisheye camera for panoramic imaging," in *Proceedings of the IEEE Workshop on Applications of Computer Vision*, 2008.

[290] G. Krishnan and S. Nayar, "Towards a true spherical camera," in *Proceedings of the SPIE — Human Vision and Electronic Imaging XIV*, 2009.

[291] Z. Kúkelová, M. Byröd, K. Josephson, T. Pajdla, and K. Åström, "Fast and robust numerical solutions to minimal problems for cameras with radial distortion," *Computer Vision and Image Understanding*, vol. 114, no. 2, pp. 234–244, 2010.

[292] Z. Kúkelová and T. Pajdla, "A minimal solution to the autocalibration of radial distortion," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, USA*, 2007.

[293] Z. Kúkelová and T. Pajdla, "Two minimal problems for cameras with radial distortion," in *Proceedings of the 7th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Rio de Janeiro, Brazil*, 2007.

[294] J. Kumler and M. Bauer, "Fish-eye lens designs and their relative performance," in *Proceedings of the SPIE Conference on Current Developments in Lens Design and Optical Systems Engineering, San Diego, USA*, pp. 360–369, 2000.

[295] S. Kuthirummal and S. Nayar, "Multiview radial catadioptric imaging for scene capture," *ACM Transactions on Graphics*, vol. 25, no. 3, pp. 916–923, 2006.

[296] S. Kuthirummal and S. Nayar, "Flexible mirror imaging," in *Proceedings of the 7th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Rio de Janeiro, Brazil*, 2007.

[297] G.-I. Kweon, H.-B. Seung, G.-H. Kim, S.-C. Yang, and Y.-H. Lee, "Wide-angle catadioptric lens with a rectilinear projection scheme," *Applied Optics*, vol. 45, no. 34, pp. 8659–8673, 2006.

[298] D. Lanman, D. Crispell, M. Wachs, and G. Taubin, "Spherical catadioptric arrays: Construction, multi-view geometry, and calibration," in *Proceedings of the Third International Symposium on 3D Data Processing, Visualization, and Transmission, Chapel Hill, USA*, pp. 81–88, 2006.

[299] D. Lanman, M. Wachs, G. Taubin, and F. Cukierman, "Reconstructing a 3D line from a single catadioptric image," in *Proceedings of the Third International Symposium on 3D Data Processing, Visualization, and Transmission, Chapel Hill, USA*, pp. 89–96, 2006.

[300] J.-M. Lavest, C. Delherm, B. Peuchot, and N. Daucher, "Implicit reconstruction by zooming," *Computer Vision and Image Understanding*, vol. 66, no. 3, pp. 301–315, 1997.
[301] J.-M. Lavest, B. Peuchot, C. Delherm, and M. Dhome, "Reconstruction by zooming from implicit calibration," in *Proceedings of the 1st IEEE International Conference on Image Processing*, pp. 1012–1016, 1994.

[302] J.-M. Lavest, M. Viala, and M. Dhome, "Do we really need an accurate calibration pattern to achieve a reliable camera calibration?," in *Proceedings of the 5th European Conference on Computer Vision, Freiburg, Germany*, pp. 158–174, May 1998.

[303] D. Lee, I. Kweon, and R. Cipolla, "A biprism-stereo camera system," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado, USA*, pp. 82–87. 1999.

[304] J. Lee, S. You, and U. Neumann, "Large motion estimation for omnidirectional vision," in *Proceedings of the IEEE Workshop on Omnidirectional Vision, Hilton Head Island, South Carolina*, 2000.

[305] R. Lenz and R. Tsai, "Techniques for calibration of the scale factor and image center for high accuracy 3D machine vision metrology," in *Proceedings of the IEEE International Conference on Robotics and Automation, Raleigh, North Carolina, USA*, pp. 68–75, 1987.

[306] R. Lenz and R. Tsai, "Techniques for calibration of the scale factor and image center for high accuracy 3D machine vision metrology," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 10, pp. 713–720, September 1988.

[307] M. Levoy and P. Hanrahan, "Light field rendering," in *Proceedings of the SIGGRAPH, New Orleans, LA*, pp. 31–42, 1996.

[308] M. Lhuillier, "Automatic structure and motion using a catadioptric camera," in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Beijing, China*, 2005.

[309] M. Lhuillier, "Effective and generic structure from motion using angular error," in *Proceedings of the 18th International Conference on Pattern Recognition, Hong Kong*, pp. 67–70, 2006.

[310] M. Lhuillier, "Automatic scene structure and camera motion using a catadioptric system," *Computer Vision and Image Understanding*, vol. 109, pp. 186–203, February 2008.

[311] H. Li and R. Hartley, "A non-iterative method for correcting lens distortion from nine point correspondences," in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Beijing, China*, 2005.

[312] H. Li and R. Hartley, "Plane-based calibration and auto-calibration of a fish-eye camera," in *Proceedings of the Asian Conference on Computer Vision, Hyderabad, India*, pp. 21–30, 2006.

[313] M. Li and J.-M. Lavest, "Some aspects of zoom lens camera calibration," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 18, pp. 1105–1110, November 1996.

[314] Y. Li, H.-Y. Shum, C.-K. Tang, and R. Szeliski, "Stereo reconstruction from multiperspective panoramas," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 26, no. 1, pp. 45–62, 2004.

[315] D. Lichti and M. Chapman, "CCD camera calibration using the finite element method," in *Proceedings of the SPIE Conference on Videometrics IV*,
Philadelphia, Pennsylvania, USA, (S. El-Hakim, ed.), SPIE - Society of Photo-Optical Instrumentation Engineers, October 1995.

[316] D. Lichti and M. Chapman, "Constrained FEM self-calibration," *Photogrammetric Engineering & Remote Sensing*, vol. 63, pp. 1111-1119, September 1997.

[317] J. Lim and N. Barnes, "Estimation of the epipole using optical flow at antipodal points," *Computer Vision and Image Understanding*, vol. 114, no. 2, pp. 245-253, 2010.

[318] J. Lim, N. Barnes, and H. Li, "Estimating relative camera motion from the antipodal-epipolar constraint," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 32, no. 10, pp. 1907-1914, 2010.

[319] S.-S. Lin and R. Bajcsy, "Single-view-point omnidirectional catadioptric cone mirror imager," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 28, pp. 840-845, May 2006.

[320] S.-S. Lin and R. Bajcsy, "Single-viewpoint, catadioptric cone mirror omnidirectional imaging theory and analysis," *Journal of the Optical Society of America A*, vol. 23, no. 12, pp. 2997-3015, 2006.

[321] S.-S. Lin and R. Bajczy, "High resolution catadioptric omni-directional stereo sensor for robot vision," in *Proceedings of the IEEE International Conference on Robotics and Automation, Taipei, Taiwan*, pp. 1694-1699, 2003.

[322] A. Lippman, "Movie-maps: An application of the optical videodisc to computer graphics," *ACM Transactions on Graphics*, vol. 14, no. 3, pp. 32-42, 1980.

[323] G. Lippmann, "Épreuves réversibles donnant la sensation du relief," *Journal de Physique*, vol. 7, pp. 821-825, 1908.

[324] H. Longuet-Higgins, "A computer program for reconstructing a scene from two projections," *Nature*, vol. 293, pp. 133-135, September 1981.

[325] C. Loop and Z. Zhang, "Computing rectifying homographies for stereo vision," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado, USA*, pp. 125-131, 1999.

[326] T. Luhmann, "A historical review on panoramic photogrammetry," in *Proceedings of the ISPRS Workshop on Panorama Photogrammetry, Dresden, Germany*, 2004.

[327] L. Ma, Y. Chen, and K. Moore, "Rational radial distortion models of camera lenses with analytical solution for distortion correction," *International Journal of Information Acquisition*, vol. 1, no. 2, pp. 135-147, 2004.

[328] Y. Ma, S. Soatto, J. Kosecka, and S. Sastry, *An Invitation to 3-D Vision — From Images to Geometric Models*. Springer-Verlag, 2005.

[329] A. Majumder, W. Seales, M. Gopi, and H. Fuchs, "Immersive teleconferencing: A new algorithm to generate seamless panoramic video imagery," in *Proceedings of the seventh ACM international conference on Multimedia, Orlando, USA*, pp. 169-178, 1999.

[330] J. Mallon and P. Whelan, "Precise radial un-distortion of images," in *Proceedings of the 15th International Conference on Pattern Recognition, Cambridge, UK*, pp. 18-21, 2004.

[331] H. Martins, J. Birk, and R. Kelley, "Camera models based on data from two calibration planes," *Computer Graphics and Image Processing*, vol. 17, pp. 173-180, 1981.
[332] T. Mashita, Y. Iwai, and M. Yachida, "Calibration method for misaligned catadioptric camera," in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras*, Beijing, China, 2005.

[333] H. Mathieu and F. Devernay, "Système de miroirs pour la stéréoscopie," Technical Report Rapport Technique 0172, INRIA, 1995.

[334] M. Maurette, "Mars rover autonomous navigation," *Autonomous Robots*, vol. 14, pp. 199-208, March 2003.

[335] B. McBride, "A timeline of panoramic cameras," http://www.panoramic photo.com/timeline.htm.

[336] C. McGlone, ed., *Manual of Photogrammetry*. Falls Church, Virginia, USA: American Society of Photogrammetry and Remote Sensing, 5th Edition, 2004.

[337] G. McLean, "Image warping for calibration and removal of lens distortion," in *Proceedings of the IEEE Pacific Rim Conference on Communications, Computers and Signal Processing, Victoria, Canada*, pp. 170-173, 1993.

[338] L. McMillan and G. Bishop, "Plenoptic modeling: An image-based rendering system," in *Proceedings of the SIGGRAPH, Los Angeles, USA*, pp. 39-46, 1995.

[339] C. Mei and P. Rives, "Single view point omnidirectional camera calibration from planar grids," in *Proceedings of the IEEE International Conference on Robotics and Automation, Rome, Italy*, pp. 3945-3950, April 2007.

[340] M. Meingast, C. Geyer, and S. Sastry, "Geometric models of rolling-shutter cameras," in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Beijing, China*, 2005.

[341] E. Menegatti, "Omnidirectional Vision for Mobile Robots," PhD thesis, Università di Padova, Italy, December 2002.

[342] M. Menem and T. Pajdla, "Constraints on perspective images and circular panoramas," in *Proceedings of the 15th British Machine Vision Conference, Kingston upon Thames, England*, 2004.

[343] H. Mitsumoto, S. Tamura, K. Okazaki, N. Kajimi, and Y. Fukui, "3-D reconstruction using mirror images based on a plane symmetry recovering method," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 14, pp. 941-946, September 1992.

[344] B. Mičušik, "Two-view geometry of omnidirectional cameras," PhD thesis, Faculty of Electrical Engineering, Czech Technical University, Prague, June 2004.

[345] B. Mičušik, D. Martinec, and T. Pajdla, "3D metric reconstruction from uncalibrated omnidirectional images," in *Proceedings of the Asian Conference on Computer Vision, Jeju Island, Korea*, 2004.

[346] B. Mičušik and T. Pajdla, "Estimation of omnidirectional camera model from epipolar geometry," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Madison, Wisconsin, USA*, 2003.

[347] B. Mičušik and T. Pajdla, "Autocalibration and 3D reconstruction with non-central catadioptric cameras," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Washington, USA*, pp. 58-65, 2004.

[348] B. Mičušik and T. Pajdla, "Para-catadioptric camera auto-calibration from epipolar geometry," in *Proceedings of the Asian Conference on Computer Vision, Jeju Island, Korea*, 2004.
[349] B. Mičušik and T. Pajdla, "Structure from motion with wide circular field of view cameras," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 28, no. 7, pp. 1-15, 2006.

[350] K. Miyamoto, "Fish eye lens," *Journal of the Optical Society of America*, vol. 54, no. 8, pp. 1060-1061, 1964.

[351] P. Moëssard, *Le cylindrographe, appareil panoramique*. Gauthier-Villars et fils, Paris, 1889.

[352] O. Morel, R. Seulin, and D. Fofi, "Catadioptric camera calibration by polarization imaging," in *Proceedings of the Iberian Conference on Pattern Recognition and Image Analysis, Girona, Spain*, pp. 396-403, 2007.

[353] T. Morita, Y. Yasukawa, Y. Inamoto, U. Takashi, and S. Kawakami, "Measurement in three dimensions by motion stereo and spherical mapping," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, California, USA*, pp. 422-428, 1989.

[354] E. Mouaddib, R. Sagawa, T. Echigo, and Y. Yagi, "Stereovision with a single camera and multiple mirrors," in *Proceedings of the IEEE International Conference on Robotics and Automation, Barcelona, Spain*, pp. 800-805, 2005.

[355] E. Mouaddib, R. Sagawa, T. Echigo, and Y. Yagi, "Two or more mirrors for the omnidirectional stereovision?," in *Proceedings of the 2nd IEEE-EURASIP International Symposium on Control, Communications, and Signal Processing, Marrakech, Morocco*, 2006.

[356] E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, "Real-time localization and 3D reconstruction," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, USA*, pp. 363-370, June 2006.

[357] E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, "Generic and real-time structure from motion," in *Proceedings of the 18th British Machine Vision Conference, Warwick, England*, 2007.

[358] E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, "Generic and real-time structure from motion using local bundle adjustment," *Image and Vision Computing*, vol. 27, no. 8, pp. 1178-1193, 2009.

[359] R. Munjy, "Calibrating non-metric cameras using the finite-element method," *Photogrammetric Engineering & Remote Sensing*, vol. 52, pp. 1201-1205, August 1986.

[360] R. Munjy, "Self-calibration using the finite element approach," *Photogrammetric Engineering & Remote Sensing*, vol. 52, pp. 411-418, March 1986.

[361] J. Murphy, "Application of panospheric imaging to a teleoperated lunar rover," in *Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Vancouver, Canada*, pp. 3117-3121, 1995.

[362] D. Murray, "Recovering range using virtual multi-camera stereo," *Computer Vision and Image Understanding*, vol. 61, no. 2, pp. 285-291, 1995.

[363] H. Nagahara, Y. Yagi, and M. Yachida, "Super wide field of view head mounted display using catadioptrical optics," *Presence*, vol. 15, no. 5, pp. 588-598, 2006.

[364] H. Nagahara, K. Yoshida, and M. Yachida, "An omnidirectional vision sensor with single view and constant resolution," in *Proceedings of the 11th IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil*, 2007.
[365] V. Nalwa, “A true omnidirectional viewer,” Technical Report Bell Laboratories Technical Memorandum, BL0115500-960115-01, AT&T Bell Laboratories, 1996.

[366] S. Nayar, “Sphereo: Determining depth using two specular spheres and a single camera,” in *Proceedings of the SPIE Conference on Optics, Illumination, and Image Sensing for Machine Vision III*, Cambridge, USA, pp. 245–254, November 1988.

[367] S. Nayar, “Catadioptric omnidirectional camera,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, Puerto Rico, USA, pp. 482–488, 1997.

[368] S. Nayar, “Omnidirectional vision,” in *Proceedings of the Eight International Symposium on Robotics Research*, Shonan, Japan, October 1997.

[369] S. Nayar, “Computational cameras: Redefining the image,” *Computer*, vol. 39, no. 8, pp. 30–38, 2006.

[370] S. Nayar, V. Branzoi, and T. Boult, “Programmable imaging: Towards a flexible camera,” *International Journal of Computer Vision*, vol. 70, no. 1, pp. 7–22, 2006.

[371] S. Nayar and A. Karmarkar, “360 × 360 mosaics,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, Hilton Head Island, South Carolina, USA, pp. 380–387, 2000.

[372] S. Nayar and V. Peri, “Folded catadioptric cameras,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, Fort Collins, Colorado, USA, pp. 217–223, 1999.

[373] R. Nelson and J. Aloimonos, “Finding motion parameters from spherical motion fields (or the advantages of having eyes in the back of your head),” *Biological Cybernetics*, vol. 58, no. 4, pp. 261–273, 1988.

[374] S. Nene and S. Nayar, “Stereo with mirrors,” in *Proceedings of the 6th IEEE International Conference on Computer Vision, Bombay, India*, pp. 1087–1094, January 1998.

[375] J. Neumann, C. Fermüller, and Y. Aloimonos, “Polydioptric camera design and 3D motion estimation,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, Madison, Wisconsin, USA, pp. 294–301, 2003.

[376] Y. Nishimoto and Y. Shirai, “A feature-based stereo model using small disparities,” in *Proceedings of the IEEE International Workshop on Industrial Applications of Machine Vision and Machine Intelligence*, Tokyo, Japan, pp. 192–196, 1987.

[377] D. Nistér, “An efficient solution to the five-point relative pose problem,” *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 26, pp. 756–770, June 2004.

[378] D. Nistér, “A minimal solution to the generalized 3-point pose problem,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, Washington, USA, pp. 560–567, 2004.

[379] D. Nistér and H. Stewénius, “A minimal solution to the generalised 3-point pose problem,” *Journal of Mathematical Imaging and Vision*, vol. 27, no. 1, pp. 67–79, 2007.
[380] D. Nistér, H. Stewénius, and E. Grossmann, "Non-parametric self-calibration," in *Proceedings of the 10th IEEE International Conference on Computer Vision, Beijing, China*, pp. 120-127, October 2005.

[381] Y. Nomura, M. Sagara, H. Naruse, and A. Ide, "Simple calibration algorithm for high-distortion-lens camera," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 14, pp. 1095-1099, November 1992.

[382] S. Oh and E. Hall, "Guidance of a mobile robot using an omnidirectional vision navigation system," in *Proceedings of SPIE, MOBILE ROBOTS II, CAMBRIDGE, USA*, pp. 288-300, 1987.

[383] S. Oh and E. Hall, "Calibration of an omnidirectional vision navigation system using an industrial robot," *Optical Engineering*, vol. 28, no. 9, pp. 955-962, 1989.

[384] T. Okatani and K. Deguchi, "On photometric aspects of catadioptric cameras," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA*, pp. 1106-1113, 2001.

[385] J. Oliensis, "Exact two-image structure from motion," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 24, no. 12, pp. 1618-1633, 2002.

[386] M. Ollis, H. Herman, and S. Singh, "Analysis and design of panoramic stereo vision using equi-angular pixel cameras," Technical Report CMU-RI-TR-99-04, Carnegie Mellon University, 1999.

[387] V. Orekhov, B. Abidi, C. Broaddus, and M. Abidi, "Universal camera calibration with automatic distortion model selection," in *Proceedings of the IEEE International Conference on Image Processing, San Antonio, USA*, pp. 397-400, 2007.

[388] R. Orghidan, J. Salvi, and E. Mouaddib, "Calibration of a structured light-based stereo catadioptric sensor," in *Proceedings of the Workshop on Omnidirectional Vision and Camera Networks, Madison, Wisconsin, USA*, 2003.

[389] T. Pajdla, "Geometry of two-slit camera," Technical Report CTU-CMP-2002-02, Center for Machine Perception, Czech Technical University, Prague, March 2002.

[390] T. Pajdla, "Stereo with oblique cameras," *International Journal of Computer Vision*, vol. 47, no. 1-3, pp. 161-170, 2002.

[391] T. Pajdla, T. Svoboda, and V. Hlavac, "Epipolar geometry of central panoramic cameras," in *Panoramic Vision: Sensors, Theory, and Applications*, (R. Benosman and S. Kang, eds.), pp. 85-114, Springer-Verlag, 2001.

[392] F. Pardo, B. Dierickx, and D. Scheffer, "CMOS foveated image sensor: Signal scaling and small geometry effects," *IEEE Transactions on Electron Devices*, vol. 44, no. 10, pp. 1731-1737, 1997.

[393] C. Pégard and E. Mouaddib, "A mobile robot using a panoramic view," in *Proceedings of the IEEE International Conference on Robotics and Automation*, pp. 89-94, April 1996.

[394] S. Peleg and M. Ben-Ezra, "Stereo panorama with a single camera," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado, USA*, pp. 1395-1401, 1999.

[395] S. Peleg, M. Ben-Ezra, and Y. Pritch, "Omnistereo: Panoramic stereo imaging," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 23, pp. 279-290, March 2001.
[396] S. Peleg, Y. Pritch, and M. Ben-Ezra, "Cameras for stereo panoramic imaging," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, Hilton Head Island, South Carolina, USA, pp. 208-214, 2000.

[397] S. Peleg, B. Rousso, A. Rav-Acha, and A. Zomet, "Mosaicing on adaptive manifolds," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, pp. 1144-1154, October 2000.

[398] M. Penna, "Camera calibration: A quick and easy way to determine the scale factor," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 13, pp. 1240-1245, December 1991.

[399] C. Perwass and G. Sommer, "The inversion camera model," in *Proceedings of the 28th DAGM Symposium, Berlin, Germany*, pp. 647-656, 2006.

[400] R. Petty, S. Godber, M. Robinson, and J. Evans, "3-D vision systems using rotating 1-D sensors," in *Proceedings of the IEE Colloquium on Application of Machine Vision*. London, UK, pp. 6/1-6/6, 1995.

[401] B. Peuchot and M. Saint-André, "CCD camera calibration virtual equivalent model," in *14th Annual International Conference IEEE EMBS, Paris*, pp. 1960-1961, October 1992.

[402] R. Pless, "Using many cameras as one," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Madison, Wisconsin, USA*, pp. 587-593, June 2003.

[403] G. Poivilliers, "Une chambre photographique quadruple de 20 millimètres de focale," in *Proceedings of the IV ISPRS-Congress, Paris, France*, pp. 132-134, 1934.

[404] M. Pollefeys, R. Koch, and L. van Gool, "A simple and efficient rectification method for general motion," in *Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Greece*, pp. 496-501, 1999.

[405] J. Ponce, "What is a camera?," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA*, 2009.

[406] V. Popescu, J. Dauble, C. Mei, and E. Sacks, "An efficient error-bounded general camera model," in *Proceedings of the Third International Symposium on 3D Data Processing, Visualization and Transmission, Thessaloniki, Greece*, 2006.

[407] I. Powell, "Panoramic lens," *Applied Optics*, vol. 33, no. 31, pp. 7356-7361, 1994.

[408] B. Prescott and G. McLean, "Line-based correction of radial lens distortion," *Graphical Models and Image Processing*, vol. 59, pp. 39-47, January 1997.

[409] C. Pulfrich, "Über die stereoskopische Betrachtung eines Gegenstandes und seines Spiegelbildes," *Zeitschrift für Instrumentenkunde*, vol. 25, no. 4, pp. 93-96, 1905.

[410] M. Qiu and S. Ma, "The nonparametric approach for camera calibration," in *Proceedings of the 5th IEEE International Conference on Computer Vision, Cambridge, Massachusetts, USA*, (E. Grimson, ed.), pp. 224-229, IEEE, IEEE Computer Society Press, June 1995.

[411] P. Rademacher and G. Bishop, "Multiple-center-of-projection images," in *Proceedings of the SIGGRAPH*, pp. 199-206, 1998.

[412] N. Ragot, "Conception d'un capteur de stéréovision omnidirectionnelle: Architecture, étalonnage et applications à la reconstruction de scènes 3D," PhD thesis, Université de Rouen, France, 2009.
[413] N. Ragot, J.-Y. Ertaud, X. Savatier, and B. Mazari, "Calibration of a panoramic stereovision sensor: Analytical vs interpolation-based methods," in *Proceedings of the 32nd Annual Conference of the IEEE Industrial Electronics Society, Paris, France*, pp. 4130-4135, 2006.

[414] S. Ramalingam, S. Lodha, and P. Sturm, "A generic structure-from-motion algorithm for cross-camera scenarios," in *Proceedings of the 5th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Prague, Czech Republic*, pp. 175-186, May 2004.

[415] S. Ramalingam, S. Lodha, and P. Sturm, "A generic structure-from-motion framework," *Computer Vision and Image Understanding*, vol. 103, pp. 218-228, September 2006.

[416] S. Ramalingam and P. Sturm, "Minimal solutions for generic imaging models," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, USA*, June 2008.

[417] S. Ramalingam, P. Sturm, and E. Boyer, "A factorization based self-calibration for radially symmetric cameras," in *Proceedings of the Third International Symposium on 3D Data Processing, Visualization and Transmission, Chapel Hill, USA*, June 2006.

[418] S. Ramalingam, P. Sturm, and S. Lodha, "Towards complete generic camera calibration," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA*, pp. 1093-1098, June 2005.

[419] S. Ramalingam, P. Sturm, and S. Lodha, "Towards generic self-calibration of central cameras," in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Beijing, China*, pp. 20-27, October 2005.

[420] S. Ramalingam, P. Sturm, and S. Lodha, "Theory and calibration algorithms for axial cameras," in *Proceedings of the Asian Conference on Computer Vision, Hyderabad, India*, pp. 704-713, January 2006.

[421] S. Ramalingam, P. Sturm, and S. Lodha, "Generic self-calibration of central cameras," *Computer Vision and Image Understanding*, vol. 114, no. 2, pp. 210-219, 2010.

[422] B. Ramsgaard, I. Balslev, and J. Arnspang, "Mirror-based trinocular systems in robot-vision," in *Proceedings of the 15th International Conference on Pattern Recognition, Barcelona, Spain*, pp. 499-502, 2000.

[423] O. Reading, "The nine lens air camera of the U.S. coast and geodetic survey," *Photogrammetric Engineering*, vol. IV, no. 3, pp. 184-192, 1938.

[424] D. Rees, "Panoramic television viewing system, U.S. patent no. 3,505,465," 1970.

[425] D. Rees, "Panoramic imaging block for three-dimensional space, U.S. patent no. 4,566,763," 1985.

[426] F. Remondino and C. Fraser, "Digital camera calibration methods: considerations and comparisons," in *ISPRS Commission V Symposium, Dresden, Germany*, pp. 266-272, 2006.

[427] S. Remy, M. Dhome, N. Daucher, and J. Lapresté, "Estimating the radial distortion of an optical system; effect on a localization process," in *Proceedings of the 12th International Conference on Pattern Recognition, Jerusalem, Israel*, pp. 997-1001, 1994.
[428] C. Ricolfe-Viala and A.-J. Sánchez-Salmerón, "Robust metric calibration of non-linear camera lens distortion," *Pattern Recognition*, vol. 43, no. 4, pp. 1688-1699, 2010.

[429] D. Roberts, "History of lenticular and related autostereoscopic methods, white paper," Leap Technologies, 2003.

[430] R. Roelofs, "Distortion, principal point, point of symmetry and calibrated principal point," *Photogrammetria*, vol. 7, pp. 49–66, 1951.

[431] S. Roy, J. Meunier, and I. Cox, "Cylindrical rectification to minimize epipolar distortion," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, USA*, pp. 393–399, 1997.

[432] R. Sagawa, N. Kurita, T. Echigo, and Y. Yagi, "Compound catadioptric stereo sensor for omnidirectional object detection," in *Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Sendai, Japan*, pp. 2612–2617, September 2004.

[433] R. Sagawa, M. Takatsuji, T. Echigo, and Y. Yagi, "Calibration of lens distortion by structured-light scanning," in *Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, Canada*, pp. 1349–1354, August 2005.

[434] K. Sarachik, "Visual navigation: Constructing and utilizing simple maps of an indoor environment," PhD thesis, Massachusetts Institute of Technology, published as technical report AITR-1113 of the MIT AI Lab, 1989.

[435] H. Sawhney and R. Kumar, "True multi-image alignment and its application to mosaicing and lens distortion correction," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 21, no. 3, pp. 235–243, 1999.

[436] D. Scaramuzza, A. Martinelli, and R. Siegwart, "A toolbox for easily calibrating omnidirectional cameras," in *Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China*, pp. 5695–5701, 2006.

[437] D. Scaramuzza and R. Siegwart, "A practical toolbox for calibrating omnidirectional cameras," in *Vision Systems: Applications*, (G. Obinata and A. Dutta, eds.), pp. 297–310, I-Tech Education and Publishing, Vienna, Austria, 2007.

[438] T. Scheimpflug, "Der Perspektograph und seine Anwendung," *Photographische Korrespondenz*, 1906.

[439] D. Schneider, E. Schwalbe, and H.-G. Maas, "Validation of geometric models for fisheye lenses," *ISPRS Journal of Photogrammetry and Remote Sensing*, vol. 64, no. 3, pp. 259–266, 2009.

[440] H. Schreier, D. Garcia, and M. Sutton, "Advances in light microscope stereo vision," *Experimental Mechanics*, vol. 44, pp. 278–288, June 2004.

[441] E. Schwartz, "Computational anatomy and functional architecture of striate cortex: A spatial mapping approach to perceptual coding," *Vision Research*, vol. 20, no. 8, pp. 645–669, 1980.

[442] G. Schweighofer and A. Pinz, "Globally optimal O(n) solution to the PnP problem for general camera models," in *Proceedings of the 19th British Machine Vision Conference, Leeds, England*, pp. 1–10, 2008.

[443] G. Schweighofer, S. Šegvić, and A. Pinz, "Online/realtime structure and motion for general camera models," in *Proceedings of the IEEE Workshop on Applications of Computer Vision, Copper Mountain, USA*, 2008.
[444] S. Seitz, A. Kalai, and H.-Y. Shum, “Omnivergent stereo,” *International Journal of Computer Vision*, vol. 48, no. 3, pp. 159–172, 2002.

[445] S. Seitz and J. Kim, “The space of all stereo images,” *International Journal of Computer Vision*, vol. 48, pp. 21–38, June 2002.

[446] O. Shakernia, R. Vidal, and S. Sastry, “Omnidirectional egomotion estimation from back-projection flow,” in *Proceedings of the Workshop on Omnidirectional Vision and Camera Networks, Madison, Wisconsin, USA*, 2003.

[447] O. Shakernia, R. Vidal, and S. Sastry, “Structure from small baseline motion with central panoramic cameras,” in *Proceedings of the Workshop on Omnidirectional Vision and Camera Networks, Madison, Wisconsin, USA*, 2003.

[448] S. Shih, Y. Hung, and W. Lin, “Accurate linear technique for camera calibration considering lens distortion by solving an eigenvalue problem,” *Optical Engineering*, vol. 32, pp. 138–149, January 1993.

[449] S. Shih, Y. Hung, and W. Lin, “When should we consider lens distortion in camera calibration,” *Pattern Recognition*, vol. 28, no. 3, pp. 447–461, 1995.

[450] J. Šivic and T. Pajdla, “Geometry of concentric multiperspective panoramas,” Technical Report CTU-CMP-2002-05, Center for Machine Perception, Czech Technical University, Prague, 2002.

[451] C. Slama, ed., *Manual of Photogrammetry*. Falls Church, Virginia, USA: American Society of Photogrammetry and Remote Sensing, 4th Edition, 1980.

[452] L. Smadja, R. Benosman, and J. Devars, “Cylindrical sensor calibration using lines,” in *Proceedings of the IEEE International Conference on Image Processing, Singapore*, pp. 1851–1854, 2004.

[453] P. Smith, K. Johnson, and M. Abidi, “Efficient techniques for wide-angle stereo vision using surface projection models,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado, USA*, pp. 113–118, 1999.

[454] W. Smith, N. Vakil, and S. Maislin, “Correction of distortion in endoscope images,” *IEEE Transactions on Medical Imaging*, vol. 11, pp. 117–122, March 1992.

[455] D. Southwell, A. Basu, M. Fiala, and J. Reyda, “Panoramic stereo,” in *Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria*, pp. 378–382, 1996.

[456] L. Spacek, “Coaxial omnidirectional stereopsis,” in *Proceedings of the 8th European Conference on Computer Vision, Prague, Czech Republic*, pp. 354–365, Lecture Notes in Computer Science, 2004.

[457] M. Srinivasan, “New class of mirrors for wide-angle imaging,” in *Proceedings of the Workshop on Omnidirectional Vision and Camera Networks, Madison, Wisconsin, USA*, 2003.

[458] R. Steele and C. Jaynes, “Overconstrained linear estimation of radial distortion and multi-view geometry,” in *Proceedings of the 9th European Conference on Computer Vision, Graz, Austria*, pp. 253–264, 2006.

[459] T. Stehle, D. Truhn, T. Aach, C. Trautwein, and J. Tischendorf, “Camera calibration for fish-eye lenses in endoscopy with an application to 3D reconstruction,” in *Proceedings IEEE International Symposium on Biomedical Imaging, Washington, D.C.*, pp. 1176–1179, 2007.
[460] G. Stein, “Accurate internal camera calibration using rotation, with analysis of sources of error,” in *Proceedings of the 5th IEEE International Conference on Computer Vision, Cambridge, Massachusetts, USA*, (E. Grimson, ed.), pp. 230–236, IEEE Computer Society Press, June 1995.

[461] G. Stein, “Lens distortion calibration using point correspondences,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, USA*, pp. 602–608, 1997.

[462] D. Stevenson and M. Fleck, “Robot aerobics: Four easy steps to a more flexible calibration,” in *Proceedings of the 5th IEEE International Conference on Computer Vision, Cambridge, Massachusetts, USA*, pp. 34–39, 1995.

[463] D. Stevenson and M. Fleck, “Nonparametric correction of distortion,” in *Proceedings of the IEEE Workshop on Applications of Computer Vision, Sarasota, Florida*, pp. 214–219, 1996.

[464] H. Stewénius, D. Nistér, M. Oskarsson, and K. Åström, “Solutions to minimal generalized relative pose problems,” in *Proceedings of the 6th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Beijing, China*, 2005.

[465] H. Stewénius, F. Schaffalitzky, and D. Nistér, “How hard is three-view triangulation really?,” in *Proceedings of the 10th IEEE International Conference on Computer Vision, Beijing, China*, pp. 686–693, 2005.

[466] R. Strand and E. Hayman, “Correcting radial distortion by circle fitting,” in *Proceedings of the 16th British Machine Vision Conference, Oxford, England*, 2005.

[467] D. Strelow, J. Mishler, D. Koes, and S. Singh, “Precise omnidirectional camera calibration,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA*, pp. 689–694, 2001.

[468] D. Strelow and S. Singh, “Reckless motion estimation from omnidirectional image and inertial measurements,” in *Proceedings of the Workshop on Omnidirectional Vision and Camera Networks, Madison, Wisconsin, USA*, 2003.

[469] P. Sturm, “Self-calibration of a moving zoom-lens camera by pre-calibration,” *Image and Vision Computing*, vol. 15, pp. 583–589, August 1997.

[470] P. Sturm, “A method for 3D reconstruction of piecewise planar objects from single panoramic images,” in *Proceedings of the IEEE Workshop on Omnidirectional Vision, Hilton Head Island, South Carolina*, pp. 119–126, IEEE, June 2000.

[471] P. Sturm, “Mixing catadioptric and perspective cameras,” in *Proceedings of the Workshop on Omnidirectional Vision, Copenhagen, Denmark*, pp. 37–44, June 2002.

[472] P. Sturm, “Multi-view geometry for general camera models,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA*, pp. 206–212, June 2005.

[473] P. Sturm and J. Barreto, “General imaging geometry for central catadioptric cameras,” in *Proceedings of the 10th European Conference on Computer Vision, Marseille, France*, (D. Forsyth, P. Torr, and A. Zisserman, eds.), pp. 609–622, October 2008.
[474] P. Sturm, Z. Cheng, P. Chen, and A. Poo, "Focal length calibration from two views: Method and analysis of singular cases," *Computer Vision and Image Understanding*, vol. 99, pp. 58-95, July 2005.

[475] P. Sturm and S. Maybank, "On plane-based camera calibration: A general algorithm, singularities, applications," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado, USA*, pp. 432-437, June 1999.

[476] P. Sturm and L. Quan, "Affine stereo calibration," Technical Report LIFIA-29, LIFIA-IMAG, Grenoble, France, June 1995.

[477] P. Sturm and S. Ramalingam, "A generic concept for camera calibration," in *Proceedings of the 8th European Conference on Computer Vision, Prague, Czech Republic*, (T. Pajdla and J. Matas, eds.), pp. 1-13, Springer-Verlag, May 2004.

[478] P. Sturm, S. Ramalingam, and S. Lodha, "On calibration, structure from motion and multi-view geometry for generic camera models," in *Imaging Beyond the Pinhole Camera*, (K. Daniilidis and R. Klette, eds.), Springer-Verlag, August 2006.

[479] T. Svoboda, "Central Panoramic Cameras: Design, Geometry, Egomotion," PhD thesis, Faculty of Electrical Engineering, Czech Technical University, Prague, September 1999.

[480] T. Svoboda, D. Martinec, and T. Pajdla, "A convenient multicamera self-calibration for virtual environments," *Presence*, vol. 14, no. 4, pp. 407-422, 2005.

[481] T. Svoboda and T. Pajdla, "Epipolar geometry for central catadioptric cameras," *International Journal of Computer Vision*, vol. 49, no. 1, pp. 23-37, 2002.

[482] T. Svoboda, T. Pajdla, and V. Hlaváč, "Epipolar geometry for panoramic cameras," in *Proceedings of the 5th European Conference on Computer Vision, Freiburg, Germany*, pp. 218-231, 1998.

[483] R. Swaminathan, M. Grossberg, and S. Nayar, "A perspective on distortions," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Madison, Wisconsin, USA*, pp. 594-601, June 2003.

[484] R. Swaminathan, M. Grossberg, and S. Nayar, "Non-single viewpoint catadioptric cameras: Geometry and analysis," *International Journal of Computer Vision*, vol. 66, no. 3, pp. 211-229, 2006.

[485] R. Swaminathan and S. Nayar, "Nonmetric calibration of wide-angle lenses and polycameras," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no. 10, pp. 1172-1178, 2000.

[486] R. Swaminathan, S. Nayar, and M. Grossberg, "Designing mirrors for catadioptric systems that minimize image errors," in *Proceedings of the 5th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Prague, Czech Republic*, 2004.

[487] R. Szeliski, "Image alignment and stitching: A tutorial," *Foundations and Trends in Computer Graphics and Vision*, vol. 2, pp. 1-104, December 2006.

[488] A. Takeya, T. Kuroda, K.-I. Nishiguchi, and A. Ichikawa, "Omnidirectional vision system using two mirrors," in *Proceedings of the SPIE, Novel Optical Systems and Large-Aperture Imaging*, pp. 50-60, 1998.
[489] J.-P. Tardif, P. Sturm, and S. Roy, "Self-calibration of a general radially symmetric distortion model," in *Proceedings of the 9th European Conference on Computer Vision, Graz, Austria*, (H. Bischof and A. Leonardis, eds.), pp. 186-199, May 2006.

[490] J.-P. Tardif, P. Sturm, and S. Roy, "Plane-based self-calibration of radial distortion," in *Proceedings of the 11th IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil*, IEEE Computer Society Press, October 2007.

[491] J.-P. Tardif, P. Sturm, M. Trudeau, and S. Roy, "Calibration of cameras with radially symmetric distortion," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 31, no. 9, pp. 1552-1566, 2009.

[492] W. Tayman, "Analytical multicollimator camera calibration," *Photogrammetry*, vol. 34, pp. 179-197, 1978.

[493] S. Teller, "Toward urban model acquisition from geo-located images," in *Proceedings of the Pacific Conference on Computer Graphics and Applications, Singapore*, pp. 45-51, 1998.

[494] S. Teller and M. Hohmeyer, "Determining the lines through four lines," *Journal of Graphics Tools*, vol. 4, no. 3, pp. 11-22, 1999.

[495] L. Teodosio and W. Bender, "Salient video stills: Content and context preserved," in *Proceedings of the first ACM International Conference on Multimedia, Anaheim, USA*, pp. 39-46, 1993.

[496] L. Teodosio and M. Mills, "Panoramic overviews for navigating real-world scenes," in *Proceedings of the first ACM International Conference on Multimedia, Anaheim, USA*, pp. 359-364, 1993.

[497] W. Teoh and X. Zhang, "An inexpensive stereoscopic vision system for robots," in *Proceedings of the IEEE International Conference on Robotics and Automation, Atlanta, Georgia, USA*, pp. 186-189, 1984.

[498] H. Teramoto and G. Xu, "Camera calibration by a single image of balls: From conics to the absolute conic," in *Proceedings of the Fifth Asian Conference on Computer Vision, Melbourne, Australia*, pp. 499-506, 2002.

[499] R. Thiele, "Métrophotographie aérienne à l'aide de mon auto-panoramographe," *International Archives of Photogrammetry*, vol. 1, no. 1, pp. 35-45, 1908.

[500] S. Thirthala and M. Pollefeys, "Multi-view geometry of 1D radial cameras and its application to omnidirectional camera calibration," in *Proceedings of the 10th IEEE International Conference on Computer Vision, Beijing, China*, pp. 1539-1546, October 2005.

[501] S. Thirthala and M. Pollefeys, "The radial trifocal tensor: A tool for calibrating the radial distortion of wide-angle cameras," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA*, pp. 321-328, 2005.

[502] E. Thompson, "The seven lens air survey camera," *Photogrammetric Engineering*, vol. IV, no. 3, pp. 137-145, 1938.

[503] T. Thormählen, H. Broszio, and I. Wassermann, "Robust line-based calibration of lens distortion from a single view," in *Proceedings of the MIRAGE Conference on Computer Vision/Computer Graphics Collaboration*
for Model-based Imaging, Rendering, Image Analysis and Graphical Special Effects, Rocquencourt, France, pp. 105-112, 2003.

[504] G. Tissandier, La photographie en ballon. Gauthier-Villars, 1886.

[505] M. Tistarelli and G. Sandini, "On the advantage of polar and log-polar mapping for direct estimation of time-to-impact from optical flow," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 15, no. 4, pp. 401-410, 1993.

[506] C. Toepfer and T. Ehlgen, "A unifying omnidirectional camera model and its applications," in *Proceedings of the 7th Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, Rio de Janeiro, Brazil*, 2007.

[507] A. Tolvanen, C. Perwass, and G. Sommer, "Projective model for central cataractic cameras using Clifford algebra," in *Proceedings of the 27th DAGM Symposium, Vienna, Austria*, pp. 192-199, 2005.

[508] C. Tomasi and T. Kanade, "Shape and motion from image streams under orthography: A factorization method," *International Journal of Computer Vision*, vol. 9, pp. 137-154, November 1992.

[509] A. Torii, A. Sugimoto, and A. Imiya, "Mathematics of a multiple omnidirectional system," in *Proceedings of the Workshop on Omnidirectional Vision and Camera Networks, Madison, Wisconsin, USA*, 2003.

[510] J. Torres and J. Menéndez, "A practical algorithm to correct geometrical distortion of image acquisition cameras," in *Proceedings of the IEEE International Conference on Image Processing, Singapore*, pp. 2451-2454, 2004.

[511] B. Triggs, "Matching constraints and the joint image," in *Proceedings of the 5th IEEE International Conference on Computer Vision, Cambridge, Massachusetts, USA*, (E. Grimson, ed.), pp. 338-343, IEEE, Computer Society Press, June 1995.

[512] B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon, "Bundle adjustment — a modern synthesis," in *Proceedings of the International Workshop on Vision Algorithms: Theory and Practice, Corfu, Greece*, (B. Triggs, A. Zisserman, and R. Szeliski, eds.), pp. 298-372, Springer-Verlag, 2000.

[513] E. Trucco and A. Verri, *Introductory Techniques for 3-D Computer Vision*. Prentice-Hall, 1998.

[514] R. Tsai, "An efficient and accurate camera calibration technique for 3D machine vision," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, Florida, USA*, pp. 364-374, 1986.

[515] R. Tsai, "A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses," *IEEE Journal of Robotics and Automation*, vol. 3, pp. 323-344, August 1987.

[516] V. Vaish, B. Wilburn, N. Joshi, and M. Levoy, "Using plane + parallax for calibrating dense camera arrays," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Washington, USA*, pp. 2-9, 2004.

[517] B. Vandeportaele, "Contributions à la vision omnidirectionnelle: Étude, Conception et Étalonnage de capteurs pour l'acquisition d'images et la modélisation 3D," PhD thesis, Institut National Polytechnique de Toulouse, France, in french, December 2006.