Anamorphic Stereoscopic Imaging
Exploiting nonlinear projections for wide-angle depth generation
Leonid Keselman
Stanford UniversityMotivation

New Technique

Traditional lenses used for computational depth generation are known
as perspective projection, or rectilinear lenses. Their projected images
preserve straight lines, and preserve constant space sampling acrossthe image. However, they present at least three main challenges.
•••

Natural vignetting – due to the geometry of perspective projectionlenses [AHA01], they will tend to exhibit light falloff as a function ofcos(θ) to the fourth power. This makes wide angle lenses
impractical, as even at 60 degrees off-axis, a rectilinear lens gather
only 6.25% the same light intensity as on axis.
Infinite Sampling – due to the requirement to have constantsampling in real-world space, rectilinear lenses require tan(θ) pixels
to capture areas θ degrees off axis. This explodes towards infinity asθ approaches 90 degrees. But even at smaller angles, rectilinear
lenses begin to devote a dispropriate amount of their pixel area tothe edges of their frame.
Stereo Accuracy -- While the first two points are applicable toalmost all problems in imaging, there ‘s a special case of interest forthis work – stereoscopic depth imaging. Traditional stereoscopic
imaging performs significantly worse at longer distances than close
ones. This is due to the geometry of the lens, and by changing to adifferent projection model, it may be possible to change theseaccuracy properties.

BA rectilinear lens projection

Two dimensional overview of ourcoordinate system. The field-ofview of each camera is identicaland equal to θ. The spacing
between the cameras is B.

First, many authors [AF05]. [GNT98] [Geh05] use wide angle lenses toperform stereo matching, but they distort the image back to a rectilinear
projection, forcing large scale resampling of the image and abandoning thebeneficial properties of using a nonlinear lens, such as the ability to image
an entire hemisphere.

To parametrize the equations of accuracy and correspondence, we cansetup a general equation to define stereosopic correspondence between
two imagers as follows
� � = � � − � + �.
Where p(x) is the projection function from the image, and d is the disparity

Repeating the previous derivations for additional lensgeometries gives us the following table�

Combining the two gives us�⋅��=���And it becomes clear that rectilinear lenses will suffer from quadratic
depth error given constant disparity space errors.

Results using a standard stereo dataset

Since a cubic lens requires very dense pixel sampling (1% off axis pixel resolution
would require 3x^2 sampling, or 30x super-sampling), we were having difficulty finding
sufficiently dense textured scenes, so we resampling a high-resolution stereo dataset
Rectilinear

X-axis Anamorphic[Fle95] Margaret M Fleck. “Perspective projection: the wrong imaging model”. In: Research report Kingslake 1992 (1995), pp. 95–01.
[GNT98] J M Gluckman, S K Nayar, and K J Thoresz. “Real-Time Omnidirectional and Panoramic Stereo”. In: DARPA Image Understanding Workshop (IUW).
[AHA01] Manoj Aggarwal, Hong Hua, and Narendra Ahuja. “On cosine-fourth and vignetting effects in real lenses”. In: Computer Vision, 2001. ICCV 2001.
[Sha02] Shishir Shah. “Depth Estimation Using Stereo Fish-Eye Lenses”. In: Image Processing, 1994. Proceedings. Vol. 2. IEEE. 2002, pp. 740–744. ISBN: 0818669500.
[AF05] Steffen Abraham and Wolfgang Forstner. “Fish-eye-stereo calibration and ¨epipolar rectification”. In: ISPRS J Photogramm 59.5 (2005),[Geh05] Sk Gehrig. “Large-field-of-view stereo for automotive applications”. In: Omnivis 2005 (2005)[Li06] Shigang Li. “Real-Time Spherical Stereo”. In: 18th International Conference on Pattern Recognition (ICPR’06). Oct. 2006,[Li08] Shigang Li. “Binocular Spherical Stereo”. In: TITS 9.4 (Dec. 2008), pp. 589–600. ISSN: 1524-9050. DOI: 10.1109/TITS.2008.2006736.
[Kit11] Nobuyuki Kita. “Dense 3D Measurement of the Near Surroundings byFisheye Stereo”. In: 2011, pp. 148–151.

��∝��� ,� ⋅�Y-axis Anamorphic

DepthGround
Truth

Results using a rendered scene

To demonstrate the value of using an anamorphic, non-linear lens, we generated
synthetic images with realistic vignetting and the appropriate lens projections. We thenran a simple stereo matching algorithm on both images. Unfortunately, the lowresolution and noisy path-traced input generates noisy depth maps for both test cases.
Experiments are on-going to use this as a basis for quantitative comparison.
Anamorphic sin(θ) lensImage��≫������ � ≫���

That is, for the center of the image, cubic lenses will tend tohave sub-quadraic depth error. For the rest of the image, theywill tend to have traditional stereo error that gets quadraticallyworse with as one samples off-axis.

Experimental Results

Lastly, Li [Li06] [Li08] developed methods of using nonlinear lens forgenerating depth, including calibration and search, but the method onlyapplied to only spherical projections. Additionally, their use of nonlinearities
was limited to only the axis perpendicular to the epipolar line.
Popular in the cinematic domain for mapping wide aspect ratio images onto
a square segment of film, anamoprhic lenses distort only one axis of theimage. For stereoscopic imaging, projection along the axis perpendicular to
the epipoles has no impact on stereo depth error

Of particular interest is the cubic depth error, which can berewritten as

Additionally, we can take the derivative, and substitute into our��
expression to generate an error relationship. Traditionally we take theerror relative to a single pixel disparity error (�� = 1).
���=−���⋅���

An f- θ lens projection

There have been many existing methods on expanding stereoscopic
correspondence algorithms to handle non ideal lenses. Unfortunately theytend to fall in two buckets.

Stereoscopic constraints for other lenses

A traditional rectilinear lens model would be�� � = � ⋅ tan � = � ∗Error with respect to distance fordifferent lens projections at u = 1%
of focal length

Related Work

Derivation of stereoscopic system constraints

Rectilinear lens