Abstract-The range of scene depths that appear focused in an image is known as the depth of field (DOF). Conventional cameras are limited by a fundamental trade-off between depth of field and signal-to-noise ratio (SNR). For a dark scene, the aperture of the lens must be opened up to maintain SNR, which causes the DOF to reduce. Also, today's cameras have DOFs that correspond to a single slab that is perpendicular to the optical axis. In this paper, we present an imaging system that enables one to control the DOF in new and powerful ways. Our approach is to vary the position and/or orientation of the image detector during the integration time of a single photograph. Even when the detector motion is very small (tens of microns), a large range of scene depths (several meters) is captured, both in and out of focus. Our prototype camera uses a micro-actuator to translate the detector along the optical axis during image integration. Using this device, we demonstrate four applications of flexible DOF. First, we describe extended DOF where a large depth range is captured with a very wide aperture (low noise) but with nearly depth-independent defocus blur. Deconvolving a captured image with a single blur kernel gives an image with extended DOF and high SNR. Next, we show the capture of images with discontinuous DOFs. For instance, near and far objects can be imaged with sharpness, while objects in between are severely blurred. Third, we show that our camera can capture images with tilted DOFs (Scheimpflug imaging) without tilting the image detector. Finally, we demonstrate how our camera can be used to realize nonplanar DOFs. We believe flexible DOF imaging can open a new creative dimension in photography and lead to new capabilities in scientific imaging, vision, and graphics. Index Terms-Imaging geometry, programmable depth of field, detector motion, depth-independent defocus blur. Ç T HE depth of field (DOF) of an imaging system is the range of scene depths that appear focused in an image. In virtually all applications of imaging, ranging from consumer photography to optical microscopy, it is desirable to control the DOF. Of particular interest is the ability to capture scenes with very large DOFs. DOF can be increased by making the aperture smaller. However, this reduces the amount of light received by the detector, resulting in greater image noise (lower SNR). This trade-off gets worse with increase in spatial resolution (decrease in pixel size). As pixels get smaller, DOF decreases since the defocus blur occupies a greater number of pixels. At the same time, each pixel receives less light and, hence, SNR falls as well. This trade-off between DOF and SNR is one of the fundamental, long-standing limitations of imaging. In a conventional camera, for any location of the image detector, there is one scene plane-the focal plane-that is perfectly focused. In this paper, we propose varying the position and/or orientation of the image detector during the integration time of a photograph. As a result, the focal plane is swept through a volume of the scene, causing all points within it to come into and go out of focus while the detector collects photons. We demonstrate that such an imaging system enables one to control the DOF in new and powerful ways: . Extended Depth of Field: Consider the case where a detector with a global shutter (all pixels are exposed simultaneously and for the same duration) is moved with uniform speed during image integration. Then, each scene point is captured under a continuous range of focus settings, including perfect focus. We analyze the resulting defocus blur kernel and show that it is nearly constant over the range of depths that the focal plane sweeps through during detector motion. Consequently, the captured image can be deconvolved with a single, known blur kernel to recover an image with significantly greater DOF without having to know or determine scene geometry. This approach is similar in spirit to Hausler's work in microscopy [1] . He showed that the DOF of an optical microscope can be enhanced by moving a specimen of depth range d, a distance 2d along the optical axis of the microscope, while filming the specimen. The defocus of the resulting captured image is similar over the entire depth range of the specimen. However, this approach of moving the scene with respect to the imaging system is practical only in microscopy and is not suitable for general scenes. More importantly, Hausler's derivation assumes that defocus blur varies linearly with scene depth which is true only for imaging systems that are telecentric on the object side, such as microscopes. . Discontinuous Depth of Field: A conventional camera's DOF is a single fronto-parallel slab located around the focal plane. We show that by moving a global-shutter detector nonuniformly, we can capture 58
Whereas sensor motion orthogonal to the optical axis can be used to capture an image with invariant blur over different velocities, Nagahara et al. REF have shown that sensor motion along the optical axis can be used to capture images with defocus blur that is invariant to the distance between the camera and objects in the scene.
Flexible Depth of Field Photography
IEEE Transactions on Pattern Analysis and Machine Intelligence
IEEE Transactions on Pattern Analysis and Machine Intelligence
Computer Science Medicine