Multifocus synthesis and its application to 3D image capturing

ABSTRACTA new technique for the high-resolution image synthesis called Multifocus synthesis is presented. As an impor-tant extension, the 3-D real-world image capture is discussed. In the approach, the object image is taken at a numberof focal distances by a single camera placed at a fixed position. Each of these images are then converted into the mul-tiresolution using the optimized QMP. The resultant volume of coefficients are then analyzed and 3-D distance infor-mation is computed.2. HOLOGRAPHY AND 3-D DISTANCE INFORMATIONHolography is widely accepted as the ideal means for the 3-D image presentation. However, direct capturing ofthe optical information requires the spatial resolution ofless than 1 jim, dark indoor studio, high-power laser, nomotion, expensive film and incompatibility with the conventional video camera system. Various separate approachesare being studied including the synthetic holography that uses images captured by a multitude of conventional videocameras"2.For practical implementation of the 3-D display system, computation of the 3-D distance is of fundamentalimportance. But as is known, the reconstruction of 3-D information from 2-D image alone is an ill-posed problem anda special arrangement has to be made.A large number of works have and are been made in two approaches: passive and active measurement3'4. Thepassive measurement includes the auto-focusing and the binocular parallax image capture with correspondence fea-ture matching'6. The active measurement includes the optical/radar range finders, the slit-beam projection, themoire-topography and other coded light pattern projection, which are all generally suitable for the distance measure-ment itself rather than for the object image capturing, and the measurement usually has to be made in the indoor andclosed environments. There is another approach involving the spatial-temporal image processing, for example, theanalysis of images recorded by a video camera moving in a constant speed in one direction4. This is an extension ofthe passive approach.Passive computation of the 3-D distance from two binocular parallax (stereoscopic) images has been also wellstudied in the 3-D image analysis and understandthg". Conventional techniques first identifies the correspondingfeatures between the left and right images by the optical flow or the pattern-matching of the discrete features such aspoints, lines and contours using various techniques including dynamic programming6, correlation matching and neu-ral network12. The parallax image capture and processing approach has however a fundamental difficulty concerningthe discontinuous line-of-sight at the object edges and contours due to the parallax camera positioning (occlusion).When the neural network is applied to the feature matching, the distance perception mechanism can be modelled bylayers of cooperative operators and a mutually exclusive (inhibitive) operator working on all the cooperative outputs.In the neural network solution, an incorrect distance computation tends to propagate to the neighboring cells, result-ing in a larger number of inaccurate estimates. Some of the recent studies are being made on the a priori training ofthe shapes of the 3-D objects and isolated point removal5, but as long as the binocular parallax approach is taken,there is no fundamental solution to the problem.This paper discusses a new approach to the 3-D distance computation using images captured by a single cameraplaced at a fixed position. This avoids the problem inherent with the bthocular parallax approach and uses a com-pletely different set of tools compared with the correspondence feature matching.In the following, the fundamental image processing tool called Multifocus Synthesis (MFS) is first described.MFS was originally proposed for the synthesis of the higher resolution image using several focused images'33. MTJLTIFOCUS SYNTHESISIn Optics, a perfect pin-hole is a mathematical concept of an ideal lens and everyday images captured through acamera are the result of pre-processing by the lens. Every lens has a (adjustable) focusing distance and the informa-