The present invention provides a three-dimensional camcorder comprising a three-dimensional imaging system acquiring the three-dimensional image information of an object and a three-dimensional viewfinder displaying three-dimensional image using at least one variable focal length micromirror array lens.

FIELD OF INVENTION

The present invention relates generally to camcorder systems, and in particular, to a three-dimensional camcorder system.

BACKGROUND OF THE INVENTION

Camcorders (video camera recorders) receive video information and convert it to electronic video signal, which is recorded on the storage medium in analog or digital formats. While most conventional camcorders record and display two-dimensional video images, some camcorders are capable of recording particular information required for stereoscopic viewing. When human eyes see a scene, right and left side eyes have two different perspectives due to their separation. The brain fuses these two images and assesses the visual depth. Three-dimensional camcorders need to provide two perspective images for each video frame. In order to provide different viewing angles simultaneously, they usually use two cameras which are separated at a certain distance as human eyes are.FIG. 1depicts the schematic illustration of a prior art using a pinhole or thin lens model. Cameras11and12have identical focal length and their image planes are coplanar in space. The distance between optical centers13and14is 2d. Two cameras have different perspectives for an object15and provide a pair of stereoscopic images, which allows three-dimensional viewing. However, these devices tend to be large and heavy, and come at huge cost due to multiple camera systems and their optical axis separation requirement.

Conventional three-dimensional display devices use two perspective images of a scene and provide methods to let each eye see an only intended perspective. While conventional three-dimensional display devices require special glasses for a viewer to see a three-dimensional image from two perspective images, autostereoscopic display devices do not require special glasses. Autostereoscopic devices display two different perspective images concurrently and use barriers to let each eye see the intended image as disclosed by U.S. Pat. No. 6,046,849 to Moseley. Usually, these devices have several viewing areas and the scene appears to leap out of the screen, which provides a virtual-reality environment. These autostereoscopic display devices can be built in the three-dimensional camcorders as electronic viewfinders like normal LCD screens in two-dimensional camcorders in order to find and focus views in the three-dimensional space and replay recorded three-dimensional images or movies.

SUMMARY OF THE INVENTION

The present invention provides a three-dimensional camcorder comprising at least one variable focal length MicroMirror Array Lens (MMAL) having a plurality of micromirrors for three-dimensional imaging, recording, and displaying.

An objective of the invention is to provide a three-dimensional camcorder that provides depthwise images and the depth information for each depthwise image, an all-in-focus image and the depth information for each pixel of the all-in-focus image, or a pair of stereoscopic images using a single camera system. The three-dimensional camcorder comprises at least one variable focal length MicroMirror Array Lens (MMAL), an imaging unit, and an image processing unit for three-dimensional imaging and recording, and a three-dimensional viewfinder for displaying three-dimensional images.

The variable focal length MMAL comprises a plurality of micromirrors, wherein each of the micromirrors in the MMAL is controlled to change the focal length of the MMAL. Each micromirror has the same function as a mirror. Micromirrors in the MMAL are arranged in a shape depending on the geometry of the imaging system on a substantially flat plane. The MMAL works as a reflective focusing lens by making all light scattered from one point of an object have the same periodical phase and converge at one point on the image plane. Each micromirror in the MMAL is controlled to have desired translation and rotation to satisfy the convergence and phase matching conditions for forming an image of the object, wherein each micromirror of the MMAL is actuated by the electrostatic and/or the electromagnetic force. The focal length and the optical axis of the MMAL are changed by controlling the rotation and translation of each micromirror in the MMAL. Also, the aberrations of the three-dimensional camcorder are corrected by controlling the rotation and translation of each micromirror in the MMAL.

The variable focal length MMAL changes its surface profile to change its focal length by controlling the rotation and translation of each micromirror. The focal length of the variable focal length MMAL is changed in a plurality of steps in order to scan the whole object (or scene).

The imaging unit captures images formed on the image plane by the variable focal length MMAL. As the focal length of the variable focal length MMAL is changed, the in-focus regions of the object are also changed accordingly.

The image processing unit produces three-dimensional image data using the images captured by the imaging unit and the focal length information of the MMAL. The image processing unit extracts the substantially in-focus pixels of each captured image received from the imaging unit and generates a corresponding depthwise image using the extracted in-focus pixels of each captured image. Depth information, or the distance between the imaging system and the in-focus region of the object is determined by known imaging system parameters including the focal length and the distance between the MMAL and the image plane. A set of depthwise images taken at different focal lengths with a fast imaging rate represents the object or scene at a given moment. The image processing unit can combine these depthwise images to make an all-in-focus image. There are several methods for the image processing unit to obtain depthwise images or an all-in-focus image (e.g. edge detection filter). Recent advances in both the image sensor and the image processing unit make them as fast as they are required to be. The pair of stereoscopic images is generated by the image processing unit using all-in-focus image and the depth information for each pixel of the all-in-focus image, where the three-dimensional information of the object is projected on two virtual image planes whose centers are separated at certain distance, which simulates two-camera systems having different viewing angles. The three-dimensional camcorder may further comprise a storage medium storing three-dimensional image data. Depending on the display method of the three-dimensional display system, the image processing unit generates a set of depthwise images representing the object at a given moment and the depth information for each depthwise image using the images captured by the imaging unit, an all-in-focus image and the depth information for each pixel of the all-in-focus image using the images captured by the imaging unit, or a pair of stereoscopic images using an all-in-focus image and the depth information for each pixel of the all-in-focus image. These image data are transferred to the three-dimensional viewfinder to find and focus the object or recorded on the storage medium. Image sensing and image processing time is faster than the persistent rate of human eyes to have real-time three-dimensional images.

Another objective of the invention is to provide a three-dimensional camcorder, further comprising additional MMAL or MMALs for providing zoom function, wherein the MMALs magnify the image and keep the image in-focus by changing the focal length of each MMAL without macroscopic mechanical movements of lenses.

Still another objective of the invention is to provide a three-dimensional camcorder having auto focusing function by changing the focal length of MMAL, wherein the focal length of MMAL for auto focusing is determined by the depth information. The auto focusing systems using the MMAL are described in U.S. patent application Ser. No. 10/896,146, the contents of which are hereby incorporated by reference.

Still another objective of the invention is to provide a three-dimensional camcorder having vibration correction capability, wherein the three-dimensional camcorder further comprises a vibration determination device, communicatively coupled to the MMAL, configured to measure vibration of the three-dimensional camcorder and to generate a vibration correction signal. The MMAL is adjusted to change its optical axis by controlling the rotation and translation of each micromirror in the MMAL, based at least in part on the vibration correction signal to correct for the vibration of the three-dimensional camcorder. The vibration correction system using the MMAL is described in U.S. patent application Ser. No. 10/979,612, the contents of which are hereby incorporated by reference.

Still another objective of the invention is to provide a three-dimensional camcorder imaging an object which does not lie on the nominal optical axis by using at least one MMAL without macroscopic mechanical movement of the imaging system.

Still another objective of the invention is to provide a three-dimensional camcorder compensating for aberrations using at least one MMAL. Since the MMAL is an adaptive optical component, the MMAL compensates for phase errors of light introduced by the medium between an object and its image and/or corrects the defects of the three-dimensional camcorder that may cause the image to deviate from the rules of paraxial imagery by controlling individual micromirrors in the MMAL.

The variable focal length MMAL is reflective lens. The three-dimensional camcorder may further comprise a beam splitter positioned in the path of light between the imaging unit and the MMAL to have normal incident optical geometry onto the MMAL. Alternatively, in order to deflect the light into a sensor, the MMAL is tilted in the imaging system of the camcorder so that the normal direction of the MMAL is different from the optical axis of the imaging system. When the MMAL is tilted about an axis (tilting axis), which is perpendicular to the optical axis of the imaging system, the surface profile of the MMAL is symmetric about an axis which is perpendicular to the optical axis and tilting axis. The tilted MMAL can cause non axis-symmetric aberrations. To have the desired focal length and compensate for non axis-symmetric aberrations, each micromirror has one translational motion along the normal axis to the plane of each MMAL and two rotational motions about two axes in the plane of each MMAL.

In order to obtain a color image, the MMAL is controlled to compensate for chromatic aberration by satisfying the phase matching condition for each wavelength of Red, Green, and Blue (RGB), or Yellow, Cyan, and Magenta (YCM), respectively. The three-dimensional camcorder may further comprise a plurality of bandpass filters for color imaging. Also, the three-dimensional camcorder may further comprise a photoelectric sensor. The photoelectric sensor comprises Red, Green, and Blue sensors or Yellow, Cyan, and Magenta sensors, wherein color images are obtained by the treatments of the electrical signals from each sensor. The treatment of electrical signal from sensor is synchronized and/or matched with the control of the MMAL to satisfy the phase matching condition for each wavelength of Red, Green, and Blue or Yellow, Cyan, and Magenta, respectively.

Furthermore, the MMAL can be controlled to satisfy phase matching condition at an optimal wavelength to minimize chromatic aberration, wherein optimal wavelength phase matching is used for getting a color image. The MMAL is controlled to satisfy phase matching condition for the least common multiple wavelength of Red, Green, and Blue or Yellow, Cyan, and Magenta lights to get a color image.

The three-dimensional camcorder may further comprise an optical filter or filters for image quality enhancement.

The three-dimensional camcorder may further comprise an auxiliary lens or group of lenses for image quality enhancement.

The three-dimensional camcorder may further comprise extra MMAL or MMALs to compensate for aberrations of the three-dimensional camcorder including chromatic aberration.

Still another objective of the invention provides a three-dimensional viewfinder for the three-dimensional camcorder, which allows users to find and focus an object in the three-dimensional space and replay recorded three-dimensional video images using one of various display methods including stereoscopic display and volumetric display.

To display three-dimensional video images in the stereoscopic display, the three-dimensional viewfinder comprises an image data input unit, a two-dimensional display, and a stereoscopic glasses. The image data input unit receives a pair of stereoscopic images from the image processing unit or from the storage medium. The two-dimensional display, communicatively coupled to the image data input unit, displays these stereoscopic images in turns within the persistent rate of the human eye. These images can be viewed by stereoscopic glasses.

Alternatively, the three-dimensional viewfinder can comprise an image input unit and an autostereoscopic two-dimensional display. The image data input unit receives a pair of stereoscopic images from the image processing unit or from the storage medium. The autostereoscopic two-dimensional display, communicatively coupled to the image data input unit, displays stereoscopic images in turns within the persistent rate of the human eyes. When autostereoscopic viewfinder is used, the image can be viewed without stereoscopic glasses.

Since the pair of stereoscopic images is generated using all-in-focus image and the depth information for each pixel of the all-in-focus image, the resulting three-dimensional image is all-in-focused (or sharp) unlike the three-dimensional images provided by conventional stereoscopic imaging systems, where the images are in-focused only for a portion of object within the depth of field of the imaging system.

To display three-dimensional video images in a volumetric display, the three-dimensional viewfinder comprises an image data input unit, a two-dimensional display, and a variable focal length MMAL. The image data input unit receives the depthwise image data or from the storage medium. The two-dimensional display, communicatively coupled to the image data input unit, displays depthwise images sequentially within the persistent rate of the human eyes. The variable focal length MMAL, optically coupled to the two-dimensional display, receives light from the two-dimensional display and forms a corresponding image at the required location in the space using the depth information of the depthwise image by changing the focal length of the MMAL. The image formed by the variable focal length MMAL is parallel to the three-dimensional viewfinder screen and located at the corresponding depth along the surface normal direction of the three-dimensional viewfinder screen. The location of the image formed in space is adjusted by changing the focal length of the variable focal length MMAL, which is synchronized with the two-dimensional display so that the variable focal length MMAL can have a focal length corresponding to the depth information of the depthwise image displayed in the two-dimensional display. As a set of depthwise images representing an object at a given moment are sequentially displayed in the two-dimensional display, a three-dimensional image of the object is formed in the space accordingly and perceived as three-dimensional by a viewer. The number of depthwise images representing the object at a given moment (number of depths) depends on the depth resolution requirement, the refresh rate of the two-dimensional display, and the focusing speed of the variable focal length MMAL, and may increase for a better image quality. A set of depthwise image representing an object at a given moment is displayed at least at the persistent rate of the human eye. The focusing speed of the variable focal length MMAL is at least equal to the product of the persistent rate of the human eye and the number of depths so that three-dimensional images formed in space looks realistic to the viewer.

The three-dimensional image information recorded on the storage medium can be displayed in the three-dimensional viewfinder as well as other three-dimensional display devices such as TV, computer monitor, etc.

Still another objective of the invention provides a camcorder, comprising a two-dimensional viewfinder displaying the all-in-focus image.

Still another objective of the invention provides the three-dimensional viewfinder, wherein the focal length of the variable focal length MMAL is fixed to be used as a two-dimensional viewfinder.

Although the present invention is briefly summarized herein, the full understanding of the invention can be obtained by the following drawings, detailed description, and appended claims.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 2illustrates a three-dimensional camcorder21comprising a three-dimensional imaging system22and a three-dimensional viewfinder23. The three-dimensional imaging system22captures at least one image of an object24to provide depthwise images and the depth information for each depthwise image, as shown inFIG.3, one all-in-focus image and the depth information for each pixel of the all-in-focus image, or a pair of stereoscopic images, as shown inFIGS. 8 and 9, depending on the display method. The three-dimensional viewfinder23, communicatively coupled to the three-dimensional imaging system to receive image data and its depth information, displays three-dimensional video image25in real-time to allow a viewer26to find and focus the object24in the three-dimensional space and replays recorded three-dimensional video images using various display methods, as shown inFIGS. 9,10, and11.

FIG. 3illustrates a three-dimensional imaging system for the three-dimensional camcorder comprising at least one variable focal length MicroMirror Array Lens (MMAL)31, an imaging unit32, optically coupled to the MMAL, capturing images formed on the image plane by the variable focal length MMAL31, and an image processing unit33, communicatively coupled to the imaging unit32, producing three-dimensional image data using the images captured by the imaging unit32and the focal length information of the variable focal length MMAL31. The variable focal length MMAL31comprising a plurality of micromirrors34changes its surface profile to change its focal length by controlling the rotation and translation of each micromirror34in the MMAL. The focal length of the variable focal length MMAL31is changed in a plurality of steps in order to scan the whole object (or scene)35.

The imaging unit32comprising at least one two-dimensional image sensor captures images formed on the image plane by the variable focal length MMAL31. As the focal length of the variable focal length MMAL31is changed, the in-focus regions of the object are also changed accordingly.

The image processing unit33extracts the substantially in-focus pixels of each captured image received from the imaging unit and generates a corresponding depthwise image using the extracted in-focus pixels of each captured image. Depth information or the distance between the imaging system and the in-focus region of the object is determined by known imaging system parameters including the focal length of the MMAL and the distance between the MMAL and the image plane. The image processing unit33can combine a set of depthwise images representing the object at a given moment to make an all-in-focus image. Depending on the display method of the three-dimensional display system, the three-dimensional imaging system can provides depthwise images and the depth information for each depthwise image, one all-in-focus image and the depth information for each pixel of the all-in-focus image, or a pair of stereoscopic images generated by the image processing unit33using all-in-focus image and the depth information for each pixel of the all-in-focus image. These image data are transferred to an image data input unit36in the three-dimensional viewfinder to find and focus the object35or recorded on the storage medium37. All the processes obtaining the three-dimensional image information representing the object at a given moment are achieved within a unit time which is less than or equal to the persistent rate of the human eye.

The three-dimensional imaging system for the three-dimensional camcorder can image an object which does not lie on the nominal optical axis by using the MMAL31without macroscopic mechanical movement of the three-dimensional imaging system, as shown in theFIG. 5.

The three-dimensional imaging system for the three-dimensional camcorder further comprises a second variable focal length MMAL38to provide zoom function. As the first variable focal length lens31changes the image size, the image is defocused because the image position is also changed. Therefore, the focal lengths of the two variable focal length MMALs31and38are changed in unison to magnify and keep the image in-focus. Variable focal length lenses provide zoom function without macroscopic mechanical movements of lenses.

The three-dimensional imaging system for the three-dimensional camcorder provides auto focusing function capturing clear images using variable focal length MMALs31and/or38, as shown inFIG. 6.

The three-dimensional imaging system for the three-dimensional camcorder provides vibration correction function using variable focal length MMALs31and/or38, as shown inFIG. 7.

The three-dimensional imaging system for the three-dimensional camcorder provides compensation for aberrations of the system using variable focal length MMALs31and/or38.

The three-dimensional imaging system for the three-dimensional camcorder may further comprise a first auxiliary lens group39A to bring the object into focus.

The three-dimensional imaging system for the three-dimensional camcorder may further comprise a second auxiliary lens group39B to focus the image onto an image sensor.

The three-dimensional imaging system for the three-dimensional camcorder may further comprise a third auxiliary lens group39C to produce an inverted image.

The three-dimensional imaging system for the three-dimensional camcorder may further comprise an optical filter or filters for image quality enhancement.

The three-dimensional imaging system for the three-dimensional camcorder may further comprise additional auxiliary lens or group of lenses for image quality enhancement.

FIG. 4shows how a three-dimensional image is obtained from two-dimensional images captured by a variable focal length MMAL41. Two-dimensional images42A,42B, and42C are captured by changing the focal length of the MMAL41. Captured two-dimensional images42A,42B, and42C have different in-focus pixels because the in-focus plane in an object43is varied as the focal length of the MMAL41is varied. The captured two-dimensional image42A with the first focal length has an in-focus image LI which is the image of the portion L of the object43. The portion L of the object43lies on the in-focus plane44A while images MD, ND of portion M, N of the object43are defocused. Therefore, the image processing unit can extract the in-focus pixels LI from the captured two-dimensional images42A and generate a depthwise image, which contains only in-focus pixels. The distance between the in-focus plane44A in the object and the imaging system can be estimated by using known imaging system parameters including the focal length of the MMAL41and the distance between the MMAL41and the image plane. The captured two-dimensional images42B and42C with the second and the third focal lengths are processed in the same manner as the first captured two-dimensional image42A to provide depthwise images and the depth information for each depthwise image. The focal length of the MMAL41is changed by controlling the rotation and/or translation of each the micromirror45using electrostatic and/or electromagnetic force. For any given focal length, the imaging system can have only one in-focus plane in the object, which may lead to infinitesimal focal length changes with infinite number of steps to provide an all-in-focus image. In practice, the imaging system has a depth of field, which is the distance from near to far that the object appears to be in-focus. In this invention, the imaging processing unit extracts substantially in-focus pixels from the captured two-dimensional image that appears to be in-focus.

FIG. 5shows how the optical axis of the MMAL changes. A bunch of light is focused by the MMAL51. InFIG. 5a, a cube object52is imaged onto the image plane. The light53A from the object52is reflected by each of the micromirror54. The reflected light55A is focused onto the focal point56A of the image and finally makes an image of a cube57A in the image sensor. During the focusing process the optical axis is defined as a surface normal direction58A of a micromirror54.

As shown inFIG. 5b, the MMAL can make a different image57B from a different object59without macroscopic movements. By changing the respective angles of the micromirrors54, this time the MMAL accepts the light53B from the sphere59. The reflected light55B is focused onto a focal point56B and makes the image of the sphere57B. This time the optical axis is changed by an angle and becomes the surface normal direction58B of a micromirror.

FIG. 6aillustrates a three-dimensional imaging system61with auto focusing function using a variable focal length MMAL. First, the light scattered from an object62is refracted by a lens63and is reflected by a MMAL64to an image sensor65. The light reflected from the MMAL is received by the image sensor65and converted into an electrical signal66A carrying the object's image data. The electrical signal is then sent to a signal processor67, where the image data is analyzed and compared to the camera focus criteria. Based on the compared image data, as discussed in further detail below, the signal processor67generates a control signal66B. The control signal is sent to the MMAL to adjust the focal length of the MMAL until the image quality of the image data meets the focus criteria.

As shown inFIG. 6b, the focal length of the MMAL64affects the quality of the image received by the image sensor65. For instance, if the MMAL has a focal length which causes the reflected light68to be in-focused at a point68A short of the image sensor65, the image sensor will generate an electrical signal66A carrying “blurred” image data. Accordingly, the signal processor will process the “blurred” signal and send a control signal66B to the MMAL, causing the positions of the micromirrors69to adjust to lengthen the focal length of the MMAL.

Similarly, if the focal length of the MMAL causes the reflected light to be in-focused at a point68C behind the image sensor65, the image sensor will likewise generate an electrical signal66A carrying “blurred” image data. Accordingly, the signal processor will process the “blurred” signal and send a control signal66B to the MMAL, causing the arrangement of the micromirrors69to adjust to shorten the focal length of the MMAL.

In that regard, the focal length of the MMAL is adjusted in an iterative process until the reflected light is in-focused at a point68B on the image sensor, which provides a “sharp” image, satisfying the camera focus criteria. The iterative process is preferably completed within the persistent rate of the human eye. Thus, the signal processor must have a speed equal to or grater than the product of the number of iterative adjustments, number of depths, and the persistent rate of the human eye.

FIG. 7illustrates a three-dimensional imaging system71having vibration correction function using a MMAL, wherein vibration is corrected without any macroscopic motion.FIG. 7ashows an imaging system with the vibration correction function in operation when there is no vibration.FIG. 7bshows an imaging system with the vibration correction function in operation when vibration exists. The imaging system71includes a lens or lens group72, the vibration correction device73, and an image sensor74. The vibration correction device73comprises a MMAL75, configured to focus an object image onto an image sensor74and to correct optical axis error and a vibration determination device, communicatively coupled to the MMAL75, configured to measure the vibration of the imaging system71and to generate a vibration correction signal. Any type of vibration determination devices may be used in conjunction with the vibration correction device73. The MMAL75is adjusted to change its optical axis76by controlling the rotation and translation of each micromirror in the MMAL, based at least in part on the vibration correction signal to correct the vibration of the imaging system71.

FIGS. 8 and 9illustrate how to generate a pair of stereoscopic image from the all-in-focus image and the depth information for each pixel of the all-in-focus image to display stereoscopic images in the three-dimensional viewfinder.FIG. 8is a schematic illustration representing the reconstruction of three-dimensional coordinates of an object81from the all-in-focus image and the depth information for each pixel of the all-in-focus image. Consider a three-dimensional camera coordinate system (X, Y, Z), of which origin O is at an optical center and Z axis is along an optical axis. Also, consider an image coordinate system (x, y, z), of which origin o is at the center of the image plane82and z axis is along the optical axis, and x and y axes are parallel to X and Y axes, respectively. The image plane has a pixel coordinate system (u, v) which satisfies

u=uc+xpixel⁢⁢width,v=vc+ypixel⁢⁢width
where (uc, vc) represents the center of the image plane in the pixel coordinate system (u, v).

The each pixel on the all-in-focus image has a corresponding focal length f=f(u,v). The position of a point P on the object81with the camera coordinates (X, Y, Z) is determined by thin lens formula and geometry as follows;

Z=bf⁡(u,v)f-bX=-ZxbY=-Zyb
The above relationship can be applied to all pixels on the image plane82.

FIG. 9is a schematic illustration generating pair of stereoscopic images using reconstructed three-dimensional position information. Reconstructed three-dimensional position information in the camera coordinates (X, Y, Z) is projected on two virtual camera systems91and92in order to create a pair of stereoscopic images. Two virtual camera systems91and92are geometrically identical with the actual camera system shown inFIG. 8. Superscripts ′ and ″ are used for virtual camera systems91and92, respectively. The virtual camera system91is placed at −d distance from the origin O in the direction of X axis and the camera system92is place at d distance from the origin O in the direction of X axis. The virtual camera systems have a focal length f0and the distance between a virtual optical centers O′ and O″ and virtual image planes93and94is b, respectively. The three-dimensional position information of the object95is projected on virtual camera systems91and92. Consider the virtual camera system91. Firstly, needed is a change of coordinates from the camera coordinate system P=(X, Y, Z) to the virtual camera coordinate system P′=(X′, Y′, Z′).
P′=R(P−t)
where R is a 3×3 rotation matrix representing the orientation of the virtual camera coordinate system with respect to the camera coordinate system and t is a translation vector representing the displacement of the origin of the virtual camera coordinate system from the origin of the camera coordinate system. For example, R is an identity matrix and t=(−d, 0, 0). The corresponding pixel location p′ is obtained

u′=uc′-bX′Z′·1pixel⁢⁢widthv′=vc′-bY′Z′·1pixel⁢⁢width
These processes are repeated for all pixels in the all-in-focus image to generate one of stereoscopic images. The same steps are applied for the virtual camera system92, where R is the identity matrix and t=(d, 0, 0) to generate the other stereoscopic image. The problem reconstructing a pair of stereoscopic images is greatly simplified for illustrative purpose. A pair of stereoscopic images is displayed in the three-dimensional viewfinder in turns within the persistent rate of the human eye. These images can be viewed by stereoscopic glasses. When autostereoscopic viewfinder is used, the image can be viewed without stereoscopic glasses. Since the pair of stereoscopic images is generated using all-in-focus image and the depth information for each pixel of the all-in-focus image, the resulting three-dimensional image is all-in-focused (or sharp).

FIG. 10shows a three-dimensional viewfinder101according to one embodiment of the present invention to display three-dimensional video images with a volumetric method. The three-dimensional viewfinder101comprises an image data input unit, a two-dimensional display102, communicatively coupled to the image data input unit, and a variable focal length MMAL103, optically coupled to the two-dimensional display102. The image data input unit receives depthwise images and the depth information for each depthwise image from the image processing unit33or storage medium37inFIG. 3. The two-dimensional display102displays one depthwise image104at a time received from the image data input unit. The two-dimensional display102displays only pixels that should be imaged at the same depth at a given frame. The variable focal length MMAL103receives light from the two-dimensional display102and forms a corresponding image at the required location in the space to generate a three-dimensional image. The location of the image formed in the space depends on the depth information of the depthwise image and is adjusted by changing the focal length of the variable focal length MMAL103. The variable focal length MMAL103is synchronized with the two-dimensional display102so that the variable focal length MMAL103can have a focal length corresponding to the depth information of the depthwise image104displayed in the two-dimensional display102. As a set of depthwise images representing an object at a given moment are sequentially displayed in the two-dimensional display102, a three-dimensional image105of the object is formed in the space accordingly and perceived as three-dimensional by a viewer106. The focal length of the variable focal length MMAL103can be fixed to be used as a two-dimensional viewfinder. Also, the three-dimensional viewfinder may further comprise an auxiliary lens for increasing the size of the three-dimensional display screen.

FIG. 11shows how a two-dimensional display and a variable focal length MMAL display three-dimensional images. At a given moment, an object is represented by a set of depthwise images111A,111B,111C, each of which represents a portion of the object having the same image depth. The number of depthwise images representing the object at a given moment (number of depths) depends on the depth resolution requirement, the refresh rate of the two-dimensional display112, and the focusing speed of the variable focal length MMAL113and may increase for a better image quality. The two-dimensional display112displays one depthwise image at a time. The variable focal length MMAL113receives light from the two-dimensional display112and forms a corresponding image114A,114B,114C at the required location in the space to generate a three-dimensional image. The set of depthwise images representing the object at a given moment are sequentially displayed in the two-dimensional display112within a unit time. In order to have realistic three-dimensional video images in the space, focusing speed of the variable focal length MMAL113and refresh rate of the two-dimensional display112must be equal or greater than the product of the persistent rate of the human eye and the number of depths.

For example, assume that the persistent rate of the human eye is 30 Hz and the number of depths is 10. In order to have realistic three-dimensional video images in the space, the focusing speed of the variable focal length MMAL113and the refresh rate of two-dimensional display112have to be at least equal to 300 Hz, respectively. The variable focal length MMAL113of the present invention is capable of changing the focal length fast enough to generate realistic three-dimensional video images.