Patent Abstract:
An imaging sub-system, a liquid crystal (LC) element, and a digital focus processor are provided. The LC element is placed in the light path of the imaging sub-system, functioning as the aperture of the imaging sub-system, and includes a periodically patterned electrode which is patterned according to a periodical modulation function and configured to blur an intermediate image captured by the imaging sub-system by applying a controllable voltage thereto. The digital focus processor is configured to deconvolute the periodical modulation function to remove the blur away from the intermediate image and determine an all-in-focus real image.

Full Description:
BACKGROUND 
     1. Technical Field 
     The present disclosure relates to imaging systems and, particularly, to a computational imaging system. 
     2. Description of Related Art 
     Generally, an image of an object captured by conventional imaging systems is in focus only over a limited object distance range which is known as depth of field (DOF). Therefore, it is difficult to sharply capture object scenes that span large distances. To obtain an extended DOF, one attempt has been made that deliberately blurs an intermediate image captured by an imaging system by placing a coded aperture in the aperture of the imaging system and then digitally removes the blur using reconstruction algorithms. The coded aperture is patterned according to a modulation transfer function (e.g., a delta function). As such, reconstruction algorithms can effectively deconvolute the modulation transfer function and restores the image to a more recognizable likeness of the object with a greater DOF than what that would have been otherwise obtainable. This is known as coded aperture imaging and is one kind of computational imaging system. See Zand, J., “Coded Aperture Imaging in High Energy Astronomy”, NASA Laboratory for High Energy Astrophysics (LHEA) at NASA&#39;s GSFC (1996); Levin, A., Fergus, R., Durand, F., Freeman, B., “Image and Depth from a Conventional Camera with a Coded Aperture”, ACM Transactions on Graphics (Proc. SIGGRAPH) (2007); Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., Tumblin, J., “Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing”, ACM Transactions on Graphics (Proc. SIGGRAPH) (2007); and Liang, C. K., Lin, T. H., Wong, B. Y., Liu, C., Chen, H. H., “Programmable Aperture Photography: Multiplexed Light Field Acquisition”, ACM Transactions on Graphics (Proc. SIGGRAPH), Vol. 27, No. 3, Article No. 55 (2008). However, to blur the intermediate image, the coded aperture (e.g., the pattern formed on the coded aperture) also blocks large amounts of light rays incident on the aperture, resulting in large amount of light loss. 
     Therefore, it is desirable to provide a computational imaging system, which can overcome the abovementioned shortcomings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Many aspects of the present computational imaging system should be better understood with reference to the following drawings. The components in the drawings are not necessarily drawn to scale, the emphasis instead being placed upon clearly illustrating the principles of the present computational imaging system. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views. 
         FIG. 1  is a schematic view of a computational imaging system, according to a first exemplary embodiment. 
         FIG. 2  is a planar view of a liquid crystal (LC) element of the computational imaging system of  FIG. 1 . 
         FIG. 3  is a planar view of the LC element, according to a second embodiment. 
         FIG. 4  is a planar view of the LC element, according to a third embodiment. 
         FIG. 5  is a planar view of the LC element, according to a fourth embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     Embodiments of the present computational imaging system will now be described in detail with reference to the drawings. 
     Referring to  FIGS. 1 and 2 , a computational imaging system  100 , according to a first embodiment, includes a lens  10 , an image sensor  20 , an LC element  30 , and a digital focus processor  40 . 
     The lens  10  and the image sensor  20  constitute an imaging sub-system. The LC element  30  functions as the aperture of the imaging sub-system constituted by the lens  10  and the image sensor  20  (placed in the light path of the imaging sub-system). 
     The LC element  30  is a transmissive LC panel that has a periodically patterned electrode  32 . The electrode  32  is patterned according to a periodical modulation transfer function (i.e., a spatial function):
 
 H ( x,y )=cos 2π( s   x   x+s   y   y ),  (1)
 
where an origin of the oxy coordinate system is the center of the LC element  30 , the x axis extends along the widthwise direction of the LC element  30 , the y axis extends along the lengthwise direction of the LC element  30 , s x  is a spatial frequency of the electrode  32  along the x axis, and s y  is a spatial frequency of the electrode  32  along the y axis. Assuming that: (i) the refractive index of the LC element  30  outside the electrode  32  is n 0 ; and (ii) the refractive index of the LC element  30  at the electrode  32  is n=n 0 +Δn, where Δn is the refractive index variance caused by applying a voltage to the electrode  32 , the refractive index of the entire LC element  30  can be expressed as a refractive index function:
 
 n ( x,y )= n   0   +Δn ×cos 2π( s   x   x+s   y   y ).  (2)
 
     Also referring to  FIG. 2 , in this embodiment, the electrode  32  is a set of concentric annuluses  322  with uniform distances between each two adjacent annuluses  322 . However, the electrode  32  is not limited to this embodiment, but can conform to other configurations, for example, a rectangular spiral line  324  as shown in  FIG. 3 , a circular dot array  326 , or a rectangular block array  328  as shown in  FIG. 5 . 
     The digital Focus processor  40  includes a Fourier transforming device  42 , a deconvolution device  44 , an inverse Fourier transforming device  46 , and a refocusing device  48 . 
     The Fourier transforming device  42  is configured for transforming a space domain amplitude function U I (x,y) of an intermediate image captured by the image sensor  20  into a frequency domain function U ƒ (x,y), where ƒ x , ƒ y  are x and y axes variables in the frequency domain, respectively. According to Fourier optics, it can be determined that: 
                         U   f     ⁡     (       f   x     ,     f   y       )       =         ⅇ     [     j   ⁢     1     2   ⁢   f       ⁢     (       f   x   2     +     f   y   2       )       ]         jλ   ⁢           ⁢   f       ·     ∫       ∫     -   ∞     ∞     ⁢         U   I     ⁡     (     x   ,   y     )       ⁢     ⅇ       -   j     ⁢       2   ⁢   π       λ   ⁢           ⁢   f       ⁢     (       xf   x     +     y   ⁢           ⁢     f   y         )         ⁢     ⅆ   x     ⁢     ⅆ   y               ,           (   3   )               
where j is the imaginary unit, λ is a wavelength of light rays that captured by the image sensor  20 , ƒ(x,y) is a focal length function of each point (e.g., pixel) (x,y) of the image sensor  20  to bring the corresponding point (x,y) into focus.
 
     In addition, the Fourier transforming device  42  is also used for transforming the spatial function of the electrode  32  H(x,y) into a corresponding frequency domain function: H ƒ (ƒ x ,ƒ y ). 
     According to complex optics, the function U ƒ (ƒ x ,ƒ y ) is the convolution of a function U S (x,y) and the function H(x,y), that is,
 
 U   I ( x,y )= U   S ( x,y )· H ( x,y ),  (4)
 
wherein the function U S (x,y) is a spatial domain amplitude function of a real (final) image of objects. As such, to obtain the real image of the objects, the function U ƒ (ƒ x ,ƒ y ) must go through deconvolution to obtain the function H ƒ (ƒ x ,ƒ y ). This is accomplished by the deconvolution device  44 . According to mathematics, it can be determined that:
 
 U   ƒ (ƒ x ,ƒ y )= F ( U   S ( x,y ))· H   ƒ (ƒ x ,ƒ y ),  (5)
 
where F(U S (x,y)) is the Fourier transform of the function U S (x,y). As such, deconvoluting of the function U ƒ (ƒ x ,ƒ y ) can be expressed as:
 
 F ( U   S ( x,y ))={ F}   −1 ( U   ƒ (ƒ x ,ƒ y ))             H   ƒ (ƒ x ,ƒ y ).  (6)
 
As such, the blur caused by the electrode  32  is digitally removed.

     The inverse Fourier transforming device  46  is configured for inversely transforming the frequency domain function F(U S (x,y)) into the spatial domain amplitude function U S (x,y) to restore the real image of the objects. 
     According to the above, it can be determined that the resulting function U S (x,y) is a function of three variables: x, y, and ƒ(x,y). Therefore, for each point (x,y) of the real image, the unique in-focus focal length ƒ(x,y) can be determined. The refocusing device  50  is configured to determine the unique in-focus focal length for each point (x,y) of the real image to bring all points of the real image into focus. As such, an all-in-focus real image of the objects can be obtained. 
     By employing the LC element  30 , transmittance of the electrode  32  can be controlled by adjusting the voltage applied thereto. As such, the amount of light loss can be controlled and minimized. Typically, to reduce light loss, a transmittance of the electrode  32  is greater than about 50%. 
     It will be understood that the above particular embodiments and methods are shown and described by way of illustration only. The principles and the features of the present disclosure may be employed in various and numerous embodiment thereof without departing from the scope of the disclosure as claimed. The above-described embodiments illustrate the scope of the disclosure but do not restrict the scope of the disclosure.

Technology Classification (CPC): 7