Abstract:
An image processing method and apparatus operable for processing images is disclosed. The image processing method allows for drawing any given image in an output image on which various types of image processing have been performed. Such various types of image processing are, for example, correction of an image having distortion due to the distortion aberration of an imaging lens, an image having unnatural perspective distortion of a subject due to high-angle or overhead photography, generation of an output image with the viewpoint changed from the viewpoint of a captured image, mirror-image conversion of a captured image, and electronic zooming. Embodiments of the present invention also relate to an imaging apparatus that uses the image processing method.

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
       [0001]    The present application is a continuation-in-part of prior U.S. application Ser. No. 12/491,437, filed on Jun. 25, 2009, which claims the benefit of Japanese Application No. 2008-169105, filed on Jun. 27, 2008, entitled “IMAGE PICKUP APPARATUS AND ELECTRONIC DEVICE”, and claims the benefit of Japanese Application No. 2009-270794, filed on Nov. 27, 2009, entitled “IMAGE PICKUP APPARATUS”. The contents of each are incorporated by reference herein in their entirety. 
     
    
     FIELD OF THE INVENTION 
       [0002]    The present invention relates generally to imaging devices (image pickup apparatuses), and more particularly relates to an imaging device which includes an image pickup device and an optical system. 
       BACKGROUND 
       [0003]    With the rapid development of digitalization of information, digitalization in image processing is increasingly required. In digital cameras in particular, solid-state image pickup devices, such as Charge Coupled Devices (CCD) and Complementary Metal Oxide Semiconductor (CMOS) sensors, have been mainly provided on imaging planes instead of films. 
         [0004]    In imaging devices including CCDs or CMOS sensors, an image of an object is optically taken by an optical system and is extracted by an image pickup device in a form of an electric signal. Such apparatuses may be used in digital still cameras, video cameras, digital video units, personal computers, mobile phones, PDAs, image inspection apparatuses, industrial cameras used for automatic control, and the like. 
         [0005]    An imaging device, in which light is regularly dispersed by a phase plate and is reconstructed by digital processing to achieve a large depth of field, has been suggested. In the present market for imaging devices, miniaturization is desired. In order to meet this need for miniaturization, a lower height of a lens unit and a higher sensitivity for imaging devices is desired. 
         [0006]    If a micro lens array is arranged at the pre-stage of a solid-state imaging element in order to improve the sensitivity, the quantity of light that enters each pixel of the solid-state imaging element can be efficiently collected. The light collecting efficiency to the pixel of the solid-state imaging element improves if the micro lens corresponding to the pixel of each solid-state imaging element is used. However, if the center of the micro lens is arranged on the normal line of the center of the pixel of each solid-state imaging element, the light collecting efficiency lowers as the light diagonally enters at the lens arranged at the peripheral part. As a result, shading in which the peripheral part becomes a dark image occurs. Consideration is made in shifting the position of the micro lens towards the center side in the direction of the peripheral part, but this is not sufficient. Thus, further increasing the quantity of light that enters each pixel and enhancing the sensitivity are desired. 
         [0007]    The shading that occurs caused by various complex factors can be appropriately corrected by combining a plurality of correcting functions for every factor. However, reducing the shading by a simple method is desired as the processing amount of the correction by image signal processing significantly increases with increase in the number of pixels and the condition parameters. 
         [0008]    In addition, in the known imaging devices, it is premised that a Point Spread Function (PSF) obtained when the above-described phase plate is placed in an optical system is constant. If the PSF varies, it becomes difficult to obtain an image with a large depth of field by convolution using a kernel. 
         [0009]    Therefore, setting single focus lens systems aside, in lens systems such as zoom systems and autofocus (AF) systems, there is a problem in adopting the above-mentioned structure because high precision is required in the optical design and costs are increased accordingly. More specifically, in known imaging devices, a suitable convolution operation cannot be performed and the optical system must be designed so as to eliminate aberrations, such as astigmatism, coma aberration, and zoom chromatic aberration that cause a displacement of a spot image at wide angle and telephoto positions. However, to eliminate the aberrations, the complexity of the optical design is increased and the number of design steps, costs, and the lens size are increased. 
         [0010]    Since a phase plate may set OTF almost constant to the broad object distance, a point image can have a rotationally asymmetric shape. As a result, using an optical system with an image processing, a picture with a large depth of field can be taken. However, the constant OTF may cause a degradation of contrast. If an image processing such as a convolution process is used in order to improve such degraded contrast, a noise would increase as a result. 
         [0011]    Therefore, there is a need for simplified image processing systems, specifically, an optical system which can achieve a high contrast without a large noise. 
       SUMMARY 
       [0012]    An image processing method and apparatus operable for processing images is disclosed. The image processing method allows for drawing any given image in an output image on which various types of image processing have been performed. Such various types of image processing are, for example, correction of an image having distortion due to the distortion aberration of an imaging lens, an image having unnatural perspective distortion of a subject due to high-angle or overhead photography, generation of an output image with the viewpoint changed from the viewpoint of a captured image, mirror-image conversion of a captured image, and electronic zooming. Embodiments of the present invention also relate to an imaging apparatus that uses the image processing method. 
         [0013]    A first embodiment comprises an imaging device. The imaging device comprises an aberration control optical system comprising a lens and an aberration control unit for causing aberration. The imaging device further comprises an imaging element for imaging a subject image passed through the aberration control optical system. The angle formed by an optical axis and a light beam passed through the aberration control optical system is smaller than an angle formed by the optical axis and a light beam from an outermost peripheral part of the lens defining an image side numerical aperture of the optical system not including the aberration control unit. 
         [0014]    A second embodiment comprises an imaging method. The method comprises receiving a light in a lens system and causing aberration to the light to obtain an aberated light. The method further comprises controlling an incident angle of the light entering an imaging element to be smaller than an incident angle of an optical system that does not cause the aberration. The method further comprises imaging the aberated light on a light receiving surface of the imaging element to obtain a one-dimensional image in which a PSF is diffused. The method also comprises performing image processing for restoring the PSF of the one-dimensional image to one pixel. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0015]    Embodiments of the present invention are hereinafter described in conjunction with the following figures, wherein like numerals denote like elements. The figures are provided for illustration and depict exemplary embodiments of the invention. The figures are provided to facilitate understanding of the invention without limiting the breadth, scope, scale, or applicability of the invention. The drawings are not necessarily made to scale. 
           [0016]      FIG. 1  illustrates an outer appearance view showing an exemplary information code reading device serving as an electronic device according to an embodiment of the present invention. 
           [0017]      FIG. 2A  illustrates an exemplary JAN code as an example of an information code. 
           [0018]      FIG. 2B  illustrates an exemplary CODE 49 code as an example of an information code. 
           [0019]      FIG. 2C  illustrates an exemplary QR code as an example of an information code. 
           [0020]      FIG. 3  illustrates a block diagram showing an exemplary configuration of an imaging device applicable to the information code reading device of  FIG. 1 . 
           [0021]      FIG. 4  illustrates a schematic diagram illustrating the structure of an imaging device and traces of light ray bundles. 
           [0022]      FIG. 5A  illustrates exemplary spot images formed on a light-receiving surface of an image pickup device in the imaging device shown in  FIG. 4  when a focal point is displaced by 0.2 mm (Defocus=0.2 mm). 
           [0023]      FIG. 5B  illustrates exemplary spot images formed on a light-receiving surface of an image pickup device in the imaging device shown in  FIG. 4  when the focal point is not displaced (Best focus). 
           [0024]      FIG. 5C  illustrates exemplary spot images formed on a light-receiving surface of an image pickup device in the imaging device shown in  FIG. 4  when the focal point is displaced by −0.2 mm (Defocus=−0.2 mm). 
           [0025]      FIGS. 6A-6B  show a basic configuration example of an imaging lens unit forming the optical system. 
           [0026]      FIG. 7  is a view for describing the conditions of the maximum incident light angle with respect to the optical axis. 
           [0027]      FIG. 8A  is a light beam diagram of the optical system including the aberration control unit according to an embodiment of the present invention. 
           [0028]      FIG. 8B  is a view showing an example of a spot image formed on the imaging plane in  FIG. 6A . 
           [0029]      FIG. 9A  is a light beam diagram of a normal optical system not including the aberration control unit. 
           [0030]      FIG. 9B  is a view showing an example of a spot image formed on the imaging plane in  FIG. 7A . 
           [0031]      FIG. 10  is a view for describing the image processing on the PSF diffused by the aberration control optical system. 
           [0032]      FIGS. 11A-11B , and  FIGS. 12A-12B  are views for describing the spherical aberration occurrence amount of the aberration control optical system according to an embodiment of the present invention. 
           [0033]      FIGS. 13A-13C  show relationships between the spherical aberration by the aberration control element and the imaging. 
           [0034]      FIG. 14  is a view showing an optical path in the case of a normal lens design in which the micro lens of the sensor is designed with the corresponding sensor opening. 
           [0035]      FIG. 15  is a view showing an optical path in the case of using a lens of a brightness exceeding the range the micro lens of the sensor can collect. 
           [0036]      FIG. 16  is a view showing a case in which the light beam incident angle to the sensor is alleviated by using the aberration control element according to an embodiment of the present invention. 
           [0037]      FIG. 17  illustrates an exemplary diagram illustrating the characteristics obtained when the image pickup device has shading. 
           [0038]      FIG. 18  illustrates an exemplary diagram illustrating the characteristics obtained by an existing optical system. 
           [0039]      FIG. 19  illustrates an exemplary diagram illustrating the characteristics obtained by the optical system including an aberration controller according to an embodiment of the invention. 
           [0040]      FIG. 20  illustrates a view showing a calculation example of the shading efficiency in the case where the normal optical system is applied. 
           [0041]      FIG. 21  illustrates a view showing a calculation example of the shading efficiency according to an embodiment of the invention. 
           [0042]      FIG. 22A  is an illustration of an MTF for defocusing in the normal optical system. 
           [0043]      FIG. 22B  is an illustration of an MTF for defocusing in a DEOS. 
           [0044]      FIG. 22C  is an illustration of an MTF for defocusing in an optical system where OTF is constant over object distances. 
           [0045]      FIG. 23A  is an illustration of a spherical aberration curve. 
           [0046]      FIG. 23B  is an illustration of the peak of an MTF in a principal-image-plane shift area MSAR at a low frequency. 
           [0047]      FIG. 23C  is an illustration of the peak of an MTF in a principal-image-plane shift area MSAR at a high frequency. 
           [0048]      FIG. 24A  is an illustration of a spherical aberration curve. 
           [0049]      FIG. 24B  is an illustration of the peak of an MTF in a principal-image-plane shift area MSAR at a low frequency. 
           [0050]      FIG. 24C  is an illustration of the peak of an MTF in a principal-image-plane shift area MSAR at a high frequency. 
           [0051]      FIG. 25  illustrates an exemplary diagram for explaining an modulation transfer function (MTF) correction process performed by an image processing device according to an embodiment of the invention. 
           [0052]      FIG. 26  illustrates an exemplary diagram for specifically explaining the MTF correction process performed by the image processing device according to an embodiment of the invention. 
           [0053]      FIG. 27  illustrates an exemplary diagram illustrating the MTF response obtained when an object is in focus and when the object is out of focus in the typical optical system. 
           [0054]      FIG. 28  illustrates an exemplary diagram illustrating the MTF response obtained when an object is in focus and when the object is out of focus in an optical system including the aberration controller according to an embodiment of the invention. 
           [0055]      FIG. 29  illustrates an exemplary diagram illustrating the MTF response obtained after image processing process in the imaging device according to an embodiment of the invention. 
           [0056]      FIG. 30  is an illustration of transfer from spectrum distribution of an object to spectrum distribution of an object image via an optical transfer function. 
           [0057]      FIG. 31  is an exemplary diagram illustrating the principle of a Depth Expansion Optical System (DEOS). 
           [0058]      FIGS. 32A and 32B  are diagrams illustrating light rays in the phase modulation section of the DEOS in the states in which the aperture stop located is at different positions.  FIG. 32A  is an exemplary diagram showing a state in which the aperture stop is close to the phase modulation surface.  FIG. 32B  is an exemplary diagram showing a state in which the aperture stop is far from the phase modulation surface. 
           [0059]      FIG. 33A  is an exemplary diagram showing a state in which an angle to the optical axis of a main light ray near the aperture stop is small. 
           [0060]      FIG. 33B  is an exemplary diagram showing a state in which an angle to the optical axis of a main light ray near the aperture stop is large. 
       
    
    
     DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 
       [0061]    The following detailed description is exemplary in nature and is not intended to limit the invention or the application and uses of the embodiments of the invention. Descriptions of specific devices, techniques, and applications are provided only as examples. Modifications to the examples described herein will be readily apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other examples and applications without departing from the spirit and scope of the invention. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, brief summary or the following detailed description. The present invention should be accorded scope consistent with the claims, and not limited to the examples described and shown herein. 
         [0062]    Embodiments of the invention are described herein in the context of practical non-limiting applications, namely image processing. Embodiments of the invention, however, are not limited to such image processing applications, and the techniques described herein may also be utilized in other optical applications. For example, embodiments may be applicable to image projection. 
         [0063]    As would be apparent to one of ordinary skill in the art after reading this description, these are merely examples and the embodiments of the invention are not limited to operating in accordance with these examples. Other embodiments may be utilized and structural changes may be made without departing from the scope of the exemplary embodiments of the present invention. 
         [0064]    The following description is presented to enable a person of ordinary skill in the art to make and use the embodiments of the invention. Descriptions of specific devices, techniques, and applications are provided only as examples. Various modifications to the examples described herein will be readily apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other examples and applications without departing from the spirit and scope of the embodiments of the present invention. Thus, the embodiments of the present invention are not intended to be limited to the examples described herein and shown, but are to be accorded the scope consistent with the claims. 
         [0065]      FIG. 1  is an outer appearance view showing one example of an information code reading device serving as an electronic device according to an embodiment of the present invention.  FIGS. 2A to 2C  are views showing an example of an information code.  FIG. 2A  shows an example of a JAN code,  FIG. 2B  shows an example of a CODE 49, and  FIG. 2C  shows an example of a QR code.  FIG. 3  is a block diagram showing a configuration of an imaging device applicable to the information code reading device of  FIG. 1 . Here, an information code reading device is illustrated as an electronic device applicable to the imaging device of the present embodiment. 
         [0066]    As shown in  FIG. 1 , the information code reading device  100  according to the present embodiment is a device, which main body  110  is connected with a processing device such as an electronic register (not shown) by way of a cable  111 , capable of reading an information code  121  such as a symbol and a code having different reflectivity printed on a reading objet  120  and the like. 
         [0067]    The information code to be read may be a one-dimensional barcode  122  ( FIG. 2A ) such as the JAN code, or a two-dimensional barcode  123  such as a stack-type CODE 49 ( FIG. 2B ) or a matrix type QR code ( FIG. 2C ). 
         [0068]      FIG. 4  is a schematic diagram illustrating a structure of an imaging device  1  and traces of light ray bundles. The imaging device  1  comprises an optical system  2  and an image pickup device  3 . The optical system  2  includes object-side lenses  21  and  22 , an aperture stop  23 , and an imaging lens  24  arranged in order from an object side (OBJS) toward the image pickup device  3 . The image pickup device  3  may, without limitation, be a semiconductor sensor such as a CCD and a CMOS sensor. 
         [0069]    The object-side lenses  21  and  22  focus the image of an object before the aperture stop  23 , and the imaging lens  24  focuses the image of an object after the aperture stop  23 . 
         [0070]    Referring to  FIG. 4 , in the imaging device  1 , the best-focus plane coincides with the plane on which the image pickup device  3  is disposed.  FIGS. 5A to 5C  illustrate spot images formed on a light-receiving surface of an image pickup device  3  in the imaging device  1  shown in  FIG. 1  when a focal point is displaced by 0.2 mm (Defocus=0.2 mm), when the focal point is not displaced (Best focus) or when the focal point is displaced by −0.2 mm (Defocus=−0.2 mm), individually. 
         [0071]      FIG. 3  illustrates a block diagram showing an exemplary configuration of an imaging device applicable to the information code reading device of  FIG. 1 . The imaging device  200  (imaging device  200 ) according to the present embodiment includes the aberration control optical system  210  and the image pickup device  220  (imaging element  220 ) for obtaining the first image, and also includes the image processing device  240  for forming the final high-definition image from the first image. The aberration control optical system  210  is provided with an aberration controlling element or an optical element, such as, without limitation, a glass element and a plastic element, having a surface processed so as to control an aberration, so that the wavefront of light can be changed (modulated). The light with the modulated wavefront forms an image, i.e., the first image, on the imaging plane (light-receiving surface) of the image pickup device  220 . 
         [0072]    According to an embodiment, the imaging device  200  is configured to function as an image-forming system that can obtain a high-definition image from the first image through the image processing device  240 . 
         [0073]    In the present embodiment, the first image obtained by the image pickup device  220  is in light conditions with an extremely large depth of field. Therefore, a modulation transfer function (MTF) of the first image is low, and is corrected by the image processing device  240 . 
         [0074]    As shown in  FIG. 3 , the imaging device  200  includes an aberration control optical system  210  and an imaging element  220  (image pickup device  220 ). The imaging device  200  further includes an analog front end unit (AFE)  230 , an image processing device  240 , a camera signal processing unit  250 , an image display memory  260 , an image monitoring device  270 , an operation unit  280 , and a control device  290 . 
         [0075]    Referring to  FIG. 3 , the image pickup device  220  may include a CCD or a CMOS sensor on which the image received from the aberration control optical system  210  is formed and which outputs first image information representing the image formed thereon to the image processing device  240  via the AFE unit  230  as a first image electric signal (FIM). 
         [0076]    The AFE unit  230  may include a timing controller  231  and an analog/digital (A/D) converter  232 . The timing controller  231  controls timing for driving the CCD in the image pickup device  220 . The ND converter  232  converts an analog signal input from the CCD into a digital signal, and can output the thus-obtained digital signal to the image processing device  240 . 
         [0077]    The image processing device  240  can receive the digital signal representing the picked-up image from the AFE unit  230 , subject the signal to a image processing process such as edge enhancement process and the like, improve the contrast which is lowered by the aberration control optical system  210 , and output the result to the camera signal processor  250  (e.g., a digital signal processor (DSP)). 
         [0078]    The camera signal processor (or the digital signal processor (DSP))  250  is operable to perform, without limitation, processes including color interpolation, white balancing, YCbCr conversion, compression, filing, etc., stores data in the memory  260 , and displays images on the image monitoring device  270 . 
         [0079]    The controller  290  is operable to perform exposure control, receive operation inputs from the operating unit  280  and the like, and determine the overall operation of the system on the basis of the received operation inputs. Thus, the controller  290  can control the AFE unit  230 , the image processing device  240 , DSP  250 , the aperture stop  214 , and the like, so as to perform aberration control of the overall system. 
         [0080]      FIG. 6  shows a basic configuration example of an imaging lens unit forming the optical system.  FIG. 6A  shows an example in which an aberration control surface is adopted for the aberration control unit.  FIG. 6B  shows an example in which the aberration control surface serving as the aberration control unit also functions as an aperture. 
         [0081]    The imaging device  200  ( FIG. 3 ) includes the aberration control optical system  210  and the imaging element  220 . The aberration control optical system  210  includes a plurality of lenses  211  to  216 , and an aberration control unit  213 . In the present embodiment, the aberration control optical system  210  is referred to as the aberration control optical system  210  as it controls the aberration. 
         [0082]    The aberration control optical system  210  provides an image photographed with a subject object OBJ to the imaging element  220 . The aberration control optical system  210  is arranged with a first lens  211 , a second lens  212 , a third lens  213 , an aperture stop  214 , a fourth lens  215 , and a fifth lens  216  in order from the object side, but is not limited thereto. The fourth lens  215  and the fifth lens  216  are joined. The aberration control optical system  210  may also include a cemented lens. 
         [0083]    The imaging element  220  ( FIG. 6A ) has a plane parallel plate made of glass (cover glass)  221 , and an imaging plane  222  of the imaging element  220  arranged in order from the fifth lens  216  side. The light from the subject OBJ through the aberration control optical system  210  is imaged on the imaging plane  222  of the imaging element  220 . A subject dispersed image imaged with the imaging element  220  is not focused on the imaging element  220  by the aberration control surface  213   a , and is an image formed with a light flux of deep depth and blurred portion. 
         [0084]    The aberration control unit  213  has an aberration control function for intentionally causing aberration. The aberration control unit  213  includes an aberration control surface  213   a  or an aberration control element, but is not limited thereto. 
         [0085]      FIG. 6A  shows an example in which the aberration control surface  213   a  is applied to the aberration control unit  213 . The aberration control surface internally includes an aberration control effect of the aberration control element, to be hereinafter described, at the lens surface, but is not limited thereto. In this embodiment, the aberration control surface  213   a  is inserted to cause spherical aberration. The aberration control surface  213   a  includes one surface of the lens. Specifically, the aberration control surface  213   a  is that in which a third lens R 2  surface that becomes the aberration control surface  213   a  is provided to the normal optical system. The aberration control surface  213   a  may be adjacent to the aperture stop  214 . 
         [0086]    In an embodiment of the present disclosure, the imaging device  200  adopts an aberration control optical system  210 A that employs an aberration control section (e.g., an aberration control surface  213   a  or an aberration control element  213   b ) to intentionally produce aberration (i.e., spherical aberration), that has a depth extending function, and that maintains the depth extending function even when an aperture diameter of a variable aperture stop changes. Alternatively, the imaging device  200  can adopt an aberration control optical system that employs an aberration control section (device) to intentionally produce aberration, that has a depth extending function, and that changes an aberration control function of the aberration control optical system in response to a change in the aperture diameter of the variable aperture stop. 
         [0087]    In the present embodiment, a chief ray does not intersect Fno ray which represent to a light touching an aperture stop among lights which focus an optical axis, as described later. In addition, in the aberration control optical system  210  according to one embodiment, a chief ray near the aperture stop may be substantially parallel to the optical axis. 
         [0088]    The change in spherical aberration before and after forming the aberration control surface  213   a  is shown at the lower side of  FIG. 6A . A uniform image quality is obtained even if the object appears at the center of the screen and even if the object appears at the periphery by arranging the aberration control surface  213   a  adjacent to the aperture stop  214  and optimizing the aberration control surface  213   a . As a result, a fixed focus lens having high performance as a final output image signal of the imaging device  200  is obtained. 
         [0089]      FIG. 6B  is an example in which the aberration control element is applied for the aberration control unit  213 . Thus, the aberration control effect can also be obtained by inserting the aberration control element of another element in the aberration control optical system  210 . A liquid lens and a liquid crystal lens are known as representative examples for realizing the aberration control element. The liquid lens and the liquid crystal lens can change the index of refraction by changing the plane shape or the orientation of the liquid crystal through application of voltage. The spherical aberration can be controlled with such change by also arranging the aberration control element adjacent to the aperture. 
         [0090]    The aberration control element can be easily inserted to an optical system. In other words, the spherical aberration can be easily controlled by inserting the aberration control element to the optical system. The aberration control element may, without limitation, be a variable control element such as liquid crystal lens or liquid lens, or may be an invariable control element made of glass or resin. If an element in which the aberration is invariable is used, the element includes the aberration control surface. The aberration control surface may be that in which the necessary aberration is added to the lens surface. 
         [0091]    The aperture function may be given to the aberration control element (such as  213   a  in  FIG. 6A ) itself, and the aperture may be omitted. 
         [0092]    If the aberration control element itself has the aperture function, a uniform image quality can be obtained even if the object appears at the center of the screen or appears at the periphery. As a result, a fixed focus lens having high performance is obtained as a final output image signal of the imaging device. 
         [0093]    In  FIGS. 6A and 6B , examples in which the aberration control unit  213  is arranged at the interior of the aberration control optical system  210  are shown, but this is not the sole case, and the aberration control unit  213  may be arranged at the exterior of the aberration control optical system  210 . 
         [0094]    The aberration control optical system  210  has an angle formed by the light beam passed through the aberration control optical system  210  and the optical axis smaller than the angle formed by the light beam from the outermost peripheral part of the lens defining the image side numerical aperture of the optical system without the aberration control unit  213  and the optical axis. In other words, the aberration control function causes at least the spherical aberration on the positive side, and the maximum incident angle of the light beam to the light receiving surface of the imaging element  220  is closer to perpendicular compared to a state in which the aberration control function is not provided. 
         [0095]    Furthermore, in the present embodiment, equation (1) shown below is satisfied, where Um is the maximum of the angles formed by the light beam passed through the aberration control optical system  210  and the optical axis, and K is the maximum angle farmed by the limit incident light beam at which the imaging element  220  can tolerate the influence of shading and the optical axis. 
         [0000]      1/(2*sin  Um )&gt;1/(2*sin  K )  (1)
 
         [0096]      FIG. 7  is a view for describing the conditions of the maximum incident light angle with respect to the optical axis. If the angle formed by the peripheral edge light beam parallel to the optical axis OX and the optical axis at the image boundary is U, sin U is the numerical aperture NA of the image side. If the sine condition of equation (1) is satisfied, the following second equation is satisfied between the NA and the F number (Fno). 
         [0000]        Fno= 1/(2 *NA )=1/(2*sin  U )  (2)
 
         [0097]    According to equation (2) above, U is determined by the Fno of the lens. Furthermore, the imaging element  220  included in the imaging device  200  also has a unique NA that takes into view the limit of influence of the shading and the like. Generally, the opening Fno of the lens is also limited with the NA of the imaging element  220 , but the aberration control optical system  210  can be optimized without lowering the quantity of light while maintaining the opening Fno of the lens in accordance with the NA of the imaging element  220  through the use of the configuration of the present embodiment. In the present embodiment, the spherical aberration is used for such means. Equation (1) is satisfied by using the spherical aberration. In equation (1), the angle formed by the peripheral edge light beam parallel to the optical axis using the aberration control element such as aberration control element  213   a  and the optical axis at the image boundary is Urn, and the angle formed by the limit incident light beam at which the imaging element  220  can tolerate the influence of shading and the optical axis is K. 
         [0098]    In the aberration control optical system  210  ( FIG. 6A ) of the present embodiment, the shape of the aspheric surface of the lens is positive in the direction from the object side towards the image plane side, and is expressed with the following equation (3) where k as the conical coefficient, A, B, C, and D are aspheric surface coefficients, and r is the center curvature radius. Furthermore, h represents the height of the light beam and c represents the inverse number of the center curvature radius. However, Z represents the depth from the tangent plane with respect to the plane vertex, A is the aspheric surface coefficient of fourth order, B is the aspheric surface coefficient of sixth order, C is the aspheric surface coefficient of eighth order, and D is the aspheric surface coefficient of tenth order. 
         [0099]    Aspheric surface equation (3): 
         [0000]    
       
         
           
             
               
                 
                   Z 
                   = 
                   
                     
                       
                         
                           ch 
                           2 
                         
                         
                           1 
                           + 
                           
                             
                               1 
                               - 
                               
                                 
                                   ( 
                                   
                                     1 
                                     + 
                                     k 
                                   
                                   ) 
                                 
                                  
                                 
                                   c 
                                   2 
                                 
                                  
                                 
                                   h 
                                   2 
                                 
                               
                             
                           
                         
                       
                        
                       
                         Ah 
                         4 
                       
                     
                     + 
                     
                       
                         B 
                          
                         h 
                       
                       6 
                     
                     + 
                     
                       
                         C 
                          
                         h 
                       
                       8 
                     
                     + 
                     
                       
                         D 
                          
                         h 
                       
                       10 
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0100]    Alternatively, the shape of the aspheric surface of the lens can be expressed with a polynomial surface equation (4) or Zernike surface equation (5) shown below. The Zernike surface equation (5) of the polynomial surface equation (4) is not limited thereto, but is a Zernike equation of fourteenth order. 
         [0101]    Polynomial surface equation (4): 
         [0000]    
       
         
           
             
               
                 
                   z 
                   = 
                   
                     exp 
                      
                     
                       { 
                       
                          
                         × 
                         
                           ( 
                           
                             
                               ∑ 
                               
                                 j 
                                 = 
                                 1 
                               
                               
                                   
                               
                             
                              
                             
                                 
                             
                              
                             
                               
                                 C 
                                 j 
                               
                                
                               
                                 x 
                                 m 
                               
                                
                               
                                 y 
                                 n 
                               
                             
                           
                           ) 
                         
                       
                       } 
                     
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0102]    Where i is sqrt(−1), x and y are axes perpendicular to the optical axis of the lens, n and m are integers, and 
         [0000]    
       
         
           
             
               j 
               = 
               
                 
                   [ 
                   
                     
                       
                         ( 
                         
                           m 
                           + 
                           n 
                         
                         ) 
                       
                       2 
                     
                     + 
                     m 
                     + 
                     
                       3 
                        
                       n 
                     
                   
                   ] 
                 
                 2 
               
             
             , 
             
               
                  
                 x 
                  
               
               ≤ 
               1 
             
             , 
             
               
                  
                 y 
                  
               
               ≤ 
               1. 
             
           
         
       
     
         [0103]    Zernike surface equation (5) (e.g., Zernike equation of fourteenth order): 
         [0000]        Z =ρ(3−12ρ 2 +10 ρ 4 )cos(θ)  (5)
 
         [0104]    where ρ=h 2 , h represents the height of the light beam and θ represents an angle of a radial direction on an optic axis. 
         [0105]    As shown in  FIGS. 6A and 4B , the imaging element  220  has a glass plate  221  and the imaging plane  222  of the imaging element  220  arranged in order from the fifth lens  216  side. The glass plate  221  may be a plane parallel plate made of glass such as cover glass, but is not limited thereto. The imaging element  220  may be a CCD or a CMOS sensor, but is not limited thereto. 
         [0106]    The light from the subject OBJ passes through the aberration control optical system  210 , and is imaged on the imaging plane  222  of the imaging element  220 . According to the present embodiment, the subject image imaged at the imaging element  220  is a dispersed image, and is not focused on the imaging element  220 . In other words, the light flux becomes a deep depth by the aberration control unit  213 , and an image including the blurred portion forms at the imaging plane  222 . 
         [0107]    In the present embodiment, the light beam incident angle to the imaging element  220  is controlled by using the aberration control unit  213 , and the spread PSF is restored by image processing. For example, the light beam incident angle is alleviated by using the aberration control element for mainly causing spherical aberration. In addition, the same operation can be performed by using the aberration control element of polynomial surface, free curved surface by Zernike and the like. 
         [0108]    Thus, the incident angle of the light beam that becomes the main cause of shading can be alleviated by the aberration control optical system  210 . The occurrence of shading thus can be reduced. 
         [0109]      FIG. 8A  is a light beam diagram of the optical system including the aberration control unit  213  of the present embodiment, and  FIG. 8B  is a view showing an example of a spot image formed on the imaging plane in  FIG. 8A .  FIG. 9A  is a light beam diagram of a normal optical system not including the aberration control unit, and  FIG. 9B  is a view showing an example of a spot image formed on the imaging plane in  FIG. 9A . In the embodiment shown in  FIGS. 8A-8B  and  9 A- 9 B, the F number (Fno) is about 3.5. 
         [0110]    As shown in  FIG. 9 , the normal optical system not including the aberration control unit  213  images the subject image with the focus on the light receiving surface of the imaging element  220 . Therefore, the light is collected so as to fit within one pixel. As shown in  FIGS. 8A-8B , the optical system including the aberration control unit  213  of the present embodiment is not focused on the light receiving surface of the imaging element  220 , and the focused position is formed at the distant side (positive side). Therefore, the PSF (point image intensity distribution) crosses two or more pixels. 
         [0111]      FIG. 10  is a view for describing the image processing on the PSF diffused by the aberration control optical system. 
         [0112]    The PSF crossed over two or more pixels is subjected to image processing such that the diffused PSF is restored to one point (one pixel) in the image processing device  240  of post stage. 
         [0113]    The characteristic configuration and function of the aberration control optical system  210  will be further described in detail below. 
         [0114]      FIGS. 11A and 11B , and  FIGS. 12A and 12B  are views for describing the spherical aberration occurrence amount of the aberration control optical system according to the present embodiment.  FIG. 11  shows a relationship between the sensor and the PSF in the case where the imaging element  220  (sensor) is fixed, and  FIG. 12  shows a relationship between the sensor and the PSF in the case where the aberration control optical system is fixed. 
         [0115]    The imaging element  220  is a sensor having a specific pixel pitch. In such a case, the spherical aberration needs to occur and the PSF needs to be made larger than one pixel in the present embodiment. 
         [0116]    As shown in  FIGS. 11A and 12A , even if the spherical aberration occurs at a size the PSF fits within one pixel, this is the same as the normal optical system. In the normal optical system, the size of the center PSF of the focus position generally becomes a minimum. On the other hand, in the aberration control optical system  210  according to the present embodiment, the PSF is controlled to a size that the focus position in addition to the out focus does not fit within one pixel, as shown in  FIG. 11B . 
         [0117]    The selection of the imaging element  220  (sensor) suited to the aberration control optical system  210  will be described below. 
         [0118]    If the aberration control optical system  210  comprising a specific PSF size is provided, the pixel pitch of the sensor smaller than the size of the PSF is preferably selected, as shown in  FIG. 12B . If the pixel pitch larger than the PSF is selected, this may become the same as the normal optical system, and the focus is focused. In such a case, the effect of the spherical aberration of the aberration control optical system  210  cannot be effectively obtained. 
         [0119]      FIGS. 13A to 13C  show relationships between the spherical aberration by the aberration control element ( 213   a ) and the imaging.  FIG. 13A  shows a relationship between the spherical aberration of a general optical system without the aberration control element and the imaging,  FIG. 13B  shows a relationship between the spherical aberration of the aberration control optical system  210  according to the present embodiment comprising the central area on the optical axis side as the close focus side and the peripheral area as the distant focus side and the imaging, and  FIG. 13C  shows a relationship between the spherical aberration of the aberration control optical system  210  according to the present embodiment comprising the central area as the distant focus side and the peripheral area as the close focus side and the imaging. 
         [0120]    As apparent from  FIGS. 13A to 13C , in the aberration control optical system  210  according to the present embodiment, the spherical aberration can be efficiently generated for an aperture diameter at the surface near the aperture compared to the general optical system. 
         [0121]    The effect of reducing shading of the aberration control optical system  210  including the aberration control unit  213  of the present embodiment will be reviewed below. 
         [0122]      FIG. 14  is a view showing an optical path in the case of a normal lens design in which the micro lens of the sensor is designed with the corresponding sensor opening.  FIG. 15  is a view showing an optical path in the case of using a lens of a brightness exceeding the range the micro lens of the sensor can collect.  FIG. 16  is a view showing a case in which the light beam incident angle to the sensor is alleviated by using the aberration control element of the present embodiment. 
         [0123]    The light beam having sharp light beam angle if the sensor is not responded even if the lens is made bright is rejected at the sensor surface. Thus, increase in sensitivity cannot be expected, which becomes the cause of stray light and flare. As a result, the image quality degrades. When attempting to collect the light in one pixel, the light beam incident angle to the sensor becomes sharper in accordance with the Fno, as shown in  FIG. 15 . On the premise of image processing the same, the light can be efficiently received by the sensor by diverging the light to the peripheral pixels, as shown in  FIG. 16 . 
         [0124]      FIG. 17  is a view showing an example of the shading characteristics of the solid-state imaging element.  FIG. 18  is a view showing an example of the shading characteristics of the normal optical system. The shading characteristics show the light receiving efficiency with respect to the light beam incident angle at which the light enters the imaging element  220 . For example, at the central area of the imaging element, the efficiency is highly satisfactory and is about 100% when the light enters the imaging element  220  perpendicularly (0 degree). However, if the light enters with an angle or at an angle of about 10 degrees, for example, the efficiency lowers to about 80%. Actually, the light beam reaches the imaging element  220  as a continuous light flux within a constant angle corresponding to the NA. 
         [0125]    In  FIG. 18 , the area surrounded with diagonal lines shows the angle characteristics in which Fno is about 3.5. In  FIG. 18 , the area surrounded with diagonal lines shows the light beam incident angle characteristics in which Fno is 3.5. 
         [0126]    The shading characteristics of the imaging element  220  are not taken into consideration for the characteristics of lowering in the peripheral light quantity by the vignetting of the lens. However, since the image that is actually output is seen through the imaging element  220 , such characteristics cannot be ignored. Thus, the actual light quantity degrades by the amount of shading. The specific correction amount will be described below. Assuming a case in which the peripheral light quantity of the lens does not influence the imaging element  220 , an area of a rectangular area shown with an angle of about ±8 of the horizontal axis and the vertical axis of 100 is set as the reference of comparison in  FIG. 18 . Actually, however, the efficiency lowers in the light having angle, and thus the area surrounded by the shaded lines of  FIG. 18  (rectangle with rounded top) becomes the target. The area ratio is found to be about 90% with respect to the light quantity of the lens. Thus, the peripheral light quantity of the lens further degrades by the shading characteristics of the imaging element  220 . 
         [0127]      FIG. 19  is a view showing an example of the shading characteristics of the aberration control optical system  210 . In  FIG. 19 , the area surrounded by the shaded lines shows the angle characteristics in which Fno is about 3.5. If is found that the aberration control optical system has small degradation amount as the absolute value of the maximum incident angle is small with respect to the conventional optical system. 
         [0128]      FIG. 20  is a view showing a calculation example of the shading efficiency in the case where the normal optical system is applied.  FIG. 21  is a view showing a calculation example of the shading efficiency according to an embodiment of the invention. 
         [0129]    In the embodiment shown in  FIG. 20 , the integral value surrounded by the maximum incident angle of the numerical aperture (NA) light beam in  FIG. 17  becomes the light quantity by the influence of the sensor shading if the signal of brightness about 100 reached the sensor, for example. The degradation amount increases as the NA becomes larger. Ideally, the efficiency of about 100% is the most satisfactory regardless of the light beam incident angle, but the brightness of about 80% is obtained in the normal optical system as in  FIG. 20 . 
         [0130]    In the embodiment shown in  FIG. 21 , the light beam that passed the aberration control element can alleviate the light beam incident angle (NA) even if the Fno is the same compared to the normal optical system. However, as the PSF also spreads, it needs to be returned to one point through image processing. Returning the PSF to one point is the same as integrating the brightness, and thus compared to a case of collecting the light at one point with the lens as in the normal case, the degradation by shading is small, and a satisfactory efficiency results with regards to sensitivity. In the example embodiment shown in  FIG. 21 , the brightness of about 90.2% is obtained. 
         [0131]    As described above, the image processing device  240  receives the first image FIM from the imaging element  220  and the process of returning the PSF diffused by the aberration control unit  213  to one point through image processing such as a process of integrating the brightness of the pixels over which the PSF is crossed is performed. 
         [0132]    The image processing device  240  also performs image processing process, without limitation, including edge emphasis and the like such as MTF correction process and the like. Then the image processing device  240  performs a process to improve the contrast which has low value caused by the aberration control optical system  201 A. Accordingly, a final high-definition image FNLIM can be controlled. 
         [0133]    In the present embodiment, using a wavefront-aberration-control optical system or a Depth Expansion Optical System (DEOS), an image of an appropriate quality having a high contrast can be obtained without an image processing. A reconstructed image after image processing is not largely influenced by noise In the present embodiment, an image with a sufficient resolution can be obtained. 
         [0134]    The basic principle of the DEOS will be described below. 
         [0135]    In the DEOS, an optical system comprises an aberration control element to control a change in resolution and contrast corresponding to object distances, and extend a depth. The “change in resolution and contrast corresponding to object distances” refers a change in resolution and contrast of an object in a required object distance. For example, if the size of the object is constant, an object image is large relative to a display when the object distance is small, and therefore, the required resolution power is relatively low. In contrast, an object image is small relative to a display when the object distance is large, and the required resolution power is relatively high. 
         [0136]    Accordingly, the aberration control element can control an Optical Transfer Function (OTF) in order to increase the OTF at low frequency for small object distance, and increase the OTF at high frequency for large object distance. 
         [0137]    The aberration control element may have a function to control the OTF in a required distance. In addition, the aberration control element may expand a depth and improve a contrast. In such system, a MTF (an absolute value of OTF) for defocusing can have two or more peaks. 
         [0138]    The DEOS is operable to perform digital signal processing of the regularly blurred image so as to reconstruct a focused image as described below. The image processing may be performed by the image processing device  140 . 
         [0139]      FIGS. 22A to 22C  show MTFs for defocusing in the normal optical system and in the aberration control optical system according to an embodiment of the invention.  FIG. 22A  shows a state of the MTF for defocusing in the normal optical system.  FIG. 22B  shows a state of the MTF for defocusing in the aberration control optical system according to one embodiment.  FIG. 22C  shows a state of the MTF for defocusing in which one peak is enlarged. 
         [0140]    The normal optical system provides one focal point at the center, as shown in  FIG. 22A . Since the second peaks on both sides are reversed at the bottom, spurious resolution occurs. For this reason, a resolution area is a hatched principal-image-plane shift area MSAR. When depth extension is performed with one peak in the normal optical system, the MTF significantly decreases, as shown in  FIG. 22C . 
         [0141]    Accordingly, in the aberration control optical system according to an embodiment, two peaks PK 11  and PK 12  are provided in the MTF for defocusing, as shown in  FIG. 22B , in contrast to one peak PK 1  in the normal optical system. While the MTF slightly decreases, the depth is substantially doubled by the two peaks, and the degree of decrease in the MTF is smaller than when depth extension is performed with one peak. 
         [0142]      FIG. 3B  shows a state of the MTF for defocusing in the aberration control optical system according to one embodiment. 
         [0143]      FIG. 22C  shows a state of the MTF for defocusing in the aberration control optical system according to one embodiment when MTF is constant over the object distance. Since OTF is set constant over the object distance, MTF for defocusing can not comprise a plurality of peaks. As a result, a contrast at out of focus is, especially, lower than DEOS. In addition, as mentioned above, an optical system with a constant MTF over the object distance can not increase OTF at high frequency in a long distance or decrease OTF at low frequency in a short distance. 
         [0144]    Next, a method for controlling OTF to divide a MTF peak for defocusing by controlling an inflection point and amplitude of a spherical aberration. At least one inflection point in a spherical aberration can divide the MTF peak for defocusing. The amplitude can widen a range for defocusing. 
         [0145]      FIGS. 23A to 23C  and  FIGS. 24A to 24C  show that two peaks can be given to the MTF for defocusing at an arbitrary frequency by a spherical aberration curve in the aberration control optical system according to an embodiment.  FIGS. 23A to 23C  show that two peaks can be given to the MTF for defocusing at an arbitrary frequency in the aberration control optical system in which the change in OTF at a high frequency is limited.  FIG. 23A  shows a spherical aberration curve,  FIG. 23B  shows the peak of the MTF in the principal-image-plane shift area MSAR at a low frequency, and  FIG. 23C  shows the peak of the MTF in the principal-image-plane shift area MSAR at a high frequency. 
         [0146]      FIGS. 24A to 24C  show that two peaks can be given to the MTF for defocusing at an arbitrary frequency in the aberration control optical system in which the change in OTF at a low frequency is limited.  FIG. 24A  shows a spherical aberration curve,  FIG. 24B  shows the peak of the MTF in the principal-image-plane shift area MSAR at a low frequency, and  FIG. 24C  shows the peak of the MTF in the principal-image-plane shift area MSAR at a high frequency. 
         [0147]    As shown in  FIGS. 24A to 24C , in order to extend the depth at the low frequency, it is necessary to increase the amplitude of spherical aberration. By controlling the amplitude, two peaks can be given to the defocusing MTF at an arbitrary frequency. That is, the depth at an arbitrary frequency can be extended. In an embodiment, the low frequency and the high frequency for defocusing. 
         [0148]    In the present disclosure, the high frequency refers to a frequency more than or equal to half a Nyquist frequency that is determined by the pixel pitch of the used solid image pickup device (image pickup device  120 ), and the low frequency refers to a frequency less than half the Nyquist frequency. 
         [0149]    The Nyquist frequency is defined as follows: 
         [0000]      Nyquist frequency=1/(pixel pitch of solid image pickup device×2)
 
         [0150]    In the present embodiment, the optical system comprises the aberration control element which comprises an optical element with different thickness between the center and the peripheral thereof such as an aspheric lens. However, the aberration control element may comprise any type of element which can generate an aberration and control the generated aberration. 
         [0151]    For example, but without limitation, an optical element in which refraction index can change such as gradient index lenses, an optical element with a different thickness and refraction index due to surface coating layer thereon and the like such as hybrid lenses, and an optical element which can control a spherical aberration such as liquid crystal lenses or liquid lenses. 
         [0152]    If more contrast is required, an image processing mentioned below may be performed, according to one embodiment of the present invention. The following is an image processing in order to obtain an intensity distribution of an object from an intensity distribution of an object image transferred from an optical system. 
         [0153]    As shown in  FIG. 30 , when a spectrum distribution F(s, t) of an object is supplied using an optical transfer function H(s, t), a spectrum distribution G(s, t) of an object image is generated. 
         [0154]    This process can be expressed by the following equation: 
         [0000]        G ( s,t )= H ( s,t )* F ( s,t ) 
         [0155]    Two dimension Fourier inverse transform can transform the intensity distribution to spectrum distribution which comprises a spectrum with a space frequency s corresponding to X and a space frequency t corresponding to y. In addition, the optical transfer function is obtained by Fourier inverse transformation of a intensity distribution PSF (point-spread-function) of a point image. Therefore, a spectrum distribution F(s, t) can be obtained from a spectrum distribution G(s, t) by the following equation: 
         [0000]        F ( s,t )= G ( s,t )/ H ( s,t ). 
         [0156]    This relationship can be rewritten as follows with using an intensity distribution g(x, y) of an object image, an intensity distribution f(x, y) of an object and an optical system h(x, y). 
         [0000]    
       
      
       f=h 
       −1 
       *g,  
      
     
         [0000]    where h −1  refers to an inverse function. 
         [0157]    As described above, since PSF of the aberration control element is known, the intensity distribution f(x, y) of the object is calculated from the intensity distribution g(x, y) of the object image. The above-mentioned image processing can be considered as a correction of the OTF in terms of spectral space. For easily illustrating, MTF which is an absolute value of OTF may be used. 
         [0158]      FIG. 25  is an exemplary diagram illustrating an MTF correction process performed by image processing device  240  according to an embodiment of the invention. In the MTF correction process performed by the image processing device  240 , the MTF of the first image, which is low as shown by the curve A in  FIG. 25 , is changed to an MTF closer to, or the same as, that shown by the curve B in  FIG. 25  by performing post-processing. The post-processing, may include, without limitation, including edge emphasis, chroma emphasis and the like using the spatial frequency as a parameter. The characteristic shown by the curve B in  FIG. 25  is obtained when, for example, the wavefront shape is not changed without using the aberration controller such as an aberration control surface and an aberration control element in the present embodiment. In the present embodiment, the corrections are performed using the spatial frequency as a parameter. 
         [0159]    According to an embodiment, in order to obtain the final MTF characteristic curve B from the optically obtained MTF characteristic curve A with respect to the special frequency as shown in  FIG. 25 , the original image (first image) is corrected by performing edge emphasis or the like for each spatial frequency as shown in  FIG. 26 . For example, the MTF characteristic shown in  FIG. 25  is processed with an edge emphasis curve with respect to the spatial frequency shown in  FIG. 26 . 
         [0160]    More specifically, in a predetermined spatial frequency range, the degree of edge emphasis is reduced at a low-frequency side and a high-frequency side and is increased in an intermediate frequency region. Accordingly, the desired MTF characteristic curve B can be virtually obtained. 
         [0161]    An MTF response of the present embodiment and that of a typical optical system are discussed below. 
         [0162]      FIG. 27  is an exemplary diagram illustrating the MTF response obtained when an object is in focus and when the object is out of focus in the typical optical system.  FIG. 28  is an exemplary diagram illustrating the MTF response obtained when an object is in focus and when the object is out of focus in an optical system including the aberration control optical system  210  according to an embodiment of the invention.  FIG. 29  is an exemplary diagram illustrating the MTF response obtained after image processing process in the imaging device according to an embodiment of the invention. 
         [0163]    As shown in  FIGS. 27-29 , in the aberration control optical system  210  including the aberration-controller (i.e., the aberration controlling surface or the aberration controlling element), variation in the MTF response obtained when the object is out of focus is smaller than that in an optical system free from the aberration-controller. The MTF response in  FIGS. 27-29  can be increased by subjecting the image formed by the optical system including the aberration-controller to an image processing using the image processor  240 . 
         [0164]    However, if the noise increases when the image processing is performed, the image processing that may enhance the response of the MTF may not be suitably performed. 
         [0165]    The OTF (MTF) value for the optical system including an aberration-controller is preferably about 0.1 or more at the Nyquist frequency shown in  FIG. 28 . The reason for this is described below. In order to obtain the OTF shown in  FIG. 29  after image processing, the gain is increased by the image processing. However, the sensor noise is also increased. Therefore, preferably, the image processing is performed without largely increasing the gain in a high-frequency range around the Nyquist frequency. In an optical system, sufficient resolution can be obtained if the MTF value at the Nyquist frequency is about 0.1 or more. Therefore, if the MTF value is at least about 0.1 before image processing, it is not necessary to increase the gain at the Nyquist frequency by image processing. If the MTF value is less than about 0.1 before image processing, the image after image processing is largely influenced by noise. That is not preferable. 
         [0166]    The spherical aberration is described in the present embodiment, but other aberration may be applied. For example, but without limitation, astigmatism, coma aberration, and distortion aberration may be applied. 
         [0167]    As explained above, the aberration control optical system  210  for intentionally controlling aberration comprises an imaging element  220  and an image processing device  240  for forming a final high-definition image from the first image. The aberration control optical system  210  comprises an aberration controller which intentionally controls an aberration, such as an aberration-controlling element or aberration-controlling optical surface. 
         [0168]    Through the use of the aberration control optical system  210 , the angle formed by the light beam passed through the aberration control optical system  210  and the optical axis is small compared to the angle formed by the light beam from the outermost peripheral part of the lens defining the image side numerical aperture of the aberration control optical system  210  without the aberration control function (unit) and the optical axis, the degradation of light quantity by shading is alleviated, and the diffused PSF is formed as a depth expansion optical system restored through image processing. 
         [0169]    Thus, in the present embodiment, the degradation of light quantity by shading can be alleviated by controlling the sensor incident angle of the light beam using the aberration control function. As a result, the light quantity entering each pixel can be efficiently obtained and the sensitivity can be enhanced, and furthermore, occurrence of shading can be significantly suppressed. 
         [0170]    In the present embodiment, the image quality can be enhanced through enhancement of the sensitivity without using a highly difficult, expensive and large optical lens. Furthermore, although the light beam incident angle to the imaging element  220  becomes smaller, the smaller the electronic device, a more satisfactory image quality can be obtained while suppressing the light beam incident angle. Moreover, a natural image can be obtained without driving the lens. According to the present embodiments, the structure of the optical system is simplified and the cost is reduced. In other words, since the light beam is not collected at one point as in the conventional optical system, tolerance of parts and requested tolerance in time of assembly are not required compared to a case of collecting the light at one point, whereby worsening of yield, increase in cost, and complication of assembly step can be avoided. 
         [0171]    Therefore, the imaging device  200  can be applied to a compact, light weight, and low cost commercial off-the-shelf device. 
         [0172]      FIG. 30  is an illustration of transfer from spectrum distribution of an object to spectrum distribution of an object image via an optical transfer function. A process in  FIG. 30  comprises an image processing. 
         [0173]    The optical system  210  comprises lenses and an aberration control element which can control OTF. Light which has passed through the optical system  210  forms an image on an imaging device. A contrast of an output signal which the imaging device outputs is increased and a reconstruction can be performed. Furthermore, in one embodiment of the present invention, the aperture stop and the aberration control element is arranged appropriately to obtain a large depth over the entire display and an appropriate image quality. 
         [0174]      FIGS. 31A and 31B  are exemplary diagrams illustrating light rays entering to the aperture stop of the DEOS in states in which the aperture stop  214  is at different positions.  FIG. 10A  shows the state in which the aperture stop  214  is close to the phase modulation surface  1130 .  FIG. 10B  shows the state in which the aperture stop  214  is far from the phase modulation surface  1130 . 
         [0175]    As shown in  FIGS. 32A and 32B , the positions at which the light rays pass through the phase modulation surface  1130  differ depending on the position of the aperture stop  214 . 
         [0176]    If the aperture diameter φ of the aperture stop  214  is maintained constant in conditional expression (1), the positions at which the chief ray at the diagonal image height and the axial chief ray pass through the phase modulation surface  1130  in the state in which the aperture stop  214  is far from the phase modulation surface  1130  differ from those in the state in which the aperture stop  214  is close to the phase modulation surface  1130 . 
         [0177]    In the embodiment shown in  FIGS. 32A and 32B , when the aperture stop  214  is far from the phase modulation surface  1130 , the outermost chief ray and the Fno ray pass through the phase modulation surface  1130  at substantially the same position. 
         [0178]    An Fno ray represents the surface of the light flux which passes through one or more lenses. In addition, when the optical axis is defined as a z axis and two axes that are perpendicular to the z axis and to each other are defined as x and y axes, the outermost line of the light represents Fno ray in the y-z plane or x-z plane, and specifically, an upper ray and a lower ray represent upper and lower lines of the Fno ray respectively. 
         [0179]    In an optical system according to an embodiment, the aperture stop  214 , the aberration control element  1130  are adjacent to each other, and a chief ray near the aperture stop is parallel to the optical axis, and an outermost chief ray does not intersect an Fno ray between the aperture stop  214  and the aberration control element  1130 . 
         [0180]    Accordingly, an angle of the chief ray to the aperture stop can be nearly equal to 0° so that a variation of a point where the light passes in the aberration control surface  1130 . In this case, the distance between the aperture stop  214  and the aberration control surface  1130  can be large. That is, in the present embodiment, the chief light near the aperture stop  214  can be substantially parallel to the axis. 
         [0181]      FIGS. 33A and 33B  are diagrams illustrating the influence of telecentricity on the aberration control element  213   b  (phase modulation element). As discussed in the context of discussion of  FIGS. 10 , when the upper ray and the lower ray vary, the spot images are deformed accordingly. Therefore, when the chief ray is substantially parallel to the optical axis, the influence of the aberration control element  213   b  (phase modulation element) can be reduced. In  FIG. 32B , φ/D=1 and α=45° are the conditions, and the chief ray intersects the Fno ray. Therefore, conditional expressions (1) and (2) are preferably satisfied. 
         [0182]    In one embodiment, the optical system  210  is structured such that the aperture stop  214  and the aberration control element  213   b  (phase modulation element) are located next to each other, so that the chief ray and the optical axis are substantially parallel to each other based on the conditional expressions (3) and (4) given below: 
         [0000]      φ/ D&lt; 1  (3)
 
         [0000]      α&lt;45°  (4)
 
         [0000]    where α is the incident angle at which the outermost chief ray is incident on the aperture stop  214 , and φ and D are as described above. 
         [0183]      FIGS. 15A and 15B  are diagrams illustrating the influence of telecentricity on the aberration control element  213   b  (phase modulation element). As discussed in the context of discussion of  FIGS. 11 and 12 , when the upper ray and the lower ray vary, the spot images are deformed accordingly. Therefore, when the aperture stop  214  (aperture stop section) is telecentric, the influence of the aberration control element  213   b  (phase modulation element) can be reduced. In  FIG. 11B , φ/D=1 and α=45° are the conditions, and the outermost chief ray intersects the Fno ray. Therefore, conditional expressions (3) and (4) are preferably satisfied. Analogously, in  FIG. 15A  the conditional expressions (3) and (4) are satisfied and the aperture stop  214  is telecentric, whereas in  FIG. 15B  the conditional expressions (3) and (4) are not satisfied and the aperture stop  214  is not telecentric 
         [0184]    In the optical system  210  according to the present embodiment, the aberration control element  213   b  (wavefront coding optical element) is positioned closer to the object-side lens than the aperture. However, the aberration control element  213   b  (wavefront coding optical element) may also be located at the same position as the aperture or at a position closer to the imaging lens than the aperture. Also in such a case, effects similar to those described above can be obtained. 
         [0185]    Such electronic devices may be, without limitation, a digital still camera, a mobile phone camera, a Personal Digital Assistant (PDA) camera, a camera used for a vehicle, an image inspection apparatus, an industrial camera used for automatic control, and the like. 
         [0186]    While at least one exemplary embodiment has been presented in the foregoing detailed description, the present invention is not limited to the above-described embodiment or embodiments. Variations may be apparent to those skilled in the art. In carrying out the present invention, various modifications, combinations, sub-combinations and alterations may occur in regard to the elements of the above-described embodiment insofar as they are within the technical scope of the present invention or the equivalents thereof. The exemplary embodiment or exemplary embodiments are examples, and are not intended to limit the scope, applicability, or configuration of the invention in any way. Rather, the foregoing detailed description will provide those skilled in the art with a template for implementing the exemplary embodiment or exemplary embodiments. It should be understood that various changes can be made in the function and arrangement of elements without departing from the scope of the invention as set forth in the appended claims and the legal equivalents thereof. Furthermore, although embodiments of the present invention have been described with reference to the accompanying drawings, it is to be noted that changes and modifications may be apparent to those skilled in the art. Such changes and modifications are to be understood as being included within the scope of the present invention as defined by the claims. 
         [0187]    Terms and phrases used in this document, and variations thereof, unless otherwise expressly stated, should be construed as open ended as opposed to limiting. As examples of the foregoing: the term “including” should be read as mean “including, without limitation” or the like; the term “example” is used to provide exemplary instances of the item in discussion, not an exhaustive or limiting list thereof; and adjectives such as “conventional,” “traditional,” “normal,” “standard,” “known” and terms of similar meaning should not be construed as limiting the item described to a given time period or to an item available as of a given time, but instead should be read to encompass conventional, traditional, normal, or standard technologies that may be available or known now or at any time in the future. Likewise, a group of items linked with the conjunction “and” should not be read as requiring that each and every one of those items be present in the grouping, but rather should be read as “and/or” unless expressly stated otherwise. Similarly, a group of items linked with the conjunction “or” should not be read as requiring mutual exclusivity among that group, but rather should also be read as “and/or” unless expressly stated otherwise. Furthermore, although items, elements or components of the invention may be described or claimed in the singular, the plural is contemplated to be within the scope thereof unless limitation to the singular is explicitly stated. The presence of broadening words and phrases such as “one or more,” “at least,” “but not limited to” or other like phrases in some instances shall not be read to mean that the narrower case is intended or required in instances where such broadening phrases may be absent. The term “about” when referring to a numerical value or range is intended to encompass values resulting from experimental error that can occur when taking measurements.