Patent Abstract:
An image pickup apparatus, which comprises a plurality of photoelectric conversion areas, and a light adjustment area including a first transmission portion for transmitting light which is provided in association with a first photoelectric conversion area included in the plurality of photoelectric conversion areas and a second transmission portion for transmitting light which is provided in association with a second photoelectric conversion area included in the plurality of photoelectric conversion areas, wherein the light adjustment area is configured to cause a part of light incident on the second transmission portion to be incident on the first transmission portion.

Full Description:
BACKGROUND OF THE INVENTION  
         [0001]    1. Field of the Invention  
           [0002]    The present invention relates to an image pickup apparatus for picking up an object image.  
           [0003]    2. Related Background Art  
           [0004]    Conventionally, in forming a color image, an image pickup technique is widely used which picks up an object image by a single image pickup element in which pixels each provided with any one of primary color filters of red (R), green (G) and blue (B) are arranged in mosaic and generates luminance information and color information corresponding to the number of pixels by subsequent signal processing. In many cases, a color filter array of the image pickup element used in this technique is the Bayer array. This image pickup technique can realize significant cost reduction without degrading quality of an image so much compared with an image pickup method of a three plate system that separates an object image into R, G and B in advance using a beam splitter for color separation and picking up an image by image pickup elements prepared for respective colors.  
           [0005]    However, the above-mentioned conventional technique has the following problems. In general, image pickup for obtaining good image characteristics is composed of a first process for forming an object image by an optical device, a second process for adjusting a high frequency component of a spatial frequency characteristic of the object image to be suppressed, a third process for photoelectrically converting the object image with the spatial frequency characteristic adjusted, and a fourth process for correcting a response to an obtained electric signal according to a spatial frequency. In this image pickup, since sampling of an optical image is performed by an image pickup element with a limited number of pixels, it is necessary to reduce components equal to or greater than the Nyquist frequency peculiar to the image pickup element in spatial frequency characteristics of the optical image. Here, the Nyquist frequency means a frequency that is a half of a sampling frequency depending on a pixel pitch. Therefore, the optimized series of processes adjusts an optical image subjected to sampling to be an optical image of characteristics corresponding to the Nyquist frequency peculiar to the image pickup element, thereby obtaining a high quality image in which aliasing distortion is not conspicuous, that is, moiré is not conspicuous.  
           [0006]    A Modulation Transfer Function (MTF) that is a spatial frequency transmission characteristic of an image is an evaluation amount with which a characteristic concerning sharpness of a digital still camera, a video camera or the like can be represented well. Specific elements affecting this MTF are a focusing optical system functioning as an optical device, an optical lowpass filter for limiting a band of an object image, an opening shape of an photoelectrical conversion area of an image pickup element, digital aperture correction and the like. An overall MTF representing final image characteristics is given as a product of MTF of each element. That is, it is sufficient to find MTFs for the above-mentioned first to fourth processes, respectively, and calculate a product of the MTFs. However, since digital filtering that is the fourth process is applied to an image output that has already been subjected to sampling by the image pickup element, it is unnecessary to take into account a high frequency wave exceeding the Nyquist frequency.  
           [0007]    Therefore, a configuration for reducing components equal to or greater than the Nyquist frequency peculiar to an image pickup element in spatial frequency characteristics of an optical image means a configuration in which components equal to or greater than the Nyquist frequency are few in a product of the MTF of the first process, the MTF of the second process and the MTF of the third process excluding the fourth process. Here, in the case in which viewing of a still image is considered a premise as in a digital still camera, it is necessary to take into account the fact that an image giving a good feeling of resolution can be easily realized if response in a frequency slightly lower than the Nyquist frequency is higher even if a high frequency wave exceeding the Nyquist frequency is not zero but remains more or less.  
           [0008]    In the formation of an object image by the focusing optical system that is the first process, in general, it is easier to correct optical aberration in a center of a screen than in a periphery of the screen. When it is attempted to obtain a good image in the periphery of the screen, it is necessary to obtain extremely good characteristics close to a diffraction limit MTF that depends on an F-number of a focusing lens in the center of the screen. In recent years, this necessity has been increasing as an image pickup element uses smaller pixels. Therefore, it is better to consider an MTF on the assumption that the focusing optical system is an ideal aplanatic lens.  
           [0009]    In addition, in an image pickup element in which light receiving opening of a width d are laid without spaces, since the width of the light receiving opening is the same as a pixel pitch, a response value of the third process at the Nyquist frequency u=½d is rather high. Due to this reason, it is a general practice to trap the vicinity of the Nyquist frequency in the second process in order to lower the overall MTF in the vicinity of the Nyquist frequency.  
           [0010]    In the second process, an optical lowpass filter is usually used. A material having a birefringence characteristic such as that of quartz is used for the optical lowpass filter. In addition, a diffraction grating of a phase type as disclosed in JP 2000-066141 A may be used. When a birefringent plate is intervened in an optical path of an optical device and the optical axis is slanted to be arranged in parallel with a horizontal direction of an focusing surface, an object image by a normal ray and an object image by an abnormal ray are formed deviating by a predetermined amount in the horizontal direction. Trapping a specific spatial frequency by the birefringent plate means deviating a bright part and a dark part of a stripe of the spatial frequency so as to overlap each other. An MTF by the optical lowpass filter is represented by Expression (1).  
             R 2( u )=|cos (π· u ·ω)|  (1)  
           [0011]    where R2(u) is response, u is a spatial frequency of an optical image and ω is a separation width of an object image.  
           [0012]    If a thickness of the birefringent plate is selected appropriately, it is possible to reduce the response to zero in the Nyquist frequency of the image pickup element. If the diffraction grating is used, it is possible to realize the same effect by separating an optical image into a plurality of images of a predetermined positional relationship and superimposing the images by diffraction. However, it is necessary to grow crystal such as quartz or lithium niobate and then grind it to be thin in order to manufacture the birefringent plate. Thus, the birefringent plate becomes very expensive. In addition, the diffraction grating is also expensive because a highly precise fine structure is required.  
           [0013]    On the other hand, JP 2001-078213 A discloses a technique that makes an effective light receiving opening larger than a pixel pitch to suppress an MTF of pixels equal to or greater than the Nyquist frequency by using a compound eye lens although an image pickup system of a single plate type is used. However, since image shift depending on an object distance occurs due to the compound eye, a sampling pitch of an object image becomes unequal in a distance other than a reference object distance. That is, misregistration occurs. Therefore, a predetermined image performance is not always realized regardless of object conditions.  
           [0014]    Moreover, as disclosed in JP 01-014749 B (FIG. 5), it is also attempted to suppress response to a high spatial frequency by forming a photoelectrical conversion area of pixels in an intricate shape with respect to neighboring pixels. However, since a shape of the pixels becomes complicated, an extremely fine structure is required. In addition, since each pixel divides a plane, an effect is not easily realized if an object image projected on the pixels has, for example, a slant line along the dividing line. In addition, in each pixel of a color image pickup element, only light transmitted through a predetermined optical filter among an incident light flux is photoelectrically converted, and the light is outputted as an electric signal. Thus, light that could not have been transmitted through the optical filter is disposed of as heat or the like.  
           [0015]    [0015]FIG. 26 shows an array of photoelectrical conversion areas  901  on an image pickup element. As shown in FIG. 27, a microlens  902  for magnifying area of an opening is provided for each of the photoelectrical conversion areas  901 . FIG. 28 is a perspective view of the microlens  902 . As shown in the figure, the microlens  902  is a lens having a positive power and has a function of converging received light fluxes to the photoelectrical conversion areas  901  of the image pickup element.  
           [0016]    For example, in a CCD image pickup element having pixels with primary color filters arranged in mosaic, which is said to have good color reproducibility, optical filters of red (R), green (G) and blue (B) are arranged one by one between the microlens  902  and the photoelectrical conversion area  901 . In this case, in the pixels having the optical filters of R arranged, only red light is photoelectrically converted, and blue light and green light are absorbed by the optical filter and generate heat. In the pixels having the optical filters of G arranged, blue light and red light are not photoelectrically converted but outputted as heat in the same manner. In the pixels having the optical filters of B arranged, green light and red light are not photoelectrically converted but outputted as heat in the same manner. FIG. 25 shows a spectral transmissivity characteristic of the color filters of R, G and B in an image pickup element. Since infrared ray has high transmissivity, infrared ray cut filters for cutting a wavelength of 650 nm or more are additionally used in stack. As is seen from this, only ⅓ of visible light is effectively used in one pixel.  
           [0017]    Considering a utilization efficiency for each color of R, G and B more in detail, for example, a ratio of area of R, G and B pixel of a color image pickup element of the Bayer array shown in FIG. 29 is ¼:{fraction (2/4)}:¼ when area of a unit constituting a regular array is assumed to be one. Thus, a utilization ratio of green light when an overall amount of light is one is ⅓×{fraction (2/4)}=⅙ as a product of a term of wavelength selectivity and a term of an area ratio, those for red light and blue light are ⅓×¼={fraction (1/12)}, respectively. When these are totaled, ⅙+{fraction (1/12)}+{fraction (1/12)}=⅓. Therefore, the utilization efficiency is still ⅓. To the contrary, when an overall amount of light is assumed to be one, ⅔×{fraction (2/4)}=⅓ of green light and ⅔×¼=⅙ of red light and blue light are not effectively utilized among the overall amount of light.  
           [0018]    The above description is for the image pickup element using primary color filters. However, ⅓ of visible light is not photoelectrically converted and is not effectively utilized in an image pickup element using complementary color filters. In this way, the conventional image pickup element of the single plate type using either primary color filters or complementary color filters has a bad light utilization efficiency because an image pickup surface is divided by color filters.  
           [0019]    On the other hand, JP 08-182006 A discloses a structure of an image pickup element that eliminates such waste of an amount of light. In this image pickup element, a prism is arranged for each spatial pixel, and object light whose color is separated by the prism is received by three color pixels of R. G and B. However, the color pixels have a size approximately ⅓ of the spatial pixels. If it is attempted to make a sensor of a small pixel pitch, an extremely fine structure is required of the color pixels, but there is a limit in making the pixel pitch smaller.  
         SUMMARY OF THE INVENTION  
         [0020]    The present invention has been devised in view of the above and other drawbacks, and it is an object of the present invention to realize an image pickup apparatus that obtains a high grade image with less moiré without requiring an expensive optical lowpass filter.  
           [0021]    In addition, it is another object of the present invention to realize an image pickup apparatus in which a utilization efficiency of incident light is increased.  
           [0022]    In order to attain the above-mentioned objects, an image pickup apparatus of a first embodiment of the present invention is constituted to have a plurality of light incident portions for causing light to be incident on a plurality of photoelectric conversion areas and an incident light adjustment portion for causing light incident on one of the plurality of light incident portions once to be incident on adjacent another light incident portion.  
           [0023]    Then, an image pickup apparatus that obtains a high grade image with less moiré can be realized without requiring an expensive optical lowpass filter. Moreover, an image pickup apparatus in which a utilization efficiency of incident light is increased can be realized.  
           [0024]    Other objects and features of the present invention will be apparent from the following descriptions and the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0025]    [0025]FIG. 1 is a sectional view of an image pickup element;  
         [0026]    [0026]FIG. 2 is a sectional view of the image pickup element;  
         [0027]    [0027]FIG. 3 is a plan view showing openings of conventional G pixels;  
         [0028]    [0028]FIG. 4 is a plan view showing openings of G pixels in an image pickup element;  
         [0029]    [0029]FIG. 5 is a plan view showing openings of conventional B pixels;  
         [0030]    [0030]FIG. 6 is a plan view showing openings of B pixels in an image pickup element;  
         [0031]    [0031]FIG. 7 shows an MTF characteristic with respect to a spatial frequency component in a horizontal direction for pixels  120 ;  
         [0032]    [0032]FIG. 8 shows an MTF of a rectangular opening pixel;  
         [0033]    [0033]FIG. 9 shows an MTF of an optical lowpass filter;  
         [0034]    [0034]FIG. 10 shows an MTF characteristic of an aplanatic lens at the time when an F number is assumed to be 4.0 and a wavelength of an object image is assumed to be 550 nm;  
         [0035]    [0035]FIG. 11 shows an overall MTF of a focusing lens and pixels of an image pickup element at the time when the pixels  120  are used;  
         [0036]    [0036]FIG. 12 shows an overall MTF of a focusing lens, an optical lowpass filter and pixels of an image pickup element at the time when conventional pixels are used;  
         [0037]    [0037]FIG. 13 shows an overall MTF at the time when an optical lowpass filter is not used in the conventional pixels;  
         [0038]    [0038]FIG. 14 is a perspective view of an interference filter layer;  
         [0039]    [0039]FIG. 15 is a plan view showing an arrangement of pixels and a shape of microlenses;  
         [0040]    [0040]FIG. 16 is a plan view showing an effective light receiving opening of each pixel;  
         [0041]    [0041]FIG. 17 is an MTF characteristic of a light receiving opening with respect to a spatial frequency component in the horizontal direction;  
         [0042]    [0042]FIG. 18 is an overall MTF of a focusing lens and pixels of an image pickup element;  
         [0043]    [0043]FIG. 19 is a plan view of an image pickup element having R, G and B stripe filters;  
         [0044]    [0044]FIG. 20 is a perspective view of an interference filter layer;  
         [0045]    [0045]FIG. 21 is a plan view showing effective light receiving openings of pixels;  
         [0046]    [0046]FIG. 22 is a sectional view of an image pickup element;  
         [0047]    [0047]FIG. 23 is a sectional view of the image pickup element;  
         [0048]    [0048]FIG. 24 is a block diagram showing an image pickup apparatus;  
         [0049]    [0049]FIG. 25 shows spectral transmissivity characteristics of R, G and B color filters;  
         [0050]    [0050]FIG. 26 is a plan view showing an arrangement of photoelectric conversion areas on an image pickup element;  
         [0051]    [0051]FIG. 27 is a plan view showing an arrangement of microlenses on an image pickup element;  
         [0052]    [0052]FIG. 28 is a perspective view of a microlens  902 ; and  
         [0053]    [0053]FIG. 29 is a plan view of color image pickup elements of the Bayer array. 
     
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0054]    Embodiments of the present invention will be hereinafter described with reference to the accompanying drawings.  
         [0055]    First Embodiment  
         [0056]    FIGS.  1  to  14  are views or graphs for explaining a first embodiment in accordance with the present invention. First, FIG. 1 is a sectional view of an image pickup element and FIG. 2 is an enlarged sectional view of the image pickup element. FIGS. 1 and 2 show a state in which object light is incident on a pixel column, in which G pixels and R pixels having different spectral sensitivity characteristics are alternately arranged as in the Bayer array or the like, and reaches photoelectric conversion areas. Note that, in the Bayer array, pixels are regularly arranged with 2×2 R, G, G and B pixels columns as one unit.  
         [0057]    In FIGS. 1 and 2, reference numeral  101  denotes a silicon substrate;  102 , photoelectric conversion areas;  103 ,  105 ,  107 ,  110  and  112 , low refraction material layers;  104 ,  106  and  108 , wiring layers of metal such as aluminum; and  109  and  111 , high refraction material layers. Silicon oxide (SiO2) having an index of refraction of 1.45 and silicon nitride (Si3N4) having an index of refraction of 2.0, both of which have high transmissivity of visible light, are preferable as a material for forming the low refraction material layer and as a material for forming the high refraction material layer, respectively. The high refraction material layer  111  is sandwiched by the low refraction material layers  110  and  112  onto its upper and lower sides and has a rotationally symmetric surface shape that is convex, on both the sides. Thus, the high refraction material layer  111  functions as a microlens having positive power. This serves to collect light from an object into the relatively small photoelectric conversion areas  102  and increase sensitivity of the image pickup element.  
         [0058]    Reference numeral  113 G denotes a G color filter using an organic pigment, which transmits green light and absorbs red light and blue light. Reference numeral  113 R adjacent to  113 G denotes an R color filter, which transmits red light and absorbs blue light and green light. Reference numerals  114  and  116  denote resin layers. Both the color filter layers and the resin layers have an index of refraction of approximately 1.50.  
         [0059]    Reference numeral  115 G denotes an interference filter layer for splitting light, which transmits green light and reflects red light and blue light. Reference numeral  115 R adjacent to  115 G also denotes an interference filter layer, which transmits red light and reflects blue light and green light. These interference filter layers are formed on slopes of a square pyramid having a vertex in a central part of each pixel as shown in FIG. 14. When a not-shown focusing lens forms an object image on a surface on which the high refraction material layers  111  functioning as microlenses are arranged, the object image becomes a focused image which is an output of the image pickup element.  
         [0060]    Note that, in order to facilitate understanding, only a ray  117  is drawn which is emitted from a pupil center of the focusing lens located in a sufficiently distant place with respect to a size of the pixels. Rays emitted from the vicinity of the pupil are incident on the image pickup element with an angle with respect to the ray  117 . Then, these rays reach the peripheral parts in the photoelectric conversion areas  102  to be photoelectrically converted there.  
         [0061]    Next, behavior of rays for each wavelength will be described with reference to FIG. 2. Object light  118  transmitted through a not-shown infrared ray cut filter comes from the upper part of the figure toward a pixel  120  for receiving green light and is incident on the resin layer  116  first. Next, the object light  118  is incident on the interference filter layer  115 G, through which only light of a green component can be transmitted to reach the G color filter  113 G from the resin layer  114 . Light of a blue component and light of a red component reflected by the interference filter layer  115 G are guided to the adjacent pixels. Since this behavior is the same as that of a ray coming into the pixel  120  from an adjacent pixel  121 , a description of the pixel  121  will substitute for that of the behavior. In the G color filter  113 G, green light that can be transmitted through the interference filter layer  115 G has reached there. Thus, most of the green light is transmitted through the G color filter  113 G and reaches the high refraction material layer  111  functioning as a microlens through the next low refraction material layer  112 . Here, the green light is subjected to a converging action and is radiated to the low refraction material layer  110  and further passes through the high refraction material layer  109  to be incident on the photoelectric conversion area  102 . An output from the photoelectric conversion area  102  is inputted to a signal processing circuit discussed later as a green component.  
         [0062]    The adjacent pixel  121  is a pixel for receiving red light. Object light  119  transmitted through the not-shown infrared ray cut filter comes to the pixel  121  from the upper part of the figure and is incident on the resin layer  116  first. Next, the object light  119  is incident on the interference filter layer  115 R, through which only light of a red component can be transmitted to reach the R color filter  113 R from the resin layer  114 . A behavior thereafter of the red light is the same as that of the green light in the pixel  120  described above.  
         [0063]    Blue light and green light reflected by the interference filter layer  115 R advance toward an interface between the resin layer  116  and the air and then are subjected to total internal reflection because an incident angle of the light on the interface becomes equal to or smaller than a critical angle due to an action of a slant that is set in the interference filter layer  115 R. The blue light and the green light returned in a direction to the inside of the image pickup element by the total internal reflection are incident on the interference filter layer  115 G. Since the interference filter layer  115 G is originally a filter provided in the pixel  120  for receiving green light, it transmits green light and reflects blue light. The reflected blue light is not illustrated in the figure because it escapes to the outside of the image pickup element.  
         [0064]    As described above, in an light adjustment area including the interference filter layers  115 R and  115 G and the resin layer  116 , green light incident on the interference filter layer  115 R is reflected by the interference filter layer  115 R and is incident on the interference filter layer  115 G and transmitted through it.  
         [0065]    The green light component transmitted through the interference filter layer  115 G reaches the G color filter  113 G through the resin layer  114 . In the G color filter  113 G, green light that can be transmitted through the interference filter layer  115 G has reached there. Thus, most of the green light can be transmitted through the G color filter  113 G and reaches the high refraction material layer  111  often functioning as a microlens, through the next low refraction material layer  112 .  
         [0066]    Rays advance aslant in the high refraction material layer  111  and are reflected on a side of the metal wiring layer  108  before or after radiation to the low refraction material layer  110 . In addition, some rays are reflected on the side of the metal wiring layer  108  before being incident on the high refraction material layer  111 .  
         [0067]    Since an angle of rays coming from the adjacent pixel  121  is obtuse, the rays cannot be incident on the photoelectric conversion area  102  directly through the high refraction material layer  109  but reaches the photoelectric conversion area  102  after inevitably being subjected to the total internal reflection on the sides of the metal wiring layers  106  and  104  or the interfaces between the high refraction material layer  109  and the low refraction material layers  107 ,  105  and  103 . The rays are photoelectrically converted in the photoelectric conversion area  102  together with a green component of the object light  118  and are inputted into a signal processing circuit as an output of the pixel  120 .  
         [0068]    Here, the first embodiment has been described in a relationship between the pixel  120  and the pixel  121 . However, if pixels adjacent to each other are not of the same color as in the Bayer array, a wavelength component unnecessary for any optical can be photoelectrically converted as an effective wavelength component of adjacent pixels by dividing the unnecessary wavelength component between the adjacent pixels. Thus, utilization efficiency of light can be significantly improved.  
         [0069]    Considering utilization efficiency for each color, this is equivalent to a increase of ratio of R, G and B pixel areas of a color image pickup element of the Bayer array to ¼×2:{fraction (2/4)}×2:¼×2, when area of a unit constituting a regular array is assumed to be one. Therefore, a utilization ratio of green light when an overall amount of light is one is ⅓×{fraction (4/4)}=⅓ as a product of a term of wavelength selectivity and a term of an area ratio, and those for red light and blue light are ⅓×{fraction (2/4)}=⅙, respectively. When these are totaled, ⅓+⅙+⅙=⅔, which means that the utilization ratio is twice as large as that in the past. Therefore, sensitivity of the image pickup element can be doubled.  
         [0070]    In addition, in the image pickup element according to this embodiment, a substantial light receiving opening becomes larger than each pixel. For ease of understanding, when compared with the conventional image pickup element of the Bayer array shown in FIG. 29 for each color of R, G and B, first, the opening of the G pixel is larger than each pixel in the image pickup element of this embodiment shown in FIG. 4, whereas the opening of the conventional G pixel is the size of the microlens  902  as shown in FIG. 3. In FIG. 4, reference numeral  130  denotes microlenses and  131  denotes substantial light receiving openings that takes into consideration a green light component spared from adjacent pixels. In the same manner, the opening of the B pixel is larger than each pixel in the image pickup element of the present embodiment as shown in FIG. 6, whereas the opening of the conventional B pixel has the size as shown in FIG. 5. In the figure, reference numeral  132  denotes microlenses and  133  denotes substantial light receiving openings that takes into consideration a blue light component spared from adjacent pixels. The R pixel is equivalent to the B pixel. Therefore, when all the pixels are considered by overlapping FIGS. 4 and 6, it is seen that the pixels have light receiving openings that effectively overlap with each other.  
         [0071]    In this way, when the substantial light receiving opening becomes larger than each pixel, an MTF characteristic that appears to be impossible in the usual image pickup apparatus of the single plate type can be obtained. As a result, a quality of an image is not deteriorated even if an optical lowpass filter is omitted. That is, it is possible to obtain a high quality image in which aliasing distortion is not conspicuous, only by the third process for photoelectrically converting an object image without the above-mentioned second process for adjusting a high frequency component of a spatial frequency characteristic of the object image to be suppressed.  
         [0072]    FIGS.  7  to  13  are graphs explaining the above.  
         [0073]    First, FIG. 7 shows an MTF characteristic with respect to a spatial frequency component in a horizontal direction for the pixel  120  of the image pickup element according to this embodiment. In addition, FIG. 8 shows an MTF characteristic of a pixel having a conventional type of a rectangular opening. In both the cases, a size of one pixel is set as 3 μm×3 μm and a microlens is assumed to have a size of one pixel. Moreover, it is assumed that the pixel of this embodiment has an opening extending to a central part of an adjacent pixel.  
         [0074]    Response of the rectangular opening pixel of the conventional type shown in FIG. 8 can be easily represented by an SINC function as in Expression (2) below.  
           R 3( u )=|sin (π· d·u )/(π· d·u )|  (2)  
         [0075]    where R3(u) is response and d is a width of the light receiving opening of the image pickup element.  
         [0076]    A first zero point (cutoff frequency) of Expression (2) is a position of u=1/d. That is, response becomes zero in a wavelength that is the same as the width of the light receiving opening. In the image pickup element in which light receiving openings are arranged without spaces, since the width of the light receiving opening is the same as the pixel pitch, a response value of Expression (2) in the Nyquist frequency u=½d is relatively high at 0.636. Therefore, it is necessary to use the optical lowpass filter of the MTF characteristic shown in FIG. 9 in the conventional rectangular opening pixel.  
         [0077]    On the other hand, in the pixel  120  according to this embodiment, response extends to a high frequency wave side due to diamond shaped openings as shown in FIG. 4. This may be considered as gathering of rectangular openings of infinitely thin rectangle shape whose MTF characteristic can be represented by Expression (2). A result of integrating the entire response is as shown in FIG. 7. In the Nyquist frequency 167/mm at the time when the pixel pitch is set at 3 μm, it is seen that the pixel  120  has relatively lower response.  
         [0078]    Next, FIG. 10 shows an MTF characteristic of an aplanatic lens in case of that it is assumed that an F number is 4.0 and a wavelength of an object image is 550 nm. In an ideal lens without aberration in terms of geometrical optics, the MTF depends on diffraction of light. A diffraction limit MTF depends on the F number and is represented by Expression (3) below.  
           R 0=2×(β−cos β×sin β)/π  (3)  
         β=cos −1  ( u·F ·λ)  
         [0079]    where u is a spatial frequency of an optical image, F is an F number of an optical system and λ is a wavelength of the optical image.  
         [0080]    A cutoff frequency of this focusing lens is 455/mm.  
         [0081]    Here, all the factors are now prepared to find the overall MTF of the first process for forming an object image by an optical apparatus, (the second process for adjusting a high frequency component of a spatial frequency characteristic of the object image to be suppressed) and the third process for photoelectrically converting the object image whose the spatial frequency characteristic is adjusted.  
         [0082]    [0082]FIG. 11 shows an overall MTF of a focusing lens and pixels of an image pickup element when the pixel  120  is used. On the other hand, FIG. 12 shows an overall MTF of a focusing lens, an optical lowpass filter and pixels of an image pickup element, when the conventional pixel is used. Both the overall MTFs have substantially equal response in the Nyquist frequency 167/mm and have very similar characteristics as a whole. On the other hand, if the optical lowpass filter is not used in the conventional pixel, response at the Nyquist frequency becomes too high as shown in FIG. 13. In this way, it is seen that the optical low pass filter can be removed if the pixel  120  is used.  
         [0083]    Second Embodiment  
         [0084]    [0084]FIGS. 15 and 16 are views for explaining a second embodiment in accordance with the present invention. FIG. 15 is a plan view showing an arrangement of pixels and a shape of microlenses. FIG. 16 is a plan view showing an effective light receiving opening of each pixel.  
         [0085]    In these figures, reference numeral  201  denotes microlenses and  202  denotes effective light receiving openings. An arrangement of pixels is  45  degrees rotated from the Bayer array. Therefore, 2×2 R, G, G and B pixel columns are one unit. As disclosed in JP 2000-184386 A, an image pickup element of such an arrangement is preferable for obtaining an image of higher resolution while suppressing increase of the number of pixels.  
         [0086]    As shown in FIG. 15, each of the pixel openings constituted by the microlens  201  is a square having four sides slanted in a 45 degree direction and is arranged densely with being in contact with adjacent pixels. This image pickup element also has the structure shown in FIG. 2 as in the first embodiment. Note that, in this case, the cross section shown in FIG. 2 is that of the pixel of FIG. 15 cut in a diagonal 45 degree direction.  
         [0087]    The effective light receiving opening  202  of such the image pickup element is equivalent to that slanted 45 degrees in FIG. 4 or FIG. 6 and is now a square having four sides in the vertical and horizontal directions. Therefore, Expression (2) is directly applied in order to find an MTF characteristic for a spatial frequency component in the horizontal direction, and the characteristic is as shown in FIG. 17. As indicated by the characteristic of the SINC function, a response curve continues while becoming lower as the frequency becomes higher. A first zero point is near the Nyquist frequency which depends on a pixel pitch in the horizontal direction.  
         [0088]    Moreover, if the MTF characteristic is multiplied by the MTF characteristic of an aplanatic lens when it is assumed that the F number is 4.0 and the wavelength of an object image is 550 nm shown in FIG. 10, an overall MTF of an focusing lens and pixels of an image pickup element is obtained as shown in FIG. 18.  
         [0089]    If the overall MTF is compared with an overall MTF of an focusing lens, an optical lowpass filter and pixels of an image pickup element when the conventional pixel shown in FIG. 12 is used, both the overall MTFs have substantially equal response in the Nyquist frequency 167/mm and have very similar characteristics as a whole. In this way, it is seen that the optical low pass filter can be removed.  
         [0090]    Third Embodiment  
         [0091]    FIGS.  19  to  21  are views for explaining a third embodiment in accordance with the present invention. FIG. 19 is a plan view of an image pickup element having R, G and B stripe filters. FIG. 20 is a perspective view of an interference filter layer. FIG. 21 is a plan view showing an effective light receiving opening of each pixel.  
         [0092]    In the image pickup element having the R, G and B stripe filters, a lengthwise pixel column having an R filter, a lengthwise pixel column having a G filter and a lengthwise pixel column having a B filter are repeated in sideways. That is, 1×3 R, G and B pixels are one unit having a regular arrangement, and among adjacent four pixels, pixels in the vertical direction are pixels of an identical filter and pixels in the horizontal direction are pixels of different filters. Even in such a structure, the pixel structure shown in the first embodiment is effective in terms of adjusting an MTF of a pixel. However, it is preferable to optimize the structure such that light not photoelectrically converted is exchanged between the adjacent pixels of different filters in terms of increasing a utilization efficiency of light.  
         [0093]    Reference numeral  302  in FIG. 20 denotes an interference filter layer for this purpose, which splits light. Here, parts of the structure other than the interference filter are the same as those in the first embodiment.  
         [0094]    Interference filters having a plurality of roof type slopes of a stripe shape are arranged, and the interference filters on the two slopes across a ridge are the same type. Moreover, a filter for transmitting red light and reflecting blue light and green light is provided for the lengthwise pixel column having the R filter, a filter for transmitting green light and reflecting blue light and red light is provided for the lengthwise pixel column having the G filter, and a filter for transmitting blue light and reflecting green light and red light is provided for the lengthwise pixel column having the B filter.  
         [0095]    An action of the interference filter layer  302  viewed in a cross section in the direction of arrangement of the R, G and B pixels is the same as the image pickup element shown,in FIG. 2 in the first embodiment. However, light is split only in a paper surface direction of FIG. 2 in this embodiment, while light is split in front and back direction on the paper surface by the action of the interference filter of a square pyramid shape in the first embodiment. The paper surface direction means a direction of adjacent pixels having different filters.  
         [0096]    As a result, effective light receiving openings are as shown in FIG. 21. For ease of understanding, pixel columns are extracted for every other column. In FIG. 21, reference numeral  303  denotes effective light receiving openings. Since light not photoelectrically converted is exchanged between the pixels having the different filters, the opening of each pixel extends sideways as in the figure to overlap an adjacent pixel.  
         [0097]    Therefore, a utilization efficiency of light is also improved to be twice as large in this case, and sensitivity is doubled.  
         [0098]    Fourth Embodiment  
         [0099]    [0099]FIG. 22 is a view for explaining a fourth embodiment in accordance with the present invention and is an enlarged sectional view of an image pickup element. FIG. 22 shows a state in which object light is incident on a pixel column, in which G pixels and R pixels are alternately arranged as in the Bayer array or the like from the upper part of the figure, and reaches photoelectric conversion areas.  
         [0100]    In FIG. 22, reference numeral  401  denotes a silicon substrate;  402 , photoelectric conversion areas;  403 ,  405 ,  410 ,  422 G and  422 R, low refraction material layers;  404  and  406 , wiring layers of metal such as aluminum; and  411 ,  422 G and  422 R, high refraction material layers. Silicon oxide (SiO2) having an index of refraction of 1.45 and silicon nitride (Si3N4) having an index of refraction of 2.0, both of which have high transmissivity of visible light, are preferable as a material for forming the low refraction material layer and as a material for forming the high refraction material layer, respectively. The high refraction material layer  411  is sandwiched by the air and the low refraction material layers  410  on its upper and lower sides, respectively, and has an axially symmetric surface shape that is flat on a ray incident side and convex on a ray radiation side. Thus, the high refraction material layer  411  functions as a microlens having positive power. This serves to collect light from an object into the relatively small photoelectric conversion areas  402  and increase sensitivity of the image pickup element. Although not shown in the figure, it is better to attach a reflection preventive film on the light incident surface of the high refraction material layer  411 . In addition, reference numerals  430 G and  430 R denote interference filter layers for splitting light. The interference filter layer  430 G transmits green light and reflects red light and blue light. The interference filter layer  430 R of the adjacent pixel has a characteristic of transmitting red light and reflecting blue light and green light. These are formed on slopes of a square pyramid having a vertex in the central part of each pixel as in the same manner as the interference filter in the first embodiment shown in FIG. 14.  
         [0101]    Note that, in order to facilitate understanding, only rays  418  and  419  are drawn which are emitted from a pupil center of a focusing lens located in a sufficiently distant place with respect to a size of the pixels. Rays emitted from the vicinity of the pupil are incident on the image pickup element with an angle with respect to the rays  418  and  419 . Then, these rays reach the peripheral parts in the photoelectric conversion areas  402  to be photoelectrically converted there.  
         [0102]    Next, behavior of rays for each wavelength will be described. Object light  418  transmitted through a not-shown infrared ray cut filter comes from the upper part of the figure toward a pixel  420  for receiving green light and is incident first on the low refraction material layer  410  functioning as a microlens. Here, the object light  418  is subjected to a converging action, is radiated to the low refraction material layer  410  and incident on the interference filter layer  430 G. Moreover, only a green component can be transmitted through the interference filter layer  430 G and reaches the high refraction material layer  422 G.  
         [0103]    A blue component of a red component reflected by the interference filter layer  430 G are guided to the adjacent pixels. Since this behavior is the same as that of a ray coming into the pixel  420  from an adjacent pixel  421 , a description of the pixel  421  will substitute for that of the behavior.  
         [0104]    In the G color filter  422 G, light that can be transmitted through the interference filter layer  430 G has reached there. Thus, most of the light is transmitted through the G color filter  422 G and incident on the photoelectric conversion area  102 . An output from the photoelectric conversion area  402  is inputted into a signal processing circuit as a green component. The adjacent pixel  421  is a pixel for receiving red light. Object light  419  transmitted through a not-shown infrared ray cut filter comes to the pixel  421  from the upper part of the figure and is incident first on the high refraction material layer  411  functioning as a microlens. Here, the red light is subjected to a converging action and is radiated to the low refraction material layer  410  and incident on the interference filter layer  430 R. Only a red component can be transmitted through the interference filter layer  430 R, and the behavior of the red light thereafter is the same as that of the green light in the pixel  420  described above.  
         [0105]    A blue component and a green component reflected by the interference filter layer  430 R advance toward the high refraction material layer  411  of the adjacent pixel and then are subjected to total internal reflection because an incident angle of the light on the interface with the air becomes equal to or smaller than a critical angle due to an action of a slant that is set in the interference filter layer  430 R. The blue light and the green light returned in a direction to the inside of the image pickup element by the total internal reflection are incident on the interference filter layer  430 G. Since the interference filter layer  430 G is originally a filter provided in the pixel  420  for receiving green light, it transmits the green component and reflects the blue component in the green light. The reflected blue light is not illustrated here in the figure because it escapes to the outside of the image pickup element.  
         [0106]    As described above, in an light adjustment area including the interference filter layers  430 R and  430 G and the high refraction material layer  411 , green light incident on the interference filter layer  115 R is incident on the interference filter layer  115 G and transmitted through it.  
         [0107]    The green light component transmitted through the interference filter layer  430 G is reflected on the side of the metal wiring layer  406  before or after incident on the interference filter layer  430 G and reaches the photoelectric conversion area  402  through the high refraction material layer  422 G. That is, since an angle of rays coming from the adjacent pixel  421  is obtuse, the rays cannot be incident on the photoelectric conversion area  402  directly from the high refraction material layer  411  but reaches the photoelectric conversion area  402  with being subjected to the total internal reflection on the sides of the metal wiring layers  406  and  404  or the interfaces between the high refraction material layer  422 G and the low refraction material layers  405  and  403 . The rays are photoelectrically converted in the photoelectric conversion area  402  together with a green component of the object light  418  and are inputted into a signal processing circuit as an output of the pixel  420 .  
         [0108]    Here, the fourth embodiment has been described in a relationship between the pixel  420  and the pixel  421 . However, if pixels adjacent to each other are not of the same color as in the Bayer array, wavelength component unnecessary for any pixel can be photoelectrically converted as an effective wavelength component of adjacent pixels by dividing the unnecessary wavelength component between the adjacent pixels. Thus, a utilization efficiency of light can be improved to be twice as large. Therefore, sensitivity of the image pickup element can be doubled.  
         [0109]    Fifth Embodiment  
         [0110]    [0110]FIG. 23 is an enlarged sectional view of an image pickup element for explaining a fifth embodiment in accordance with the present invention and shows an example of variation in which the interference filters of the fourth embodiment are arranged along a curved surface of a microlens.  
         [0111]    In FIG. 23, reference numeral  501  denotes a silicon substrate;  502 , a photoelectric conversion area;  503 ,  505 ,  510 ,  522 G and  522 R, low refraction material layers;  504  and  506 , wiring layers of metal such as aluminum;  511 ,  522 G and  522 R, high refraction material layers; and  530 G and  530 R, color filter layers. The high refraction material layer  511  is sandwiched by the air and the low refraction material layers  510  on its upper and lower sides, respectively, and has an axially symmetric surface shape that is flat on a ray incident side and convex on a ray radiation side. Thus, the high refraction material layer  511  functions as a microlens having positive power. This serves to collect light from an object to the relatively small photoelectric conversion areas  502  and increase a sensitivity of the image pickup element. Although not shown in the figure, it is preferable to attach a reflection preventive film on the light incident surface of the high refraction material layer  511 .  
         [0112]    In addition, reference numerals  509 G and  509 R are interference filter layers for splitting light. The interference filter layer  509 G transmits green light and reflects red light and blue light. The interference filter layer  509 R of the adjacent pixel has a characteristic of transmitting red light and reflecting blue light and green light. These are formed along the curved surface of the high refraction material layer  511 .  
         [0113]    Note that, in order to facilitate understanding, only rays  518  and  519  are drawn which are emitted from a pupil center of a focusing lens located in a sufficiently distant place with respect to a size of the pixels. Rays emitted from the vicinity of the pupil are incident on the image pickup element with an angle with respect to the rays  518  and  519 . Then, these rays reach the peripheral parts in the photoelectric conversion areas  502  to be photoelectrically converted there.  
         [0114]    Next, behavior of rays for each wavelength will be described. Object light  518  transmitted through a not-shown infrared ray cut filter comes from the upper part of the figure toward a pixel  520  for receiving green light and is incident first on the high refraction material layer  511  functioning as a microlens. Here, the object light  518  is subjected to a converging action and is radiated to the low refraction material layer  510 . Only a green component can be transmitted through the interference filter layer  530 G and reaches the photoelectric conversion area  502 .  
         [0115]    A blue component and a red component reflected by the interference filter layer  530 G are guided to the adjacent pixels. Since this behavior is the same as that of a ray coming into the pixel  520  from an adjacent pixel  521 , a description of the pixel  421  will substitute for that of the behavior. An output from the photoelectric conversion area  502  is inputted into a signal processing circuit as a green component.  
         [0116]    The adjacent pixel  521  is a pixel for receiving red light. Object light  519  transmitted through a not-shown infrared ray cut filter comes to the pixel  521  from the upper part of the figure and is incident first on the high refraction material layer  511  functioning as a microlens. Here, the red light is subjected to a converging action and is radiated to the low refraction material layer  510  through the interference filter layer  530 R. Here, only a red component can be transmitted through the interference filter layer  530 R, and the behavior of the red light thereafter is the same as that of the green light in the pixel  520  described above. A blue component and a green component reflected by the interference filter layer  530 R advance toward the high refraction material layer  511  of the adjacent pixel and then are subjected to total internal reflection because an incident angle of the light on the interface with the air becomes equal to or smaller than a critical angle due to an action of a curvature of the interference filter layer  530 R. The blue light and the green light returned in a direction to the inside of the image pickup element by the total internal reflection are incident on the interference filter layer  530 G. Since the interference filter layer  530 G is originally a filter provided in the pixel  520  for receiving green light, it transmits the green component and reflects the blue component in the green light. The reflected blue light is not illustrated here in the figure because it escapes to the outside of the image pickup element.  
         [0117]    As described above, in an light adjustment area including the interference filter layers  509 R and  509 G and the high refraction material layer  511 , green light incident on the interference filter layer  509 R is incident on the interference filter layer  509 G and transmitted through it.  
         [0118]    The green light component transmitted through the interference filter layer  530 G is reflected on the side of the metal wiring layer  506  before or after incident on the interference filter layer  530 G and reaches the photoelectric conversion area  502  through the color filter layer  530 G. The rays are photoelectrically converted in the photoelectric conversion area  502  together with a green component of the object light  518  and are inputted into a signal processing circuit as an output of the pixel  520 .  
         [0119]    In this way, it is possible to photoelectrically convert an unnecessary wavelength component as an effective wavelength component in an adjacent pixel to improve a utilization ratio of light.  
         [0120]    The interference filter layers  115 R,  115 G,  302 ,  430 R,  430 G,  509 R and  509 G of the first to fifth embodiments described above have the following structure.  
         [0121]    Examples of incorporating an interference filter layer in an image pickup element for color division are disclosed in JP 63-269567 B and JP 09-219505 A. The image pickup element comprises silicon nitride (Si3N4) as a high refraction material and silicon oxide (SiO2) as a low refraction material that are alternately laminated as a dielectric body. Tantalum oxide and zirconium oxide can be used as the high refraction material and magnesium fluoride can be used as the low refraction material.  
         [0122]    Here, silicon nitride (Si3N4) is used as the high refraction material and silicon oxide (SiO2) is used as the low refraction material, λ0/4 is set as a base optical thickness (λ0 is a base wavelength) and eleven layers are laminated to obtain reflectivity of approximately 90% at a predetermined base wavelength. The base wavelength is 620 nm in an R pixel, 550 nm in a G pixel and 460 nm in a B pixel. It is sufficient to form silicon nitride (Si3N4) and silicon oxide (SiO2) by a method such as the chemical vapor deposition (CVD) or the electron beam vapor deposition.  
         [0123]    Next, the color filters  113 R,  113 G,  422 R,  422 G,  530 R and  530 G of the first to fifth embodiments described above have the following structure.  
         [0124]    A rare-earth metal ion for cutting off an intermediate wavelength of R, G and B light is dispersed in the color filters. This is because a wavelength region to be divided shifts by a change in an incident angle of a ray in an interference filter layer. Therefore, a wavelength region reaching a photoelectric-conversion area through the color filters becomes a transmitted wavelength region that is slightly narrower than a transmitted wavelength region of the interference filter.  
         [0125]    As the rare-earth metal ion, there are one or two types such as a neodymium ion, a praseodymium ion, an erbium ion and a holmium ion. It is preferable to use at least the neodymium ion as an essential ion. Note that trivalent ion is usually used as these ions.  
         [0126]    In addition, it is possible to change the interference filter layer to a photonic crystal layer and give it the same characteristics as the color A/D converter for performing analog/digital conversion of the image signal outputted from the image pickup element  4 ;  7 , a signal processing circuit for applying various kinds of correction to image data outputted from the A/D converter  6  or compressing the data;  8 , a timing generation unit for outputting various timing signals to the image pickup element  4 , the image pickup signal processing circuit  5 , the A/D converter  6  and the signal processing circuit  7 ;  9 , a system control and operation unit for controlling various arithmetic operations and the entire still video camera;  10 , a memory unit for temporarily storing image data;  11 , an interface unit for recording data in and reading data from a recording medium;  12 , a detachably attachable recording medium such as a semiconductor memory for recording or reading out image data; and  13 , an interface unit for communicating with an external computer and the like.  
         [0127]    Next, operations of the image pickup apparatus at the time of photographing in the above-mentioned configuration will be described.  
         [0128]    When the barrier  1  is opened, a main power supply is turned on, a power supply of a control system is turned on next, and a power supply of an image pickup system circuit such as the A/D converter  6  is turned on. Then, the system control and filters. In this case, if a photonic band gap structure is adopted such that a photonic band gap (PBG) phenomenon occurs, the transmitted wavelength region does not shift depending on the incident angle of light. The technique disclosed in JP 10-175960 A can be applied for manufacture of a photonic crystal. In this case, it is no more necessary to disperse the rare-earth metal ion for cutting off the intermediate wavelength of the R, G and B light in the color filters.  
         [0129]    Note that, although reflection is used as a method of splitting light in the above-description, refraction may be used as well. In recent years, “super prism effect” for changing a refraction angle extremely sensitive to a wavelength of light utilizing a wavelength dispersity has been reported (H. Kosaka et al., “Super prism phenomena in photonic crystals”, Physical Review B, vol. 58, no. 16, p. R10096, 1998). An effect of a curving angle having remarkable wavelength dependency when a beam of light or a wave flux crosses a boundary from an inside to an outside of a photonic crystal in which materials with different indexes of refraction are arranged periodically or in the opposite direction is called the super prism effect. If this effect is utilized, it is possible to deflect a specific wavelength component in a direction of adjacent pixels. Note that the method disclosed in JP 2000-232258 A can be used for manufacturing a semiconductor photonic crystal.  
         [0130]    Here, in the first to fifth embodiments, the image pickup element may not be provided with any color filter, although a quality of an image of the image pickup element is lower than the above-mentioned case.  
         [0131]    In addition, the image pickup element of the first to fifth embodiments may be an image pickup element of a CCD type or may be an image pickup element of an XY address type such as a CMOS image sensor.  
         [0132]    Sixth Embodiment  
         [0133]    An embodiment in which any one of the image pickup elements of the first to fifth embodiments is applied to an image pickup apparatus (still camera) will be described in detail with reference to FIG. 24.  
         [0134]    In FIG. 24, reference numeral  1  denotes a barrier functioning both as a protect of a lens and a main switch;  2 , a lens for focusing an optical image of a subject on an image pickup element  4 ;  3 , an iris for varying an amount of light that has passed through the lens  2 ;  4 , an image pickup element for capturing the subject image focused by the lens  2  as an image signal;  5 , an image pickup signal processing circuit for processing an image pickup signal;  6 , an operation unit  9  opens the iris  3  in order to control an amount of exposure. A signal outputted from the solid-state image pickup element  4  is converted in the A/D converter  6  and then inputted into the signal processing circuit  7 . Arithmetic operation of exposure is performed in the system control and operation unit  9  based on the data. Brightness is judged according to a result of performing this photometry, and the system control and operation unit  9  controls the iris  3  according to the result.  
         [0135]    Next, a high frequency component is extracted and arithmetic operation of a distance to an object is performed in the system control and operation unit  9  based on the signal outputted from the image pickup element  4 . Thereafter, the lens  2  is driven to judge whether or not the lens  2  is focused and, if it is judged that the lens  2  is not focused, the lens  2  is driven again to perform distance measurement. Then, after focusing is confirmed, main exposure is started.  
         [0136]    When the exposure ends, the image signal outputted from the image pickup element  4  is A/D converted by the A/D converter  6 , passes through the signal processing circuit  7  and is written in the memory unit  10  by the system control and operation unit  9 . Thereafter, the data stored in the memory unit  10  passes through the recording medium control I/F unit  11  and is recorded in the detachably attachable recording medium  12  such as a semiconductor memory by the control of the system control and operation unit  9 . In addition, the data may be directly inputted into a computer or the like through the external I/F unit  13  to process an image.  
         [0137]    As described above, in the image pickup element in which a plurality of pixels are arranged regularly, at least two pixels in one unit constituting the regular arrangement have light splitting means for forming light receiving openings effectively overlapping each other, respectively. Thus, an image pickup apparatus that obtains a high grade image with less moiré without requiring an expensive optical lowpass filter can be realized.  
         [0138]    In addition, since a plurality of pixels having different spectral sensitivity characteristics are arranged regularly in the image pickup element, a utilization ratio of incident light and sensitivity of the image pickup element can be increased. As a result, it has become possible to easily photograph a darker object. When this image pickup element is applied to a digital camera or the like, since a fast shutter speed can be selected, failure of photographing due to hand vibration can be reduced.  
         [0139]    Many widely different embodiments of the present invention may be constructed without departing from the spirit and scope of the present invention. It should be understood that the present invention is not limited to the specific embodiment described in the specification except as defined in the appended claims.

Technology Classification (CPC): 7