Abstract:
To output a color image signal of an arbitrary basic block and facilitate processing on the output side, an image pickup element including a plurality of photodetectors each having a color filter array, a vertical direction selection circuit for selecting in the vertical direction an arbitrary basic block having at least one of the plurality of photodetectors, a horizontal direction selection circuit for selecting the arbitrary basic block in the horizontal direction, and an output circuit for parallelly outputting outputs from the photodetectors in the arbitrary basic block selected by the vertical direction selection circuit and horizontal direction selection circuit is provided.

Description:
BACKGROUND OF THE INVENTION 
   1. Field of the Invention 
   The present invention relates to an image pickup element for outputting an arbitrary image region and an image pickup device for interpolating an arbitrary image region. 
   2. Related Background Art 
   In an image pickup device, to obtain a color image from a single image pickup element, light is incident on the image pickup element through color filter arrays. Color filter arrays are roughly classified into a primary color filter array and a complementary color filter array. In a primary color filter array, three filters, i.e., an R filter for transmitting only red light within the visible light range, a G filter for transmitting only green light within the visible light range, and a B filter for transmitting only blue light within the visible light range are arrayed in, e.g., a matrix. In a complementary color filter array, a cyan (to be referred to as “Cy” hereinafter) filter for shielding only red light within the visible light range, a magenta (to be referred to as “Mg” hereinafter) filter for shielding only green light within the visible light range, and a yellow (to be referred to as “Ye” hereinafter) filter for shielding only blue light within the visible light range are arrayed in, e.g., a matrix. There are various color filter array patterns.  FIG. 1  shows an example of the primary color filter array. This is called a Bayer matrix. 
   The Bayer matrix will be described. Of the basic units of the array pattern of 2×2 color filters, only the upper left pixel can directly output a red signal from a photoelectric conversion element as a detection unit of an image pickup element. Only the upper right and lower left pixels can output a green signal, and only the lower right pixel can output a blue signal. To obtain the respective color signals from these pixels, interpolation processing is required. In interpolation, the value of a pixel to be interpolated is obtained by signal processing or calculation based on the values of the neighboring pixels. 
     FIGS. 2A to 2C  are views for explaining an example of interpolation corresponding to the Bayer matrix shown in  FIG. 1 . Symbol ◯ indicates an original pixel obtained from a photoelectric conversion element. Symbol Δ indicates an interpolated pixel obtained by interpolation. An arrow indicates the relationship between an interpolated pixel and its original pixel. 
     FIG. 3  is a block diagram of a conventional single-plate color image pickup device. Referring to  FIG. 3 , the device comprises a CCD image pickup element  901 , an A/D converter (ADC)  902 , a memory  903 , an interpolation circuit  904 , a signal processing circuit  905 , and a D/A converter (DAC)  906 . The memory  903  requires a capacity corresponding to the number of bits of at least (two lines+two pixels)×ADC. The memory  903  is constituted by, e.g., a FIFO having an output terminal at its middle point. 
   In the CCD image pickup element  901 , a photodetection signal obtained by each photoelectric conversion element is transferred in the vertical direction by a CCD and then transferred in the horizontal direction by a CCD. More specifically, when all photodetection signals of a certain line are transferred by the vertical CCD to reach the horizontal CCD, all the photodetection signals are sequentially transferred in the horizontal direction by the horizonal CCD and output from an output terminal. This operation is sequentially performed for all lines. Therefore, photodetection signals (original pixels) are output from the CCD image pickup element  901  in the order of scanning lines, as shown on the CCD image pickup element  901  in  FIG. 3 . 
   Each original pixel output from the CCD image pickup element  901  is A/D-converted by the ADC  902  and stored in the memory  903 . A plurality of original pixels on each of the first, second, and third lines output from the memory  903  are input to the interpolation circuit  904 . The interpolation circuit  904  interpolates as shown in  FIGS. 2A to 2C  on the basis of these original pixels and outputs RGB signals interpolated by the interpolated pixels. The signal processing circuit  905  processes the RGB signals by color gain adjustment or edge enhancement. The DAC  906  D/A-converts the processed RGB signals to output analog RGB signals. 
   In the above-described prior art, interpolated pixels must be obtained on the basis of original pixels output from the CCD image pickup element  901  in the order of scanning lines. For this purpose, the ADC  902 , memory  903 , and DAC  906  are required in addition to the interpolation circuit  904  and signal processing circuit  905 , resulting in an increase in circuit scale. 
   SUMMARY OF THE INVENTION 
   It is an object of the present invention to provide an image pickup element capable of outputting a signal from a pixel in an arbitrary basic block. 
   It is an another object of the present invention to provide an image pickup element capable of outputting a signal from a pixel in an arbitrary basic block which is appropriate to calculation of interpolated pixel. 
   It is an another object of the present invention to provide an image pickup device capable of reducing circuit scale. 
   It is another object of the present invention to provide an image pickup device capable of obtaining an interpolated pixel without any A/D converter, memory for a plurality of lines, and D/A converter. 
   In order to achieve the above object, according to an aspect of the present invention, there is provided an image pickup element comprising a plurality of photodetectors each having a color filter array, vertical direction selection means for selecting in a vertical direction an arbitrary basic block having at least two of the plurality of photodetectors, horizontal direction selection means for selecting the arbitrary basic block in a horizontal direction, and output means for parallelly outputting outputs from the photodetectors in the arbitrary basic block selected by the vertical direction selection means and the horizontal direction selection means. 
   According to another aspect of the present invention, there is provided an image pickup device comprising: 
   (A) an image pickup element including;
         (a) a plurality of photodetectors each having a color filter array;   (b) vertical direction selection means for selecting in a vertical direction an arbitrary basic block having at least two of the plurality of photodetectors;   (c) horizontal direction selection means for selecting the arbitrary basic block in a horizontal direction; and   (d) output means for parallelly outputting outputs from the photodetectors in the arbitrary basic block selected by the vertical direction selection means and the horizontal direction selection means;       

   (B) block storage means for storing a plurality of outputs from the image pickup elements in units of basic blocks; and 
   (C) interpolation means for calculating an interpolated pixel on the basis of an output from the block storage means. 
   By the image pickup element having the above arrangement, processing on the output side is facilitated. 
   In addition, by the above image pickup device, the circuit scale can be reduced. 
   Other objects, features, and advantages of the present invention will be apparent from the following specification and accompanying drawings. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
       FIG. 1  is a view showing the basic pattern of a color filter array in a Bayer matrix; 
       FIGS. 2A ,  2 B and  2 C are views for explaining an example of interpolation of RGB signals corresponding to the Bayer matrix; 
       FIG. 3  is a block diagram showing the arrangement of a conventional single-plate color image pickup device; 
       FIG. 4  is a block diagram showing the arrangement of a single-plate color image pickup device according to the present invention; 
       FIG. 5  is a first view for explaining interpolation processing according to the first embodiment of the present invention; 
       FIG. 6  is a second view for explaining interpolation processing according to the first embodiment of the present invention; 
       FIG. 7  is a view showing the basic pattern of a complementary color filter array; 
       FIG. 8  is a first view for explaining interpolation processing according to the second embodiment of the present invention; 
       FIG. 9  is a second view for explaining interpolation processing according to the second embodiment of the present invention; 
       FIG. 10  is a third view for explaining interpolation processing according to the second embodiment of the present invention; 
       FIG. 11  is a block diagram showing the arrangement of an image pickup element according to the third embodiment of the present invention; and 
       FIG. 12  is a block diagram showing the arrangement of an image pickup element according to the fourth embodiment of the present invention. 
   

   DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
   In embodiments to be described below, an image pickup element such as so-called CMOS image pickup element capable of reading a pixel at an arbitrary portion at random is used. 
     FIG. 4  is a block diagram showing the arrangement of an image pickup device according to the first embodiment. Referring to  FIG. 4 , the image pickup device comprises an image pickup element  101 , a block memory  102 , an interpolation circuit  103 , and a signal processing circuit  104 . 
   Original pixels are read from the image pickup element  101  in units of 2×2 basic (minimum) blocks of a Bayer matrix. The block memory  102  stores read original pixels of a plurality of basic blocks. In this embodiment, the block memory  102  stores three basic blocks in the horizontal direction and three basic blocks in the vertical direction, i.e., a total of nine basic blocks. The interpolation circuit  103  interpolates as shown in  FIGS. 2A to 2C  on the basis of the original pixels stored in the block memory  102  and outputs interpolated RGB signals. The signal processing circuit  104  processes the interpolated RGB signals by color gain adjustment, low-frequency filtering, or edge enhancement and outputs the processed RGB signals. The signals obtained from the image pickup element  101  and output from the signal processing circuit  104  are analog signals. 
   The block memory  102 , interpolation circuit  103 , and signal processing circuit  104  are formed on one chip together with the image pickup element  101 . For example, when the image pickup element  101  is a CMOS image pickup element, the image pickup element  101 , block memory  102 , interpolation circuit  103 , and signal processing circuit  104  are formed on one chip by the same CMOS process. 
     FIG. 5  is a view showing the array of original pixels stored in the block memory  102 . Referring to  FIG. 5 , symbols A, B, C, D, E, F, G, H, I, J, K, L, M, N, P, and Q denote basic blocks. The block memory  102  simultaneously stores 3×3 basic blocks, as indicated by reference numeral  20  in  FIG. 5 . 
     FIG. 6  is a view for explaining interpolation processing by the interpolation circuit  103 . The interpolation circuit  103  generates interpolated pixels pointed by arrowheads in  FIG. 6  on the basis of the original pixels stored in the block memory  102 , which are indicated by reference numeral  20  in  FIG. 5 , in accordance with the following equations exemplifying the basic block F:
   R   F2 =( R   F1   +R   G1 )/2   R   F3 =( R   F1   +R   J1 )/2   R   F4 =( R   F1   +R   K1 )/2   G   F1 =( G   E2   +G   F2 )/2   G   F4 =( G   F3   +G   G3 )/2   B   F1 =( B   A4   +B   F4 )/2   B   F2 =( B   B4   +B   F4 )/2   B   F3 =( B   E4   +B   F4 )/2 
   When interpolation of the basic block F is complete, the basic blocks D, H, and L are read from the image pickup element  101  and written in the block memory  102 . The basic blocks A, E, and I are erased from the block memory  102 . The interpolation circuit  103  generates the interpolated pixels of the basic block G on the basis of the original pixels of the basic blocks B, C, D, F, G, H, J, K, and L in the same manner as described above. 
   By repeating the block reading and interpolation, the interpolation circuit  103  outputs interpolated RGB signals in the order of, e.g., the basic blocks F, G, H, . . . , J, K, L, . . . , N, P, Q, . . . . However, the basic block output order is not limited to this. For example, when block processing such as calculation of the discrete cosine transforms (DCTs) of 8×8 pixels is to be done on the output side of the signal processing circuit, the signals can be read in the order of the basic blocks F, G, J, K, . . . to continuously output 8×8 pixel data. The basic block size may be equal to the block processing size. 
   An image pickup device according to the second embodiment has the same arrangement as that of the first embodiment shown in  FIG. 4 . In the second embodiment, a complementary color filter array is used as a color filter array. The arrangement of the image pickup device of the second embodiment is the same as that of the first embodiment shown in  FIG. 4 , and a detailed description thereof will be omitted. 
     FIG. 7  is a view showing the basic pattern of the complementary color filter array in this embodiment. Referring to  FIG. 7 , the basic pattern has a size of 4 lines×2 pixels. 
   The basic block size in block reading of this embodiment is the same as in the first embodiment, i.e., 2×2 pixels. The basic pattern of the color filter array is different from the basic block in block reading. 
     FIG. 8  is a view showing the array of original pixels stored in a block memory  102 . Referring to  FIG. 8 , symbols A, B, C, D, E, F, G, H, I, J, K, L, M, N, P, and Q denote basic blocks. The block memory  102  simultaneously stores 3×3 basic blocks, as indicated by reference numeral  40  or  50  in  FIG. 8 . 
     FIGS. 9 and 10  are views for explaining interpolation processing by an interpolation circuit  103 .  FIG. 9  shows interpolation processing for basic blocks at an upper portion  201  of the basic pattern shown in  FIG. 7 .  FIG. 10  shows interpolation processing for basic blocks at a lower portion  202  of the basic pattern shown in  FIG. 7 . Referring to  FIG. 8 , the basic blocks A, B, C, D, I, J, K, and L correspond to the upper portion  201 , and the basic blocks E, F, G, H, M, N, P, and Q correspond to the lower portion  202 . 
   For the basic blocks at the upper portion  201 , the interpolation circuit  103  generates interpolated pixels pointed by arrowheads in  FIG. 9  on the basis of the original pixels stored in the block memory  102 , which are indicated by reference numeral  40  in  FIG. 8 , in accordance with the following equations exemplifying the basic block J:
 
 Cy   J2 =(Cy J1 +Cy K1 )/2
 
 Cy   J3 =(Cy J1 +Cy N1 )/2
 
 Cy   J4 =(Cy J1 +Cy P1 )/2
 
 Ye   J1 =(Ye I2 +Ye J2 )/2
 
 Ye   J3 =(Ye J2 +Ye M2 )/2
 
 Ye   J4 =( Ye   J2   +Ye   N2 )/2
 
 Mg   J3 =( Mg   I4   +Mg   J4 )/2
 
 Mg   J1 =( Mg   F3   +Mg   J3 )/2
 
 Mg   J2 =( Mg   G3   +Mg   J3 )/2
 
 G   J4 =( G   J3   +G   K3 )/2
 
 G   J1 =( G   E4   +G   J4 )/2
 
 G   J2 =( G   F4   +G   J4 )/2
 
   For the basic blocks at the lower portion  202 , the interpolation circuit  103  generates interpolated pixels pointed by arrowheads in  FIG. 10  on the basis of the original pixels stored in the block memory  102 , which are indicated by reference numeral  50  in  FIG. 8 , in accordance with the following equations exemplifying the basic block F:
 
 Cy   F2 =( Cy   F1   +Cy   G1 )/2
 
 Cy   F3 =( Cy   F1   +Cy   I1 )/2
 
 Cy   F4 =( Cy   F1   +Cy   K1 )/2
 
 Ye   F1 =( Ye   E2   +Ye   F2 )/2
 
 Ye   F3 =( Ye   F2   +Ye   J2 )/2
 
 Ye   F4 =( Ye   F2   +Ye   J2 )/2
 
 Mg   F4 =( Mg   F3   +Mg   G3 )/2
 
 Mg   F1 =( Mg   A4   +Mg   F4 )/2
 
 Mg   F2 =( Mg   B4   +Mg   F4 )/2
 
 G   F3 =( G   E4   +G   F4 )/2
 
 G   F1 =( G   B3   +G   F3 )/2
 
 G   F2 =( G   C3   +G   F3 )/2
 
   The interpolation circuit  103  switches the two interpolation modes in accordance with the type of basic blocks. The original pixels are read from an image pickup element  101  and written in the block memory in the same manner as in the first embodiment. 
   By repeating the block reading and interpolation, the interpolation circuit  103  outputs interpolated CyMgYe signals in the order of, e.g., the basic blocks F, G, H, . . . , J, K, L, . . . , N, P, Q, . . . . However, the basic block output order is not limited to this. For example, when the DCTs of 8×8 pixels are to be calculated on the output side of the signal processing circuit, the signals can be read in the order of the basic blocks F, G, J, K, . . . to continuously output 8×8 pixel data. 
   The arrangement of the image pickup element  101  will be described next.  FIG. 11  is a block diagram showing the arrangement of an image pickup element  101  in the third embodiment. 
   Referring to  FIG. 11 , the image pickup element comprises photodetectors  301  such as photodiodes, a vertical direction (row direction) read-out block selection circuit  302 , a horizontal direction (column direction) read-out block selection circuit  303 , transfer switches  304 , vertical direction block selection lines  305 , horizontal direction block selection lines  306 , and output lines  307 . The output lines are connected to an output terminal. 
   For the vertical direction block selection lines  305  equal in number to the basic blocks in the vertical direction, the vertical direction read-out block selection circuit  302  activates only lines of selected blocks. Similarly, for the horizontal direction block selection lines  306  equal in number to the basic blocks in the horizontal direction, the horizontal direction read-out block selection circuit  303  activates only lines of selected blocks. Each photodetector  301  outputs a detection signal only when both the vertical direction block selection line  305  and horizontal direction block selection line  306  are activated. Each of the transfer switches  304  equal in number to the basic blocks in the horizontal direction receives the detection signal when any one of the basic blocks  301  of a line to which the transfer switches  304  belong outputs a detection signal. The outputs from the transfer switches  304  are coupled and sent to the output terminal. Since one of the outputs is selected by the horizontal direction block selection lines  306 , only the detection signal from the basic block selected by the vertical direction block selection line  305  and horizontal direction block selection line  306  is output from the output terminal. 
   When the vertical direction read-out block selection circuit  302  and horizontal direction read-out block selection circuit  303  are simultaneously operated, the detection signal from an arbitrary basic block can be output from the output terminal. 
     FIG. 12  is a block diagram showing the arrangement of an image pickup element  101  according to the fourth embodiment. 
   Referring to  FIG. 12 , the device comprises photodetectors  401  such as photodiodes, a vertical direction (row direction) read-out block selection circuit  302 , a horizontal direction (column direction) read-out block selection circuit  303 , a two-line memory  402 , vertical direction photodetector selection lines  403 , horizontal direction block selection lines  306 , and output lines  307 . The output lines are connected to an output terminal. The vertical direction read-out block selection circuit  302 , horizontal direction read-out block selection circuit  303 , vertical direction block selection lines  305 , and horizontal direction block selection lines  306  are the same as those of the third embodiment. 
   For the vertical direction photodetector selection lines  403  equal in number to the photodetectors in the vertical direction, the vertical direction read-out block selection circuit  302  activates only lines of selected blocks. Since the outputs from photodetectors on one line are output to the same line, the lines are sequentially activated. For the horizontal direction block selection lines  306  equal in number to the basic blocks in the horizontal direction, the horizontal direction read-out block selection circuit  303  activates only lines of selected blocks. Each photodetector  401  outputs a detection signal only when a corresponding vertical direction photodetector selection line  403  is activated. The two-line memory  402  sequentially receives the outputs from basic blocks selected in the vertical direction. The two-line memory  402  outputs the signal of a basic block selected by a horizontal direction block selection line  306 . Hence, only the detection signals from basic blocks selected by the vertical direction photodetector selection lines  403  selected by the vertical direction read-out block selection circuit  302 , and the horizontal direction block selection lines  306 , are output from the output terminal. 
   When the vertical direction read-out block selection circuit  302  and horizontal direction read-out block selection circuit  303  are simultaneously operated, the detection signal from an arbitrary basic block can be output from the output terminal. 
   As has been described above, according to the first to fourth embodiments, since photodetection signals are read from the image pickup elements in units of basic blocks, basic blocks necessary for interpolation processing are stored in the block memory, and interpolated pixels of the respective colors are obtained on the basis of the photodetection signals stored in the block memory, an interpolated signal of each color in an arbitrary region can be obtained at random. 
   In addition, since all signal processing operations can be performed as analog processing, the A/D and D/A converters can be omitted. Therefore, the circuit scale can be reduced. 
   Furthermore, since the image pickup element, block memory, interpolation circuit, and signal processing circuit can be formed on one chip by the same process, a one-chip image pickup device can be realized. 
   Many widely different embodiments of the present invention may be constructed without departing from the spirit and scope of the present invention. It should be understood that the present invention is not limited to the specific embodiments described in the specification, except as defined in the appended claims.