Signal processing apparatus for solid-state imaging device, signal processing method, and imaging system

A signal processing apparatus corrects color mixture between pixel cells in a solid-state imaging device in which the pixel cells including photoelectric transducers are two-dimensionally arranged in an array and in which color filters having primary color components for generating luminance components and other color components are arranged over the pixel cells. The signal processing apparatus includes correction processing means for performing the correction to the signal from a target pixel by using the signals from multiple neighboring pixels adjacent to the target pixel in the solid-state imaging device and correction parameters independently set for the signals.

CROSS REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese Patent Application JP 2005-332308 filed in the Japanese Patent Office on Nov. 17, 2005, the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to signal processing apparatuses for solid-state imaging devices, signal processing methods, and imaging systems. More particularly, the present invention relates to a signal processing apparatus capable of correcting color mixture in the pixel area of a solid-state imaging device, a signal processing method in the signal processing apparatus, and an imaging system including the signal processing apparatus.

2. Description of the Related Art

Solid-state imaging devices, such as charge coupled device (CCD) image sensors and complementary metal oxide semiconductor (CMOS) image sensors, have structures in which condenser microlenses are layered on color separation filters that are layered on pixel cells including photoelectric transducers.

In color solid-state imaging devices having the above structures, the distances between the pixel areas and the microlenses are increased because of the color filters located between the pixel areas and the microlenses. The distance between the pixels, that is, the pixel pitch decreases with decreasing size of the pixel cells involved in an increase in number of the pixels. Accordingly, light transmitting through the color filter for a certain pixel cell can enter neighboring pixel cells to cause a problem of color mixture.

In order to resolve the problem of the color mixture caused by the decreased size of the pixel cells, in solid-state imaging devices in related art in which the pixels having the three primary colors including red (R), green (G), and blue (B) are arranged in a checker pattern, signal components corresponding to a certain ratio are subtracted from the signal of a pixel having a given color (for example, refer to Japanese Unexamined Patent Application Publication No. 2004-135206). The signal components corresponding to the certain ratio are calculated from the signals of pixels that are adjacent to the pixel having the given color and that have a color other than the given color.

SUMMARY OF THE INVENTION

It has been considered that the colors of multiple neighboring pixels adjacent to a target pixel are isotropically mixed into the target pixel in the pixel area, that is, the color mixture from the multiple neighboring pixels occurs in the same ratio in the pixel area. Under this consideration, the same correction parameter is used for the multiple neighboring pixels to resolve the problem of the color mixture in the related art disclosed in, for example, Japanese Unexamined Patent Application Publication No. 2004-135206.

However, since the photoelectric transducers (photosensitive sections) can be shifted from the centers of the pixel cells depending on the layout of the circuit sections, the wiring, or the signal reading sections in actual solid-state imaging devices, the physical center of each pixel cell does not necessarily coincide with the optical center thereof. Accordingly, the color mixture from the neighboring pixels into the target pixel does not necessarily occur isotropically but occurs directionally.

As a result, in related art disclosed in, for example, Japanese Unexamined Patent Application Publication No. 2004-135206, in which the same correction parameter is used for multiple neighboring pixels to correct the color mixture, it is not possible to correct the color mixture in accordance with the degree of the color mixture from the neighboring pixels into the target pixel, that is, it is not possible to correct the color mixture with directionality.

It is desirable to provide a signal processing apparatus for a solid-state imaging device, a signal processing method, and an imaging system, capable of realizing the correction of the color mixture with directionality.

According to an embodiment of the present invention, a signal processing apparatus correcting color mixture between pixel cells in a solid-state imaging device in which the pixel cells including photoelectric transducers are two-dimensionally arranged in an array and in which color filters having primary color components for generating luminance components and other color components are arranged over the pixel cells includes correction processing means for performing the correction to the signal from a target pixel by using the signals from multiple neighboring pixels adjacent to the target pixel in the solid-state imaging device and correction parameters independently set for the signals.

According to another embodiment of the present invention, a signal processing method of correcting color mixture between pixel cells in a solid-state imaging device in which the pixel cells including photoelectric transducers are two-dimensionally arranged in an array and in which color filters having primary color components for generating luminance components and other color components are arranged over the pixel cells includes the step of performing the correction to the signal from a target pixel by using the signals from multiple neighboring pixels adjacent to the target pixel in the solid-state imaging device and correction parameters independently set for the signals.

According to another embodiment of the present invention, an imaging system includes a solid-state imaging device in which pixel cells including photoelectric transducers are two-dimensionally arranged in an array and in which color filters having primary color components for generating luminance components and other color components are arranged over the pixel cells; an optical system through which light from a subject is led to the solid-state imaging device; and correction processing means for performing correction to the signal from a target pixel by using the signals from multiple neighboring pixels adjacent to the target pixel in the solid-state imaging device and correction parameters independently set for the signals.

Since the correction parameters for the signals from the multiple neighboring pixels are independent of each other in the correction of the color mixture between the pixels in the solid-state imaging device, the amount of correction of the color mixture from the neighboring pixels into the target pixel can be arbitrarily set for every neighboring pixel by using the independent correction parameters. As a result, it is possible to give the directionality to the amount of correction of the color mixture from the neighboring pixels into the target pixel, that is, to set different amounts of correction for different neighboring pixels.

According to the present invention, the directionality can be given to the amount of correction of the color mixture from the neighboring pixels into the target pixel, so that the color mixture can be corrected in accordance with the degree of the color mixture from the neighboring pixels into the target pixel. Consequently, it is possible to realize the correction of the color mixture in accordance with the directionality even if the color mixture from the neighboring pixels into the target pixel has the directionality.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1is a block diagram showing an example of the structure of an imaging system, such as a video camera, according to an embodiment of the present invention. Although the present invention is embodied by the video camera, the present invention is not limited to the application to the video camera. The present invention is applicable to other imaging systems including a digital still camera.

Referring toFIG. 1, the video camera according to this embodiment of the present invention includes an optical system1, a CMOS image sensor2, which is a solid-state imaging device, an analog front end (AFE)3, a digital signal processing circuit4, a camera controller5, a human interface (I/F) controller6, a user interface7, a timing generator8, an optical system driving circuit9, and a camera shaking sensor10.

The optical system1includes a lens1athrough which incident light from a subject (not shown) forms an image on the imaging surface of the CMOS image sensor2and an aperture1bcontrolling the light intensity of the incident light through the lens1a. The CMOS image sensor2performs photoelectric conversion to the incident light through the optical system1in units of pixels to output an analog electrical signal. The structure of the CMOS image sensor2will be described in detail below. The electrical signal is output from the CMOS image sensor2through multiple channels, for example, four channels.

The AFE3is an analog signal processing circuit. After performing signal processing including sample/hold (S/H) and automatic gain control (AGC) to the analog signal supplied from the CMOS image sensor2through the four channels, the AFE3performs analog-to-digital (A/D) conversion to the analog signal and supplies the digital signal to the digital signal processing circuit4. The digital signal processing circuit4performs a variety of signal processing to the digital signal supplied from the AFE3through the four channels in accordance with instructions from the camera controller5.

The signal processing performed in the digital signal processing circuit4includes so-called camera signal processing, such as white balancing, gamma correction, and color difference processing, and a calculation process of detected data (data indicating information, such as the brightness, contrast, and hue, in the screen) used for controlling the camera. The digital signal processing circuit4includes a color-mixture correction circuit11, by which the present invention is characterized, in addition to the circuits performing the above signal processing. The color-mixture correction circuit11will be described in detail below.

The camera controller5is, for example, a microcomputer. The camera controller5acquires the state of the current input image on the basis of the detected data supplied from the digital signal processing circuit4and information concerning camera shaking supplied from the camera shaking sensor10, and performs the camera control in accordance with various setting modes supplied through the human I/F controller6. The camera controller5supplies the processed data to the digital signal processing circuit4as camera image control data, to the optical system driving circuit9as lens control data or aperture control data, to the timing generator8as timing control data, and to the AFE3as gain control data.

The digital signal processing circuit4, the optical system driving circuit9, the timing generator8, and the AFE3perform desired signal processing, driving of the optical system1, timing generation, and gain processing, respectively, in accordance with the control values supplied from the camera controller5. The CMOS image sensor2sequentially retrieves signals in an arbitrary area from a pixel array unit, described below, in response to various timing signals generated by the timing generator8and supplies the retrieved signals to the AFE3.

Menu operation and others performed by a user are controlled by the human I/F controller6through the user interface7. The human I/F controller6is, for example, a microcomputer. The human I/F controller6detects which shooting mode the user currently selects or which control the user wants and supplies information concerning an instruction from the user to the camera controller5. Conversely, the camera controller5supplies the camera control information including the distance between the subject and the camera, the f-number, the shutter speed, and the magnification to the human I/F controller6to indicate the current camera information to the user through the user interface7.

The color-mixture correction circuit11by which the present invention is characterized has slightly different configurations or slightly differently operates depending on how pixels are arranged in the CMOS image sensor2or the color coding of the color separation filter. The arrangement of the pixels in the CMOS image sensor2, the color coding of the color separation filter, and the configuration and operation of the color-mixture correction circuit11corresponding to the arrangement of the pixels in the CMOS image sensor2and the color coding of the color separation filter will be described in first to third embodiments of the present invention.

First Embodiment

FIG. 2shows an example of the color coding of the CMOS image sensor according to a first embodiment of the present invention.

The CMOS image sensor has a pixel array unit in which pixel cells21including the photoelectric transducers are two-dimensionally arranged in an array. As shown inFIG. 2, in the CMOS image sensor according to the first embodiment of the present invention, the pixel cells21are arranged at an angle of 45° with respect to the pixel array in a square lattice, typified by the common pixel array in a checker pattern.

The CMOS image sensor according to the first embodiment of the present invention has pixel shifted arrangement in which the pixels are shifted by half of the pixel pitch √2d for every row and column, where the distance (hereinafter referred to as “pixel pitch”) between the pixels in the pixel array in the square lattice is denoted by “d” and the horizontal and vertical pixel pitch with respect to the pixel pitch “d” is denoted by “√2d”. Specifically, the pixels in odd-numbered rows are horizontally (in the direction in which the columns are arranged) shifted from the pixels in even-numbered rows by half of the pixel pitch √2d, and the pixels in odd-numbered columns are vertically (in the direction in which the rows are arranged) shifted from the pixels in even-numbered columns by half of the pixel pitch √2d.

In the color coding of the color separation filter in the above pixel shifted arrangement, the first line is a GR line in which G pixels and R pixels are alternately arranged, the second line is a G line in which only G pixels are arranged, the third line is a GB line in which B pixels and G pixels are alternately arranged, and the fourth line is a G line in which only G lines are arranged. The above four lines are repeated in units of four lines in the subsequent lines in the color coding of the color separation filter.

In the color coding of the color separation filter, as apparent fromFIG. 2, the primary color components (G components in this example) for generating the luminance (Y) components are arranged so as to surround the other color components (R and B components in this example). The R components and B components are horizontally and vertically arranged at intervals of 2√2d.

In this color coding, the horizontal and vertical sampling rate of the G components is d/√2 and the horizontal and vertical sampling rate of the R and B components is 2√2d. In other words, the R and B components are arranged every two columns (odd-numbered columns in this example) and every two rows (odd-numbered rows in this example) such that the horizontal and vertical sampling rate is equal to one fourth of that of the G components. Accordingly, the G components have the horizontal and vertical resolution four times higher than that of the R and B components. When the sampling rate is yielded at an angle of 45°, the sampling rate of the G components is equal to “d” and the sampling rate of the R and B components is equal to “2d”.

Special frequency characteristics will now be considered. Since the sampling rate of the G components is d/√2 in the horizontal and vertical directions, it is possible to sample signals having a frequency of as much as (1/√2)fs according to the sampling theorem. Since the sampling rate of the G components is “d” at an angle of 45°, it is possible to sample signals having a frequency of as much as (¼)fs according to the sampling theorem.

Since the R components are arranged at the same interval as the B components, the R components can be considered in the same manner as in the B components. Accordingly, only the R components will be described here.

In terms of the special frequency characteristics of the R components, since the sampling rate of the R components is 2√2d in the horizontal and vertical directions, it is possible to sample signals having a frequency of as much as (¼√2)fs according to the sampling theorem. Since the sampling rate of the R components is “2d” at an angle of 45°, it is possible to sample signals having a frequency of as much as (½)fs according to the sampling theorem.

Adopting the color coding in which the primary color components (G components in this example) for generating the luminance (Y) components are arranged so as to surround the other color components (R and B components in this example) in the pixel shifted arrangement allows the G components to exist in all the rows and columns. Since the special frequency characteristics of the G components, to which human beings have higher visibility, can be improved, the resolution of not only subjects with achromatic colors but also subjects with chromatic colors can be increased. In addition, since the need to balance the levels of the RGB components is eliminated, there is the advantage of no color fault.

The pixel shifted arrangement has the following advantages, compared with the pixel array in a square lattice. Since the pixel shifted arrangement has a pixel pitch smaller than that of the pixel array in a square lattice, the pixel shifted arrangement can provide a higher resolution. If the pixel shifted arrangement has the same resolution as that of the pixel array in a square lattice, the pixel cells can be arranged at a pixel pitch greater than that of the pixel array in a square lattice. Accordingly, the opening of the pixel cells can be widened, thus improving the S/N ratio.

FIG. 3is a block diagram schematically showing an example of the structure of a CMOS image sensor20A according to the first embodiment of the present invention.

Referring toFIG. 3, the CMOS image sensor20A includes a pixel array unit22in which the pixel cells21including the photoelectric transducers have the pixel shifted arrangement, and adopts the color coding in which the G components are arranged so as to surround the R and B components in the pixel shifted arrangement. In the CMOS image sensor20A, each pixel drive line23is commonly wired to the pixel cells21in a horizontal zigzag row in units of two rows and a vertical scanning circuit24sequentially selects and scans the pixel drive lines23.

The signals from the pixel cells21in the horizontal zigzag row, which are selected and scanned by the vertical scanning circuit24through the pixel drive lines23, are held in column processing circuits26provided for every pixel column through vertical signal lines25wired for every pixel column. The signals corresponding to one row (horizontal zigzag row), held in each column processing circuit26, are sequentially output to four horizontal signal lines29-1to29-4in units of four pixels through horizontal selector switches28sequentially selected by a horizontal scanning circuit27in units of four switches.

As described above, in the CMOS image sensor20A according to the first embodiment of the present invention, the signals in units of multiple neighboring pixels, for example, four neighboring pixels for every row are read out in parallel through the four horizontal signal lines29-1to29-4by using multiple channels (four channels in this example) and are horizontally scanned over one screen. After all the signals during one horizontal scanning period (1H) have been read out, the signals in the subsequent row are horizontally scanned to read out the signals from the pixels over one screen. According to the first embodiment of the present invention, one row means one horizontal zigzag row.

For convenience, sixteen R, G, and B pixels in the first to eighth columns in the first and second rows in the color coding shown inFIG. 2are defined as shown inFIG. 4. Specifically, in the first row, the pixel R among the four pixels in the first unit is defined as a pixel R1and the pixel R among the four pixels in the second unit is defined as a pixel R2. In the second row, the pixel B among the four pixels in the first unit is defined as a pixel B1and the pixel B among the four pixels in the second unit is defined as a pixel B2.

In the first row, the pixel G among the four pixels in the first unit, in contact with the pixels R1and B1with sides, is defined as a pixel Ggo1; the pixel G among the four pixels in the first unit, in contact with pixels R1and R2with apices, is defined as a pixel Gr1; and the pixel G among the four pixels in the first unit, in contact with the pixels B1and R2with sides, is defined as a pixel Gge1. Also in the first row, the pixel G among the four pixels in the second unit, in contact with the pixels R2and B2with sides, is defined as a pixel Ggo2; the pixel G among the four pixels in the second unit, in contact with the pixels R2and (R3) with apices, is defined as a pixel Gr2; and the pixel G among the four pixels in the second unit, in contact with the pixels B2and (R3) with sides, is defined as a pixel Gge2.

In the second row, the pixel G among the four pixels in the first unit, in contact with the pixels R1and B1with apices, is defined as a pixel Gb1; the pixel G among the four pixels in the first unit, in contact with the pixel B1with a side, is defined as a pixel Ggo1; and the pixel G among the four pixels in the first unit, in contact with the pixel B1with a side, is defined as a pixel Gge1. Also in the second row, the pixel G among the four pixels in the second unit, in contact with the pixels B1, R2, and B2with apices, is defined as a pixel Gb2; the pixel G among the four pixels in the second unit, in contact with the pixel B2with a side, is defined as a pixel Ggo2; and the pixel G among the four pixels in the second unit, in contact with the pixel B2with a side, is defined as a pixel Gge2.

Under the above definition, the signals in units of the four neighboring pixels for every row (horizontal zigzag row) are read out in parallel through the four channels in one clock cycle of a clock signal, on which the operation of the CMOS image sensor20A is based, to output an R/Gb signal, a Gr/B signal, a Gge signal, and a Ggo signal through the four channels, as shown inFIG. 4.FIG. 5shows a sequence of the output signals through the channels in the above readout method.

The present invention does not depend on the readout method. Even if the number of the channels or the readout method is varied, the present invention is applicable to all the cases only by building subsequent processing in accordance with the channels through which the signals are read out or in accordance with the readout method (only by incorporating a mechanism corresponding to the varied sequence).

For simplicity, the above readout method is exemplified in which the signals in units of the four neighboring pixels for every row (horizontal zigzag row) are read out in parallel through the four channels in one clock cycle.

Digital Signal Processing Circuit

FIG. 6is a block diagram showing an example of the configuration of the digital signal processing circuit4. Referring toFIG. 6, the digital signal processing circuit4includes a camera signal processing circuit41, a communication I/F42, and a signal generator43. The R/Gb, Gr/B, Ggo, and Gge signals are supplied in parallel from the CMOS image sensor2to the digital signal processing circuit4through the AFE3by using the four channels.

The camera signal processing circuit41performs a variety of camera signal processing, such as, digital clamp, noise reduction, defect correction, demosaicing (interpolation), white balancing, and resolution conversion, to the R/Gb, Gr/B, Ggo, and Gge signals through the four channels, in parallel, in response to instructions supplied from the camera controller5through the communication I/F42on the basis of various timing signals supplied from the signal generator43. Then, the camera signal processing circuit41supplies the processed signals to a video system processing block as Y (luminance) and C (chroma) signals. Since the camera signal processing does not directly relate to the present invention, a detailed description of the camera signal processing is omitted herein.

Camera Signal Processing Circuit

The camera signal processing circuit41includes a color-mixture correction circuit11to which the present invention is applied.FIG. 7is a block diagram showing an example of the internal configuration of the camera signal processing circuit41.

Referring toFIG. 7, the camera signal processing circuit41includes a first camera signal processor group411and a second camera signal processor group412, in addition to the color-mixture correction circuit11. The first camera signal processor group411is provided upstream of the color-mixture correction circuit11and the second camera signal processor group412is provided downstream of the color-mixture correction circuit11. The camera signal processing circuit41processes, in parallel, the R/Gb, Gr/B, Ggo, and Gge signals for every four pixels, shown inFIG. 4, in one clock of the clock signal on which the camera signal processing is based.

In the camera signal processing circuit41inFIG. 7, the first camera signal processor group411performs the digital clamp, the defect correction, and the noise reduction to the signals and supplies the processed signals to the color-mixture correction circuit11to which the present invention is applied. The first camera signal processor group411is a processor group that performs various correction processes before camera YC processing. After the second camera signal processor group412performs the demosaicing (interpolation), the second camera signal processor group412generates the luminance and chroma signals in the YC processing. Finally, the second camera signal processor group412performs the resolution conversion and supplies the processed signals having a size suitable for the format to the downstream video system processing block.

The various timing signals are distributed from the signal generator43to all the circuit blocks including the color-mixture correction circuit11in the camera signal processing circuit41. Each circuit block generates timings necessary for the various processes on the basis of the various timing signals. The operations of all the circuit blocks are controlled by the camera controller5through the communication I/F42.

Color-Mixture Correction Circuit

FIG. 8is a block diagram showing an example of the configuration of the color-mixture correction circuit11. Referring toFIG. 8, the color-mixture correction circuit11includes a line memory group111, a memory controller112, and a correction block113.

The line memory group111is provided for the R/Gb, Gr/B, Ggo, and Gge signals from the pixels through the four channels. The line memory group111includes line memories111-1,111-2,111-3, and111-4for causing delays in units of rows. The line memories111-1,111-2,111-3, and111-4form, for example, a single port static random access memory (SRAM). The line memories111-1and111-2are 1H (“H” denotes one horizontal scanning period) delay memories and the line memories111-3and111-4are 2H delay memories.

The memory controller112controls writing in and readout from the line memories111-1,111-2,111-3, and111-4on the basis of the various timing signals supplied from the signal generator43shown inFIG. 7. The correction block113corrects the color mixture of the pixels in response to a control signal supplied through the communication I/F42shown inFIG. 7.

In the color-mixture correction circuit11, the R/Gb, Gr/B, Ggo, and Gge signals from the pixels through the four channels are input in the line memories111-1,111-2,111-3, and111-4in parallel, respectively. A group of signals having 0H to 2H delays for every channel is generated and output in response to a writing enabling signal WEN, a writing address signal WADRS, a readout enabling signal REN, and a readout address signal RADRS, supplied from the memory controller112.

The line memory group111is structured such that a no-delay signal (Sig_R_Gb_0h) and a 1H-delay signal (Sig_R_Gb_1h) are output through the R/Gb channel, a no-delay signal (Sig_Gr_B_0h) and a 1H-delay signal (Sig_Gr_B_1h) are output through the Gr/B channel, a 1H-delay signal (Sig_Ggo_1h) and a 2H-delay signal (Sig_Ggo_2h) are output through the Ggo channel, and a 1H-delay signal (Sig_Gge_1h) and a 2H-delay signal (Sig_Gge_2h) are output through the Gge channel.

The group of the signals delayed by the line memories111-1,111-2,111-3, and111-4is supplied to the correction block113. The correction block113corrects the color mixture of the pixels in parallel for every channel in accordance with the control signal supplied through the communication I/F42and supplies Sig_R_Gb′, Sig_Gr_B′, Sig_Ggo′, and Sig_Gge′ signals after the correction to the downstream blocks. The correction block113includes four correction sub-blocks provided for the R/Gb, Gr/B, Ggo, and Gge signals from the pixels through the four channels.

FIG. 9is a block diagram showing an example of the configuration of an R/Gb channel correction sub-block113A. The R/Gb channel correction sub-block113A includes a correction circuit30and five delay circuits31to35. The 1H-delay signal Sig_R_Gb_1h, the 2H-delay signal Sig_Gge_2h, the 2H-delay signal Sig_Ggo_2h, the 1H-delay signal Sig_Gge_1h, and the 1H-delay signal Sig_Ggo_1h, among the total of eight signals output from the line memory group111inFIG. 8, the two signals being output for every channel, are input in the R/Gb channel correction sub-block113A.

The correction circuit30is shared between the channels. The circuit configuration of the correction circuit30will be described in detail below. The delay circuit31delays the 1H-delay signal Sig_R_Gb_1hby one clock cycle of the clock signal having a pixel period on which the correction of the color mixture is based and supplies the delayed signal to the correction circuit30as a signal from a correction object pixel. In the R/Gb channel correction sub-block113A, the R/Gb pixel is the correction object pixel, as shown inFIG. 10.

The delay circuit32delays the 2H-delay signal Sig_Gge_2hby two clock cycles and supplies the delayed signal to the correction circuit30as a signal from an upper left pixel (a) in contact with the correction object pixel R/Gb with a side. The delay circuit33delays the 2H-delay signal Sig_Ggo_2hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from an upper right pixel (b) in contact with the correction object pixel R/Gb with a side.

The delay circuit34delays the 1H-delay signal Sig_Gge_1hby two clock cycles and supplies the delayed signal to the correction circuit30as a signal from a lower left pixel (c) in contact with the correction object pixel R/Gb with a side. The delay circuit35delays the 1H-delay signal Sig_Ggo_1hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from a lower right pixel (d) in contact with the correction object pixel R/Gb with a side.

As described above, when the 1H-delay signal Sig_R_Gb_1h, the 2H-delay signal Sig_Gge_2h, the 2H-delay signal Sig_Ggo_2h, the 1H-delay signal Sig_Gge_1h, and the 1H-delay signal Sig_Ggo_1hpass through the delay circuits31to35to cause the signal from the correction object pixel R/Gb to be a signal having a 1H delay plus one clock cycle delay, the signals from the four neighboring pixels diagonally adjacent to the correction object pixel R/Gb are extracted and the extracted signals are supplied to the correction circuit30along with the signal from the correction object pixel R/Gb.

Correction parameters Ka, Kb, Kc, and Kd and a control signal indicating whether the correction is turned on or off are supplied to the correction circuit30through the communication I/F42shown inFIG. 7. The correction parameters Ka, Kb, Kc, and Kd have independent values (amounts of correction). The correction parameters Ka, Kb, Kc, and Kd are set by the camera controller5and are supplied to the correction circuit30through the communication I/F42. Accordingly, the camera controller5serves as setting means. The control signal indicating whether the correction is turned on or off instructs whether the correction of the color mixture is performed in the system.

If the control signal instructs turning on of the correction, the correction circuit30performs the correction of the color mixture to the signal from the correction object pixel R/Gb by using the correction parameters Ka, Kb, Kc, and Kd on the basis of the signals from the four neighboring pixels in contact with the correction object pixel R/Gb with sides. Since the 1H-delay signal Sig_R_Gb_1his selected as an input signal to the correction circuit30, the signal Sig_R_Gb′ from the correction object pixel R/Gb after the correction is delayed by 1H with respect to the input signal to the color-mixture correction circuit11.

FIG. 11is a block diagram showing an example of the configuration of a Gr/B channel correction sub-block113B. The Gr/B channel correction sub-block113B includes five delay circuits36to40, in addition to the correction circuit30as in the R/Gb channel correction sub-block113A. The 1H-delay signal Sig_Gr_B_1h, the 2H-delay signal Sig_Ggo_2h, the 2H-delay signal Sig_Gge_2h, the 1H-delay signal Sig_Ggo_1h, and the 1H-delay signal Sig_Gge_1h, among the total of eight signals output from the line memory group111inFIG. 8, the two signals being output for every channel, are input in the Gr/B channel correction sub-block113B.

The delay circuit36delays the 1H-delay signal Sig_Gr_B_1hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from a correction object pixel. In the Gr/B channel correction sub-block113B, the Gr/B pixel is the correction object pixel, as shown inFIG. 12.

The delay circuit37delays the 2H-delay signal Sig_Ggo_2hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from an upper left pixel (a) in contact with the correction object pixel Gr/B with a side. The delay circuit38delays the 2H-delay signal Sig_Gge_2hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from an upper right pixel (b) in contact with the correction object pixel Gr/B with a side.

The delay circuit39delays the 1H-delay signal Sig_Ggo_1hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from a lower left pixel (c) in contact with the correction object pixel Gr/B with a side. The delay circuit40delays the 1H-delay signal Sig_Gge_1hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from a lower right pixel (d) in contact with the correction object pixel Gr/B with a side.

As described above, when the 1H-delay signal Sig_Gr_B_1h, the 2H-delay signal Sig_Ggo_2h, the 2H-delay signal Sig_Gge_2h, the 1H-delay signal Sig_Ggo_1h, and the 1H-delay signal Sig_Gge_1hpass through the delay circuits36to40to cause the signal from the correction object pixel Gr/B to be a signal having a 1H delay plus one clock cycle delay, the signals from the four neighboring pixels diagonally adjacent to the correction object pixel Gr/B are extracted and the extracted signals are supplied to the correction circuit30along with the signal from the correction object pixel Gr/B.

If the control signal instructs turning on of the correction, the correction circuit30performs the correction of the color mixture to the signal from the correction object pixel Gr/B by using the correction parameters Ka, Kb, Kc, and Kd on the basis of the signals from the four neighboring pixels in contact with the correction object pixel Gr/B with sides. Since the 1H-delay signal Sig_Gr_B_1his selected as an input signal to the correction circuit30, the signal Sig_Gr_B′ from the correction object pixel Gr/B after the correction is delayed by 1H with respect to the input signal to the color-mixture correction circuit11.

FIG. 13is a block diagram showing an example of the configuration of a Ggo channel correction sub-block113C. The Ggo channel correction sub-block113C includes five delay circuits41to45, in addition to the correction circuit30as in the R/Gb channel correction sub-block113A. The 1H-delay signal Sig_Ggo_1h, the 1H-delay signal Sig_R_Gb_1h, the 1H-delay signal Sig_Gr_B_1h, the no-delay signal Sig_R_Gb_0h, and the no-delay signal Sig_Gr_B_0h, among the total of eight signals output from the line memory group111inFIG. 8, the two signals being output for every channel, are input in the Ggo channel correction sub-block113C.

The delay circuit41delays the 1H-delay signal Sig_Ggo_1hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from a correction object pixel. In the Ggo channel correction sub-block113C, the Ggo pixel is the correction object pixel, as shown inFIG. 14.

The delay circuit42delays the 1H-delay signal Sig_R_Gb_1hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from an upper left pixel (a) in contact with the correction object pixel Ggo with a side. The delay circuit43delays the 1H-delay signal Sig_GR_B_1hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from an upper right pixel (b) in contact with the correction object pixel Ggo with a side.

The delay circuit44delays the no-delay signal Sig_R_Gb_0hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from a lower left pixel (c) in contact with the correction object pixel Ggo with a side. The delay circuit45delays the no-delay signal Sig_Gr_B_0hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from a lower right pixel (d) in contact with the correction object pixel Ggo with a side.

As described above, when the 1H-delay signal Sig_Ggo_1h, the 1H-delay signal Sig_R_Gb_1h, the 1H-delay signal Sig_Gr_B_1h, the no-delay signal Sig_R_Gb_0h, and the no-delay signal Sig_Gr_B_0hpass through the delay circuits41to45to cause the signal from the correction object pixel Ggo to be a signal having a 1H delay plus one clock cycle delay, the signals from the four neighboring pixels diagonally adjacent to the correction object pixel Ggo are extracted and the extracted signals are supplied to the correction circuit30along with the signal from the correction object pixel Ggo.

If the control signal instructs turning on of the correction, the correction circuit30performs the correction of the color mixture to the signal from the correction object pixel Ggo by using the correction parameters Ka, Kb, Kc, and Kd on the basis of the signals from the four neighboring pixels in contact with the correction object pixel Ggo with sides. Since the 1H-delay signal Sig_Ggo_1his selected as an input signal to the correction circuit30, the signal Sig_Ggo′ from the correction object pixel Ggo after the correction is delayed by 1H with respect to the input signal to the color-mixture correction circuit11.

FIG. 15is a block diagram showing an example of the configuration of a Gge channel correction sub-block113D. The Gge channel correction sub-block113D includes three delay circuits46to48, in addition to the correction circuit30as in the R/Gb channel correction sub-block113A. The 1H-delay signal Sig_Gge_1h, the 1H-delay signal Sig_Gr_B_1h, the 1H-delay signal Sig_R_Gb_1h, the no-delay signal Sig_Gr_B_0h, and the no-delay signal Sig_R_Gb_0h, among the total of eight signals output from the line memory group111inFIG. 8, the two signals being output for every channel, are input in the Gge channel correction sub-block113D.

The delay circuit46delays the 1H-delay signal Sig_Gge_1hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from a correction object pixel. In the Gge channel correction sub-block113D, the Gge pixel is the correction object pixel, as shown inFIG. 16.

The delay circuit47delays the 1H-delay signal Sig_Gr_B_1hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from an upper left pixel (a) in contact with the correction object pixel Gge with a side. The 1H-delay signal Sig_R_Gb_1his directly supplied to the correction circuit30as a signal from an upper right pixel (b) in contact with the correction object pixel Gge with a side.

The delay circuit48delays the no-delay signal Sig_Gr_B_0hby one clock cycle and supplies the delayed signal to the correction circuit30as a signal from a lower left pixel (c) in contact with the correction object pixel Gge with a side. The no-delay signal Sig_R_Gb_0his directly supplied to the correction circuit30as a signal from a lower right pixel (d) in contact with the correction object pixel Gge with a side.

As described above, when the 1H-delay signal Sig_Gge_1h, the 1H-delay signal Sig_Gr_B_1h, and the no-delay signal Sig_Gr_B_0hpass through the delay circuits46to48(the 1H-delay signal Sig_R_Gb_1hand no-delay signal Sig_R_Gb_0hare directly supplied to the correction circuit30) to cause the signal from the correction object pixel Gge to be a signal having a 1H delay plus one clock cycle delay, the signals from the four neighboring pixels diagonally adjacent to the correction object pixel Gge are extracted and the extracted signals are supplied to the correction circuit30along with the signal from the correction object pixel Gge.

If the control signal instructs turning on of the correction, the correction circuit30performs the correction of the color mixture to the signal from the correction object pixel Gge by using the correction parameters Ka, Kb, Kc, and Kd on the basis of the signals from the four neighboring pixels in contact with the correction object pixel Gge with sides. Since the 1H-delay signal Sig_Gge_1his selected as an input signal to the correction circuit30, the signal Sig_Gge′ from the correction object pixel Gge after the correction is delayed by 1H with respect to the input signal to the color-mixture correction circuit11.

Correction Circuit

The configuration of the correction circuit30common to the channels will now be described in first to third examples.

First Example

FIG. 17is a block diagram showing an example of the configuration of a correction circuit30A in a first example. The correction circuit30A calculates differences between the signal Sig_C (Sig_R_Gb/Sig_Gr_B/Sig_Ggo/Sig_Gge) from the correction object pixel and the signals (upper left pixel: Sig_UL, upper right pixel: Sig_UR, lower left pixel: Sig_LL, and lower right pixel: Sig_LR) from the pixels diagonally adjacent to the correction object pixel. The correction circuit30A, then, multiplies the differences by the independent correction parameters Ka, Kb, Kc, and Kd and adds the results of the multiplication to calculate a correction signal Sig_C′ (Sig_R_Gb′/Sig_Gr_B/Sig_Ggo′/Sig_Gge′).

Specifically, the correction circuit30A includes four subtractors301to304, four multipliers305to308, one adder309, and one selector310, as shown inFIG. 17.

The subtractor301calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_UL from the upper left pixel. The subtractor302calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_UR from the upper right pixel. The subtractor303calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_LL from the lower left pixel. The subtractor304calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_LR from the lower right pixel.

The multiplier305multiplies the output signal from the subtractor301by the correction parameter Ka. The multiplier306multiplies the output signal from the subtractor302by the correction parameter Kb. The multiplier307multiplies the output signal from the subtractor303by the correction parameter Kc. The multiplier308multiplies the output signal from the subtractor304by the correction parameter Kd. The adder309adds the output signals from the multipliers305to308to the signal Sig_C from the correction object pixel and outputs the added result as the correction signal Sig_C′.

This calculation process can be represented by the following equation:

The selector310selects and outputs the correction signal Sig_C′ output from the adder309if the control signal indicating whether the correction is turned on or off (1:ON, 0:OFF), supplied through the communication I/F42inFIG. 7, is set to one (ON), and selects and outputs the signal Sig_C from the correction object pixel if the control signal is set to zero (OFF).

Although the multipliers305to308multiply the differences calculated by the subtractors301to304by the independent correction parameters Ka, Kb, Kc, and Kd in the first example, this calculation process can be realized by bit shift. Which method is adopted can be determined on the basis of the balance between the correction accuracy and the circuit size.

FIG. 18illustrates the correction model equation shown in (1). Among the neighboring eight pixels around the correction object pixel, the upper, lower, left, and right pixels with respect to the correction object pixel are √2 times farther away from the correction object pixel than the upper left pixel, the upper right pixel, the lower left pixel, and the lower right pixel with respect to the correction object pixel. Accordingly, the upper left pixel, the upper right pixel, the lower left pixel, and the lower right pixel with respect to the correction object pixel have a more dominant influence of the color mixture on the correction object pixel, compared with the upper, lower, left, and right pixels with respect to the correction object pixel. Consequently, it is assumed in this example that the color mixture between the correction object pixel and the upper, lower, left, and right pixels can be negligible, and the upper, lower, left, and right pixels are excluded from the description.

The correction circuit30is structured so as to adopt the correction model, in which the color of the correction object pixel is added back by an amount corresponding to the color mixture ratio K when the color of the correction object pixel leaks into the upper left, upper right, lower left, and lower right pixels by the above amount and the color of the correction object pixel is subtracted by the amount corresponding to the color mixture ratio K when the color of the upper left, upper right, lower left, or lower right pixel leaks into the correction object pixel by the above amount, to alleviate the color mixture. In this model, the color mixture ratios between the correction object pixel and the upper left, upper right, lower left, and the lower right pixels are respectively denoted by Ka, Kb, Kc, and Kd. In other words, since the amount of the color mixture increases as the difference in level between the correction object pixel and the neighboring pixels is increased, the correction circuit30performs the correction of the color mixture in accordance with the amount of difference.

The above structure achieves the following advantages:

The amount of the correction of the color mixture can be externally controlled in real time through the communication I/F42(refer toFIGS. 6 and 7).

Varying the values of the correction parameters Ka, Kb, Kc, and Kd can realize the correction of the color mixture with directionality (can also realize isotropic correction if Ka=Kb=Kc=Kd).

Although the correction model equation (1) is used in this example, the correction circuit30A is not limited to the circuit configuration realizing the calculation in (1) because the present invention is not focused on the model equation itself.

Second Example

FIG. 19is a block diagram showing an example of the configuration of a correction circuit30B in a second example. The correction circuit30B calculates differences between the signal Sig_C (Sig_K_Gb/Sig_Gr_B/Sig_Ggo/Sig_Gge) from the correction object pixel and the signals (upper left pixel: Sig_UL, upper right pixel: Sig_UR, lower left pixel: Sig_LL, and lower right pixel: Sig_LR) from the pixels diagonally adjacent to the correction object pixel. The correction circuit30B, then, adds any pair of the differences in accordance with a directional selection control signal (value), supplied through the communication I/F42. The correction circuit30B multiplies the addition results by independent correction parameters K1and K2and adds the results of the multiplication to calculate a correction signal Sig_C′ (Sig_R_Gb′/Sig_Gr_B′/Sig_Ggo′/Sig_Gge′).

Specifically, the correction circuit30B includes four subtractors311to314, three selectors315to317, three adders318to320, two multipliers321and322, one comparator323, one adder324, and one selector325, as shown inFIG. 19.

The subtractor311calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_UL from the upper left pixel. The subtractor312calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_UR from the upper right pixel. The subtractor313calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_LL from the lower left pixel. The subtractor314calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_LR from the lower right pixel.

The selector315receives output signals B, C, and D from the subtractors312,313, and314, respectively. The selector315selects and outputs the output signal B from the subtractor312if the directional selection control signal has a value “0”, selects and outputs the output signal C from the subtractor313if the directional selection control signal has a value “1”, and selects and outputs the output signal D from the subtractor314if the directional selection control signal has a value “2”. The selector316receives output signals C and D from the subtractors313and314, respectively. The selector316selects and outputs the output signal D from the subtractor314if the directional selection control signal has a value “1”, and selects and outputs the output signal C from the subtractor313if the directional selection control signal has a value “2”.

The adder318adds the output signal from the selector315to an output signal A from the subtractor311. The adder319adds the output signal from the selector316to the output signal B from the subtractor312. The adder320adds the output signal C from the subtractor313to the output signal D from the subtractor314.

The comparator323outputs a control signal having a value “1” if the directional selection control signal has a value “0” and outputs a control signal having a value “0” if the directional selection control signal has other values. The selector317receives the output signals from the adders319and320. The selector317selects and outputs the output signal from the adder319if the control signal supplied from the comparator323has a value “0” and selects and outputs the output signal from the adder320if the control signal supplied from the comparator323has a value “1”.

The multiplier321multiplies the output signal from the adder318by the correction parameter K1. The multiplier322multiplies the output signal from the selector317by the correction parameter K2. The adder324adds the output signals from the multipliers321and322to the signal Sig_C from the correction object pixel to output a correction signal Sig_C′.

This calculation process can be represented by the following equations:

If the directional selection control signal has a value “0”,

If the directional selection control signal has a value “1”,

If the directional selection control signal has a value “2”,

These correction model equations can be switched. The selector325selects and outputs the correction signal Sig_C′ output from the adder324if the control signal indicating whether the correction is turned on or off (1:ON, 0:OFF), supplied through the communication I/F42, is set to one (ON), and selects and outputs the signal Sig_C from the correction object pixel if the control signal is set to zero (OFF).

Although the multipliers321and322multiply the output signals from the adder318and the selector317by the independent correction parameters K1and K2in the second example, this calculation process can be realized by bit shift. Which method is adopted can be determined on the basis of the balance between the correction accuracy and the circuit size.

FIG. 20illustrates the correction model equations shown in (2), (3), and (4). The concept of the correction model equations is the same as in the correction model equation inFIG. 18.FIG. 20shows combination of the color mixture ratios and the correction model equations depending on the value (0, 1, or 2) of the directional selection control signal.

In the correction circuit30A (refer toFIG. 17) in the first example, since the calculations should be simultaneously performed in parallel every clock cycle, the four multipliers305to308are basically provided for every channel. In contrast, in the correction circuit30B in the second example, since the degree of freedom of the directionality of the correction is reduced and the function similar to that of the correction circuit30A is realized only by the two multipliers321and322, the circuit size can be greatly reduced. Although the degree of freedom of the directionality is restricted in order to reduce the circuit size, the combination of the color mixture ratios, most suitable for the characteristics of the CMOS image sensor, is used to correct the color mixture in order to improve the degree of freedom as much as possible.

The above structure achieves the following advantages: The amount of the correction of the color mixture can be externally controlled in real time through the communication I/F42(refer toFIGS. 6 and 7).

The correction of the color mixture with the circuit size being greatly reduced can be realized while keeping a certain degree of freedom of the directionality of the correction (can also realize isotropic correction if K1=K2).

Third Example

FIG. 21is a block diagram showing an example of the configuration of a correction circuit30C in a third example. The correction circuit30C calculates differences between the signal Sig_C (Sig_R_Gb/Sig_Gr_B/Sig_Ggo/Sig_Gge) from the correction object pixel and the signals (upper left pixel: Sig_UL, upper right pixel: Sig_UR, lower left pixel: Sig_LL, and lower right pixel: Sig_LR) from the pixels diagonally adjacent to the correction object pixel and adds all the differences. The correction circuit30C, then, multiplies the addition result by a correction parameter K and adds the multiplication result to the original signal Sig_C to calculate a correction signal Sig_C′ (Sig_R_Gb′/Sig_Gr_B′/Sig_Ggo′/Sig_Gge′).

Specifically, the correction circuit30C includes four subtractors331to334, two adders335and336, one multiplier337, and one selector338, as shown inFIG. 21.

The subtractor331calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_UL from the upper left pixel. The subtractor332calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_UR from the upper right pixel. The subtractor333calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_LL from the lower left pixel. The subtractor334calculates a difference between the signal Sig_C from the correction object pixel and the signal Sig_LR from the lower right pixel.

The adder335adds all the output signals from the subtractors331to334. The multiplier337multiplies the output signal from the adder335by the correction parameter K. The adder336adds the output signal from the multiplier337to the signal Sig_C from the correction object pixel to output the correction signal Sig_C′.

This calculation process can be represented by the following equation:

The correction circuit30C can further reduce the circuit size, compared with the correction circuit30B in the second example, because the only one multiplier is used to perform the correction of the color mixture, although the degree of freedom of the directionality is lost. The correction circuit30C has a very effective circuit configuration, for example, when the directionality in the color mixture in the image sensor can be negligible in order to achieve a desired image quality.

The selector338selects and outputs the correction signal Sig_C′ output from the adder336if the control signal indicating whether the correction is turned on or off (1:ON, 0:OFF), supplied through the communication I/F42, is set to one (ON), and selects and outputs the signal Sig_C from the correction object pixel if the control signal is set to zero (OFF).

Although the multiplier337multiplies the output signal from the adder335by the correction parameter K in the third example, this calculation process can be realized by bit shift. Which method is adopted can be determined on the basis of the balance between the correction accuracy and the circuit size.

FIG. 22illustrates the correction model equation shown in (5). The concept of the correction model equation is the same as in the correction model equation inFIG. 18.

The above structure achieves the following advantages: The amount of the correction of the color mixture can be externally controlled in real time through the communication I/F42(refer toFIGS. 6 and 7).

The correction of the color mixture can be realized with the circuit size being greatly reduced, although the degree of freedom of the directionality of the correction is lost.

As described above, the color-mixture correction circuit has a structure in which a condenser microlens is layered on a color separation filter that is layered on pixel cells including photoelectric transducers, and has the color coding in which the G components surround the R and B components in the pixel shifted arrangement. The color-mixture correction circuit corrects the color mixture between the pixels in the CMOS image sensor20A, which performs vertical scanning for every horizontal zigzag row. The color-mixture correction circuit uses the signals from the multiple neighboring pixels adjacent to a target pixel (correction object pixel) and the correction parameters independently set for the signals to perform the correction of the color mixture to the signal from the target pixel. Since the amount of correction of the color mixture from the neighboring pixels into the target pixel can be independently set for every neighboring pixel, it is possible to give the directionality to the amount of correction of the color mixture from the neighboring pixels into the target pixel, that is, it is possible to set different amounts of correction for different neighboring pixels.

Accordingly, the correction of the color mixture can be performed in accordance with the degree of the color mixture from the neighboring pixels into the target pixel. Even if the photoelectric transducers (photosensitive sections) are shifted from the centers of the pixel cells depending on the layout of the circuit sections, the wiring, or the signal reading sections and, therefore, the physical center of each pixel cell does not necessarily coincide with the optical center thereof to cause the directionality to the color mixture from the neighboring pixels into the target pixel, the correction of the color mixture can be realized in accordance with the directionality to reduce the degree of the color mixture.

Particularly, in the color coding in which the G components surround the R and B components in the pixel shifted arrangement, all the neighboring pixels are the G components when the correction object pixel (target pixel) is the R or B component. Since the value of the correction parameter (the amount of correction) can be set for every G neighboring pixel, the correction of the color mixture can be realized with more effective directionality. In addition, isotropic correction can be realized depending on the values of the correction parameters. Since the external camera controller5can set the values of the correction parameters through the communication I/F42, it is also possible to set the amount of correction in accordance with the shooting conditions in real time.

When the degree of freedom in the four directions is not necessary in order to keep the balance between the desired image quality and the circuit size, the use of the correction circuit30B in the second example allows the correction circuit keeping a higher degree of freedom of the directionality to be configured while greatly reducing the circuit size. When the degree of freedom in the four directions is not necessary and it is sufficient to realize the isotropic correction in order to keep the balance between the desired image quality and the circuit size, the use of the correction circuit30C in the third example allows the correction circuit having a further reduced circuit size to be configured.

When the physical center of each pixel cell does not necessarily coincide with the optical center thereof to cause the directionality to the color mixture from the neighboring pixels into the target pixel, the values of the correction parameters (the amounts of correction) independently set for the signals from the multiple neighboring pixels are appropriately set on the basis of the degree of the color mixture from the neighboring pixels into the target pixel.

In the above embodiment, the upper, lower, left, and right pixels, which are in contact with the target pixel with apices, are excluded because the color mixture between the target pixel and the upper, lower, left, and right pixels can be negligible in the pixel shifted arrangement shown inFIG. 2, and the correction of the color mixture is performed by using the signals from the upper left, upper right, lower left, and lower right pixels, which are in contact with the target pixel with sides. However, the color mixture may be corrected by using the signals from the upper, lower, left and right pixels. Also in this case, independent correction parameters are used for the signals from the upper, lower, left and right pixels.

Although the amount of correction is set independently of the color of the pixel in the above embodiment, the amount of correction may be varied for every color. Specifically, the camera controller5generates R correction parameters Kar, Kbr, Kcr, and kdr, G correction parameters Kag, Kbg, Kcg, and Kdg, and B correction parameters Kab, Kbb, Kcb, and Kdb and supplies the generated correction parameters to the R/Gb channel correction sub-block113A and the Gr/B channel correction sub-block113B through the communication I/F42.

As shown inFIG. 23, a switch SW1selectively receiving the R correction parameters Kar, Kbr, Kcr, and kdr and the G correction parameters Kag, Kbg, Kcg, and Kdg is provided in the R/Gb channel correction sub-block113A. The switch SW1is switched in response to a timing signal H_TOGLE, which is one of the timing signals from the signal generator43and whose level is switched between “High”(1) and “Low”(0) every 1H. The R correction parameters Kar, Kbr, Kcr, and kdr and the G correction parameters Kag, Kbg, Kcg, and Kdg can be alternately supplied to the correction circuit30for every 1H to realize the correction of the color mixture in which the amount of correction is varied for every R pixel and for every G pixel.

In contrast, as shown inFIG. 24, a switch SW2selectively receiving the G correction parameters Kag, Kbg, Kcg, and kdg and the B correction parameters Kab, Kbb, Kcb, and Kdb is provided in the Gr/B channel correction sub-block113B. The switch SW2is switched in response to the timing signal H_TOGLE. The G correction parameters Kag, Kbg, Kcg, and kdg and the B correction parameters Kab, Kbb, Kcb, and Kdb can be alternately supplied to the correction circuit30for every 1H to realize the correction of the color mixture in which the amount of correction is varied for every G pixel and for every B pixel.

Second Embodiment

FIG. 25is a block diagram schematically showing an example of the structure of a CMOS image sensor20B according to a second embodiment of the present invention. The same reference numerals are used inFIG. 25to identify the same components shown inFIG. 3.

The CMOS image sensor20B according to the second embodiment has the same pixel arrangement and color coding as those of the CMOS image sensor20A according to the first embodiment. Specifically, the CMOS image sensor20B has the color coding, shown inFIG. 2, in which the G components surround the R and B components in the pixel shifted arrangement.

In the CMOS image sensor20B, pixel drive lines23are wired for every row. The pixel cells21in a pixel array unit22are selected in units of rows by selection and scanning by a vertical scanning circuit24through the pixel drive line23. Each vertical signal line25is commonly wired to the pixel cells21in a vertical zigzag column in units of two columns.

One column processing circuit26is connected to one end of each vertical signal line25. In other words, the column processing circuits26are arranged for every two columns and hold the signals supplied from the pixel cells21through the vertical signal lines25. The signals corresponding to one row, held in each column processing circuit26, are sequentially output to two horizontal signal lines29-1and29-2in units of two pixels through horizontal selector switches28sequentially selected by a horizontal scanning circuit27in units of two switches.

As described above, in the CMOS image sensor20B according to the second embodiment of the present invention, the signals in units of two neighboring pixels for every row are read out in parallel through the two horizontal signal lines29-1and29-2by using two channels and are horizontally scanned over one screen. After all the signals during one horizontal scanning period (1H) have been read out, the signals in the subsequent row are horizontally scanned to read out the signals from the pixels over one screen.

For convenience, sixteen R, G, and B pixels in the first to eighth columns in the first to fourth rows in the color coding shown inFIG. 2are defined as shown inFIG. 26, as in the first embodiment of the present invention.

Under the above definition, the signals in units of the two neighboring pixels for every row are read out in parallel through the two channels in one clock cycle of a clock signal, on which the operation of the CMOS image sensor20B is based, to output an R/Ggo/Gb/Ggo signal and a Gr/Gge/B/Gge signal through the two channels, as shown inFIG. 26.FIG. 27shows a sequence of the output signals through the channels in the above readout method.

Camera Signal Processing Circuit

FIG. 28is a block diagram showing an example of the internal configuration of a camera signal processing circuit41according to the second embodiment of the present invention. The same reference numerals are used inFIG. 28to identify the same components shown inFIG. 7.

Referring toFIG. 28, the camera signal processing circuit41according to the second embodiment of the present invention includes a rearrangement processing circuit413, in addition to a first camera signal processor group411, a color-mixture correction circuit11, and a second camera signal processor group412. The rearrangement processing circuit413is provided upstream of the first camera signal processor group411. The rearrangement processing circuit413is provided to rearrange the sequence of the output signals from the CMOS image sensor20B, shown inFIG. 27, into the sequence of the output signals from the CMOS image sensor20A, shown inFIG. 5.

Rearrangement Processing Circuit

FIG. 29is a block diagram showing an example of the configuration of the rearrangement processing circuit413. Referring toFIG. 29, the rearrangement processing circuit413includes a line memory group4131, a memory controller4132, switches4133and4134.

The line memory group4131is provided for the R/Ggo/Gb/Ggo and Gr/Gge/B/Gge signals from the pixels through the two channels. The line memory group4131includes line memories4131-1and4131-2for causing delays in units of rows. The line memories4131-1and4131-2form, for example, a single port SRAM. The memory controller4132controls writing in and readout from the line memories4131-1and4131-2on the basis of various timing signals supplied from a signal generator43shown inFIG. 28.

The switch4133receives the R/Ggo/Gb/Ggo signal from the signal through one channel and performs the switching in response to a timing signal H_TOGLE, which is one of the timing signals from the signal generator43and whose level is switched between “High”(1) and “Low” (0) every 1H. The switch4134receives the Gr/Gge/B/Gge signal from the pixel through the other channel and performs the switching in response to the control signal H_TOGLE.

The signals input in the odd-numbered rows (the 4N+1-th row, 4N+3-th row, . . . inFIG. 27) are delayed by 1H by the line memories4131-1and4131-2and are output as R/Gb and Gr/B signals. In contrast, the signals input in the even-numbered rows (the 4N+2-th row, 4N+4-th row, . . . inFIG. 27) do not pass through the line memories4131-2and4131-2and are output as Ggo and Gge signals.

Owing to the effect of the rearrangement processing circuit413having the above configuration, the R/Ggo/Gb/Ggo and Gr/Gge/B/Gge signals from the pixels through the two channels are rearranged into the R/Gb, Ggo, Gr/B, and Gge signals through the four channels, shown inFIG. 30, and the rearranged R/Gb, Ggo, Gr/B, and Gge signals are output. Performing the rearrangement of the signals upstream of the first camera signal processor group411in the above manner allows the circuit blocks according to the first embodiment to be used as the first camera signal processor group411, the color-mixture correction circuit11, and the second camera signal processor group412.

However, although the sequence after the rearrangement in the rearrangement processing circuit413is similar to that in the first embodiment, the signals after the rearrangement are transmitted every 1H. Accordingly, for example, a timing signal similar to the timing signal H_TOGLE may be applied to the circuit blocks including the first camera signal processor group411, the color-mixture correction circuit11, and the second camera signal processor group412, and the circuit blocks may perform the processing only if the level of the timing signal is “High”.

As described above, the CMOS image sensor20B according to the second embodiment of the present invention has the color coding in which the G components surround the R and B components in the pixel shifted arrangement and performs the vertical scanning for every row, instead of for every horizontal zigzag row in units of two rows. In the correction of the color mixture according to the second embodiment of the present invention, the amount of correction of the color mixture from the neighboring pixels into the target pixel can be independently set for every neighboring pixel, so that advantages similar to those in the first embodiment can be achieved.

Also in the correction of the color mixture according to the second embodiment, the amount of correction may be varied for every color, as in the first embodiment.

Although the correction of the color mixture in the solid-state imaging device having the color coding in which the G components surround the R and B components in the pixel shifted arrangement is exemplified in the first and second embodiments, this color coding is only an example. The present invention is applicable to the correction of the color mixture in a solid-state imaging device having another color coding, for example, the one shown inFIG. 31.

Third Embodiment

FIG. 32shows an example of the color coding in a CMOS image sensor according to a third embodiment of the present invention.

In the CMOS image sensor according to the third embodiment of the present invention, a pixel array unit in which pixel cells including the photoelectric transducers are two-dimensionally arranged in an array has pixel arrangement in a square lattice. The color coding in the pixel arrangement in a square lattice has, for example, a Bayer array shown inFIG. 32.

FIG. 33is a block diagram schematically showing an example of the configuration of a CMOS image sensor20C according to the third embodiment of the present invention. The same reference numerals are used inFIG. 33to identify the same components shown inFIG. 3.

Referring toFIG. 33, a pixel array unit22has pixel cells21including photoelectric transducers, two-dimensionally arranged in a square lattice. In the pixel arrangement in a square lattice, one pixel drive line23is wired for every two rows and two vertical signal lines25are wired for every column. A vertical scanning circuit24sequentially selects and scans the pixel cells21in the pixel array unit22in units of two rows through the pixel drive lines23.

The signals from the pixel cells21corresponding to two rows, selected by the scanning by the vertical scanning circuit24, are read out through the vertical signal lines25for odd-numbered rows and the vertical signal lines25for even-numbered rows, and are held in the corresponding column processing circuits26. The signals corresponding to two rows, held in each column processing circuit26, are sequentially output to four horizontal signal lines29-1to29-4in units of four pixels (two rows×two columns) through horizontal selector switches28sequentially selected by a horizontal scanning circuit27in units of four switches.

As described above, in the CMOS image sensor20C according to the third embodiment of the present invention, the signals in units of four neighboring pixels for every two rows are read out in parallel through the four horizontal signal lines29-1to29-4by using four channels and are horizontally scanned over one screen. After all the signals during one horizontal scanning period (1H) have been read out, the signals in the subsequent row are horizontally scanned to read out the signals from the pixels over one screen.

For convenience, twelve R, G, and B pixels in the first to sixth columns in the first and second rows in the color coding shown inFIG. 32are defined as shown inFIG. 34. Specifically, in the first row, the pixel R among the four pixels in the first unit is defined as a pixel R1; the pixel R among the four pixels in the second unit is defined as a pixel R2; and the pixel R among the four pixels in the third unit is defined as a pixel R3. In the second row, the pixel B among the four pixels in the first unit is defined as a pixel B1; the pixel B among the four pixels in the second unit is defined as a pixel B2; and the pixel B among the four pixels in the third unit is defined as a pixel B3.

In the first row, the pixel G among the four pixels in the first unit, adjacent to the pixel R1, is defined as a pixel Gr1; the pixel G among the four pixels in the second unit, adjacent to the pixel R2, is defined as a pixel Gr2; and the pixel G among the four pixels in the third unit, adjacent to the pixel R3, is defined as a pixel Gr3. In the second row, the pixel G among the four pixels in the first unit, adjacent to the pixel B1, is defined as a pixel Gb1; the pixel G among the four pixels in the second unit, adjacent to the pixel B2, is defined as a pixel Gb2; and the pixel G among the four pixels in the third unit, adjacent to the pixel B3, is defined as a pixel Gb3.

Under the above definition, the signals in units of the four neighboring pixels for every two rows are read out in parallel through the four channels in one clock cycle of a clock signal, on which the operation of the CMOS image sensor20C is based, to output R, Gr, Gb, and B signals through the four channels, as shown inFIG. 34.FIG. 35shows a sequence of the output signals through the channels in the above readout method.

The internal configuration of the camera signal processing circuit41according to the third embodiment is basically the same as that of the camera signal processing circuit41according to the first embodiment, shown inFIG. 7. Specifically, the camera signal processing circuit41includes a first camera signal processor group411and a second camera signal processor group412, in addition to the color-mixture correction circuit11. The first camera signal processor group411is provided upstream of the color-mixture correction circuit11and the second camera signal processor group412is provided downstream of the color-mixture correction circuit11. The camera signal processing circuit41processes, in parallel, the R, Gr, Gb, and B signals for every four pixels, shown inFIG. 34, in one clock of the clock signal on which the camera signal processing is based.

Color-Mixture Correction Circuit

FIG. 36is a block diagram showing an example of the configuration of a color-mixture correction circuit11according to the third embodiment of the present invention. The same reference numerals are used inFIG. 36to identify the same components shown inFIG. 8.

As shown inFIG. 36, in the color-mixture correction circuit11according to the third embodiment, a line memory group111includes an R line memory111-5, a Gr line memory111-6, a Gb line memory111-7, and a B line memory111-8, instead of the R/Gb line memory111-1, the Gr/B line memory111-2, the Ggo line memory111-3, the Gge line memory111-4in the color-mixture correction circuit11according to the first embodiment.

The line memories111-5and111-6are 1H line memories and the line memories111-7and111-8are 2H line memories. A memory controller112controls writing in and readout from the line memories111-5,111-6,111-7, and111-8on the basis of the various timing signals supplied from the signal generator43shown inFIG. 7. A correction block113corrects the color mixture of the pixels in response to the control signal supplied through the communication I/F42shown inFIG. 7.

In the color-mixture correction circuit11, the R, Gr, Gb, and B signals from the pixels through the four channels are input in the line memories111-5,111-6,111-7, and111-8in parallel, respectively. A group of signals having 0H to 2H delays for every channel is generated and output in response to a writing enabling signal WEN, a writing address signal WADRS, a readout enabling signal REN, and a readout address signal RADRS, supplied from the memory controller112.

The line memory group111is structured such that a no-delay signal (Sig_R_0h) and a 1H-delay signal (Sig_R_1h) are output through the R channel, a no-delay signal (Sig_Gr_0h) and a 1H-delay signal (Sig_Gr_1h) are output through the Gr channel, a 1H-delay signal (Sig_Gb_1h) and a 2H-delay signal (Sig_Gb_2h) are output through the Gb channel, and a 1H-delay signal (Sig_B_1h) and a 2H-delay signal (Sig_B_2h) are output through the B channel.

The group of the signals delayed by the line memories111-5,111-6,111-7, and111-8is supplied to the correction block113. The correction block113corrects the color mixture of the pixels in parallel for every channel in accordance with the control signal supplied through the communication I/F42and supplies Sig_R′, Sig_Gr′, Sig_Gb′, and Sig_B′ signals after the correction to the downstream blocks. The correction block113includes four correction sub-blocks provided for the R, Gr, Gb, and B signals from the pixels through the four channels.

R Channel Correction Sub-Block

FIG. 37is a block diagram showing an example of the configuration of an R channel correction sub-block113E. The R channel correction sub-block113E includes a correction circuit50and five delay circuits51to55. The 1H-delay signal Sig_R_1h, the 2H-delay signal Sig_Gb_2h, the 1H-delay signal Sig_Gr_1h, the 1H-delay signal Sig_Gr_1h, and the 1H-delay signal Sig_Gb_1h, among the total of eight signals output from the line memory group111inFIG. 36, the two signals being output for every channel, are input in the R channel correction sub-block113E.

The correction circuit50is shared between the channels. The circuit configuration of the correction circuit50will be described in detail below. The delay circuit51delays the 1H-delay signal Sig_R_1hby one clock cycle of the clock signal having a pixel period on which the correction of the color mixture is based and supplies the delayed signal to the correction circuit50as a signal from a correction object pixel. In the R channel correction sub-block113E, the R pixel is the correction object pixel, as shown inFIG. 38A.

The delay circuit52delays the 2H-delay signal Sig_Gb_2hby two clock cycles and supplies the delayed signal to the correction circuit50as a signal from an upper pixel (a) in contact with the correction object pixel R with a side. The delay circuit53delays the 1H-delay signal Sig_Gr_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a right pixel (b) in contact with the correction object pixel R with a side.

The delay circuit54delays the 1H-delay signal Sig_Gr_1hby two clock cycles and supplies the delayed signal to the correction circuit50as a signal from a left pixel (c) in contact with the correction object pixel R with a side. The delay circuit55delays the 1H-delay signal Sig_Gb_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a lower pixel (d) in contact with the correction object pixel R with a side.

As described above, when the 1H-delay signal Sig_R_1h, the 2H-delay signal Sig_Gb_2h, the 1H-delay signal Sig_Gr_1h, the 1H-delay signal Sig_Gr_1h, and the 1H-delay signal Sig_Gb_1hpass through the delay circuits51to55to cause the signal from the correction object pixel R to be a signal having a 1H delay plus one clock cycle delay, the signals from the four neighboring pixels in contact with the correction object pixel R with sides: that is, the upper, right, left, and lower pixels with respect to the correction object pixel R are extracted and the extracted signals are supplied to the correction circuit50along with the signal from the correction object pixel R.

Gr Channel Correction Sub-Block

FIG. 39is a block diagram showing an example of the configuration of a Gr channel correction sub-block113F. The Gr channel correction sub-block113F includes four delay circuits56to59, in addition to the correction circuit50as in the correction block113E. The 1H-delay signal Sig_Gr_1h, the 2H-delay signal Sig_B_2h, the 1H-delay signal Sig_R_1h, the 1H-delay signal Sig_R_1h, and the 1H-delay signal Sig_B_1h, among the total of eight signals output from the line memory group111inFIG. 36, the two signals being output for every channel, are input in the Gr channel correction sub-block113F.

The delay circuit56delays the 1H-delay signal Sig_Gr_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a correction object pixel. In the Gr channel correction sub-block113F, the Gr pixel is the correction object pixel, as shown inFIG. 38B. The delay circuit57delays the 2H-delay signal Sig_B_2hby two clock cycles and supplies the delayed signal to the correction circuit50as a signal from an upper pixel (a) in contact with the correction object pixel Gr with a side.

The 1H delay signal Sig_R_1his directly supplied to the correction circuit50as a signal from a right pixel (b) in contact with the correction object pixel Gr with a side. The delay circuit58delays the 1H-delay signal Sig_R_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a left pixel (c) in contact with the correction object pixel Gr with a side. The delay circuit59delays the 1H-delay signal Sig_B_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a lower pixel (d) in contact with the correction object pixel Gr with a side.

As described above, when the 1H-delay signal Sig_Gr_1h, the 2H-delay signal Sig_B_2h, the 1H-delay signal Sig_R_1h, the 1H-delay signal Sig_R_1h, and the 1H-delay signal Sig_B_1hpass through the delay circuits56to59to cause the signal from the correction object pixel Gr to be a signal having a 1H delay plus one clock cycle delay, the signals from the four neighboring pixels in contact with the correction object pixel Gr with sides: that is, the upper, right, left, and lower pixels with respect to the correction object pixel Gr are extracted and the extracted signals are supplied to the correction circuit50along with the signal from the correction object pixel Gr.

Gb Channel Correction Sub-Block

FIG. 40is a block diagram showing an example of the configuration of a Gb channel correction sub-block113G. The Gb channel correction sub-block113G includes five delay circuits61to65, in addition to the correction circuit50as in the correction block113E. The 1H-delay signal Sig_Gb_1h, the 1H-delay signal Sig_R_1h, the 1H-delay signal Sig_B_1h, the 1H-delay signal Sig_B_1h, and the non-delay signal Sig_R_0h, among the total of eight signals output from the line memory group111inFIG. 36, the two signals being output for every channel, are input in the Gb channel correction sub-block113G.

The delay circuit61delays the 1H-delay signal Sig_Gb_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a correction object pixel. In the Gb channel correction sub-block113G, the Gb pixel is the correction object pixel, as shown inFIG. 38C.

The delay circuit62delays the 1H-delay signal Sig_R_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from an upper pixel (a) in contact with the correction object pixel Gb with a side. The delay circuit63delays the 1H-delay signal Sig_B_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a right pixel (b) in contact with the correction object pixel Gb with a side.

The delay circuit64delays the 1H-delay signal Sig_B_1hby two clock cycles and supplies the delayed signal to the correction circuit50as a signal from a left pixel (c) in contact with the correction object pixel Gb with a side. The delay circuit65delays the 1H-delay signal Sig_R_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a lower pixel (d) in contact with the correction object pixel Gb with a side.

As described above, when the 1H-delay signal Sig_Gb_1h, the 1H-delay signal Sig_R_1h, the 1H-delay signal Sig_B_1h, the 1H-delay signal Sig_B_1h, and the non-delay signal Sig_R_0hpass through the delay circuits61to65to cause the signal from the correction object pixel Gb to be a signal having a 1H delay plus one clock cycle delay, the signals from the four neighboring pixels in contact with the correction object pixel Gb with sides: that is, the upper, right, left, and lower pixels with respect to the correction object pixel Gb are extracted and the extracted signals are supplied to the correction circuit50along with the signal from the correction object pixel Gb.

B Channel Correction Sub-Block

FIG. 41is a block diagram showing an example of the configuration of a B channel correction sub-block113H. The B channel correction sub-block113H includes four delay circuits66to69, in addition to the correction circuit50as in the correction block113E. The 1H-delay signal Sig_B_1h, the 1H-delay signal Sig_Gr_1h, the 1H-delay signal Sig_Gb_1h, the 1H-delay signal Sig_Gb_1h, and the 0H-delay signal Sig_Gb_0h, among the total of eight signals output from the line memory group111inFIG. 36, the two signals being output for every channel, are input in the B channel correction sub-block113H.

The delay circuit66delays the 1H-delay signal Sig_B_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a correction object pixel. In the B channel correction sub-block113H, the B pixel is the correction object pixel, as shown inFIG. 38D. The delay circuit67delays the 1H-delay signal Sig_Gr_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from an upper pixel (a) in contact with the correction object pixel B with a side.

The 1H delay signal Sig_Gb_1his directly supplied to the correction circuit50as a signal from a right pixel (b) in contact with the correction object pixel B with a side. The delay circuit68delays the 1H-delay signal Sig_Gb_1hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a left pixel (c) in contact with the correction object pixel B with a side. The delay circuit69delays the non-delay signal Sig_Gr_0hby one clock cycle and supplies the delayed signal to the correction circuit50as a signal from a lower pixel (d) in contact with the correction object pixel B with a side.

As described above, when the 1H-delay signal Sig_B_1h, the 1H-delay signal Sig_Gr_1h, the 1H-delay signal Sig_Gb_1h, the 1H-delay signal Sig_Gb_1h, and the 0H-delay signal Sig_Gb_0hpass through the delay circuits66to69to cause the signal from the correction object pixel B to be a signal having a 1H delay plus one clock cycle delay, the signals from the four neighboring pixels in contact with the correction object pixel B with sides: that is, the upper, right, left, and lower pixels with respect to the correction object pixel B are extracted and the extracted signals are supplied to the correction circuit50along with the signal from the correction object pixel B.

Correction Circuit

The configuration of the correction circuit50common to the channels will now be described in first to third examples.

First Example

FIG. 42is a block diagram showing an example of the configuration of a correction circuit50A in a first example. The correction circuit50A has the same circuit configuration as the correction circuit30A shown inFIG. 17except for the input signals.

Specifically, the correction circuit50A calculates differences between the signal Sig_C (Sig_R /Sig_Gr/Sig_Gb/Sig_B) from the correction object pixel and the signals (upper: Sigup, right pixel: Sig_R, left pixel: Sig_L, and lower pixel: Sig_Lo) from the pixels adjacent to the correction object pixel horizontally and vertically. The correction circuit50A, then, multiplies the differences by independent correction parameters Ka, Kb, Kc, and Kd and adds the results of the multiplication to calculate a correction signal Sig_C′ (Sig_R′/Sig_Gr′/Sig_Gb′/Sig_B′).

“Ka” denotes the color mixture (correction) ratio between the correction object pixel and the upper pixel; “Kb” denotes the color mixture (correction) ratio between the correction object pixel and the right pixel; “Kc” denotes the color mixture (correction) ratio between the correction object pixel and the left pixel; “Kd′” denotes the color mixture (correction) ratio between the correction object pixel and the lower pixel.

This calculation process in the correction circuit50A can be represented by the following equation:

FIG. 43illustrates the correction model equation shown in (6). Among the neighboring eight pixels around the correction object pixel, the upper left pixel, the upper right pixel, the lower left pixel, and the lower right pixel with respect to the correction object pixel are √2 times farther away from the correction object pixel than upper, lower, left, and right pixels the with respect to the correction object pixel. Accordingly, the upper, right pixel, left, and lower pixels with respect to the correction object pixel have a more dominant influence of the color mixture on the correction object pixel, compared with the upper left pixel, the upper right pixel, the lower left pixel, and the lower right pixel with respect to the correction object pixel. Consequently, it is assumed in this example that the color mixture between the correction object pixel and the upper left pixel, the upper right pixel, the lower left pixel, and the lower right pixel can be negligible, and the upper left pixel, the upper right pixel, the lower left pixel, and the lower right pixel are excluded from the description.

Second Example

FIG. 44is a block diagram showing an example of the configuration of a correction circuit50B in a second example. The correction circuit50B has the same circuit configuration as the correction circuit30B shown inFIG. 19except for the input signals.

Specifically, the correction circuit50B calculates differences between the signal Sig_C (Sig_R/Sig_Gr/Sig_Gb/Sig_B) from the correction object pixel and the signals (upper pixel: Sig_Up, right pixel: Sig_R, left pixel: Sig_L, and lower pixel: Sig_Lo) from the pixels adjacent to the correction object pixel horizontally and vertically. The correction circuit50B, then, adds any pair of the differences in accordance with a directional selection control signal (value) supplied through the communication I/F42. The correction circuit30B multiplies the addition results by independent correction parameters K1and K2and adds the results of the multiplication to calculate a correction signal Sig_C′ (Sig_R′/Sig_Gr′/Sig_Gb′/Sig_B′).

This calculation process in the correction circuit50B can be represented by the following equations:

If the directional selection control signal has a value “0”,

If the directional selection control signal has a value “1”,

If the directional selection control signal has a value “2”,

FIG. 45illustrates the correction model equations shown in (7), (8), and (9). The concept of the correction model equations is the same as in the correction model equation inFIG. 43.FIG. 45shows combination of the color mixture ratios and the correction model equations depending on the value (0, 1, or 2) of the directional selection control signal.

Third Example

FIG. 46is a block diagram showing an example of the configuration of a correction circuit50C in a third example. The correction circuit50C has the same circuit configuration as the correction circuit30C shown inFIG. 21except for the input signals.

Specifically, the correction circuit50C calculates differences between the signal Sig_C (Sig_R/Sig_Gr/Sig_Gb/Sig_B) from the correction object pixel and the signals (upper pixel: Sig_Up, right pixel: Sig_R, left pixel: Sig_L, and lower pixel: Sig_Lo) from the pixels adjacent to the correction object pixel horizontally and vertically and adds all the differences. The correction circuit50C, then, multiplies the addition result by a correction parameter K and adds the multiplication result to the original signal Sig_C to calculate a correction signal Sig_C′ (Sig_R′/Sig_Gr′/Sig_Gb′/Sig_B′).

This calculation process in the correction circuit50C can be represented by the following equation:

FIG. 47illustrates the correction model equation shown in (10). The concept of the correction model equation is the same as in the correction model equation inFIG. 43.

The correction circuits50A,50B, and50C in the first to third examples according to the third embodiment of the present invention have advantages similar to those of the correction circuits30A,30B, and30C in the first to third examples according to the first embodiment of the present invention.

Although the correction model equation (6) is used in the correction circuit50A in the first example, the correction model equations (7) to (9) are used in the correction circuit50B in the second example, and the correction model equation (10) is used in the correction circuit50C in the third example, the correction circuits50A,50B, and50C are not limited to the circuit configurations realizing the calculations in (6) to (10) because the present invention is not focused on the model equation itself.

In the correction of the color mixture according to the third embodiment, the pixel arrangement is in a square lattice, the color coding has, for example, a Bayer array, and the amount of correction of the color mixture from the neighboring pixels into the target pixel can be independently set for every neighboring pixel in the CMOS image sensor20C performing the vertical scanning in units of two rows, so that advantages similar to those in the first embodiment can be achieved. Also in the correction of the color mixture according to the third embodiment, the amount of correction may be varied for every color, as in the first embodiment.

Although the color coding has a Bayer array in the pixel arrangement in a square lattice in the third embodiment of the present invention, the present invention is not limited to the application to the Bayer array. The present invention is applicable to any color coding in a square lattice.

Fourth Embodiment

The correction of the color mixture in the case where the physical center of each pixel cell does not necessarily coincide with the optical center thereof to vary the color mixture ratio (degree) from the neighboring pixels into the target pixel is described in the first to third embodiments described above. However, it is known that the color mixture ratio is varied with the f-number (focal ratio) of the lens1ain the optical system1(refer toFIG. 1). The correction of the color mixture in a fourth embodiment of the present invention is based on the f-number.

FIGS. 48A and 48Bare conceptual diagrams showing the relationship between the aperture diameter (focal ratio/f-number) of the aperture1band the color mixture in the CMOS image sensor2.

The aperture diameter of the aperture1bis varied in accordance with the status of a subject or an instruction from a user in order to adjust the light intensity. Light through the aperture1bpasses through a condenser lens2aon the CMOS image sensor2and a color filter2bfor discriminating the color of the subject and is received by a pixel cell21. The color filter2bgenerally has three colors of R, G, and B and has the color coding used in the first, second, or third embodiment of the present invention.

If the aperture diameter of the aperture1bis small, that is, the f-number is large (FIG. 48A), light condensed on a pixel cell21passes through only the color filter corresponding to the pixel. However, if the aperture diameter of the aperture1bis increased, that is, the f-number is decreased (FIG. 48B), light through color filters that do not correspond to the pixel is filtered into the pixel cell21.

Accordingly, in the correction of the color mixture according to the fourth embodiment of the present invention, the values of the correction parameters Ka, Kb, Kc, and Kd used in the correction of the color mixture in the first to third embodiments are set in accordance with the aperture diameter, that is, the f-number of the aperture1bto constantly realize appropriate correction of the color mixture even if the f-number is varied due to the status of the subject or the instruction from the user. The setting of the correction parameters Ka, Kb, Kc, and Kd in accordance with the f-number is performed by the camera controller5shown inFIG. 1.

FIG. 49is a flowchart showing a correction process performed by the camera controller5. Specifically,FIG. 49shows a process of setting the correction values of the correction parameter Ka, Kb, Kc, and Kd in accordance with the f-number. This correction process is repeated every update period of an image.

After the correction process is started, in Step S11, the camera controller5reads the current f-number of the aperture1bfrom detected data supplied from the digital signal processing circuit4or from data set by the user, supplied from the human I/F controller6. The values of the correction parameters Ka, Kb, Kc, and Kd corresponding to the f-number are held in advance in a correction table (for example, a ROM table). In Step S12, the camera controller5reads out the correction values corresponding to the f-number, acquired in Step S11, from the correction table. In Step S13, the camera controller5sets the correction values read out from the correction table and transmits the set values to the color-mixture correction circuit11in the digital signal processing circuit4(refer toFIGS. 7 and 28).

The correction of the color mixture according to the first to third embodiments of the present invention is performed to the signal from the target pixel by using the signals from the multiple neighboring pixels adjacent to the target pixel and the correction parameters independently set for the signals. Setting the values of the correction parameters in accordance with the f-number (the aperture diameter of the aperture1b) in this correction of the color mixture can provide the directionality to the amount of correction of the color mixture from the neighboring pixels into the target pixel. In addition, it is possible to constantly perform appropriate correction of the color mixture eve if the f-number is varied due to the status of the subject or the instruction from the user.

Although the CMOS image sensor is exemplified as the solid-state imaging device in the above embodiments of the present invention, the present invention is not limited to the application to the CMOS image sensor. The present invention is applicable to amplified solid-state image sensors other than CMOS image sensors and, further, to any solid-state imaging device, such as charge transfer solid-state imaging devices typified by CCD image sensors.