Photoelectric conversion apparatus provided with arrangement of plurality of pixels each having plurality of photoelectric conversion devices and accumulating unit for temporarily accumulating charges accumulated in plurality of photoelectric conversion devices

A pixel unit in which a plurality of pixels shares an accumulating unit for temporarily accumulating charges accumulated in photoelectric conversion devices, is arranged so that a control unit of a photoelectric conversion apparatus is adapted, when an accumulated charge amount of a first photodiode exceeds a saturation charge amount, to effect control in accordance with a first operation to discharge excess charges to a floating diffusion FD, and adapted, when an accumulated charge amount of a second photodiode exceeds a saturation charge amount, to effect control in accordance with a second operation to discharge excess charges to a charge discharge area, thereby expanding a dynamic range.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a photoelectric conversion apparatus, and a method for controlling the same, and an image pickup apparatus.

2. Description of the Related Art

Conventionally, many methods for expanding a dynamic range of an image pickup device have been proposed.

Japanese Patent Application Laid-Open No. H11-313257 discloses a technique for outputting a signal corresponding to the logarithm of light incident on a photodiode (hereinafter referred to as “PD”).

Japanese Patent Application Laid-Open No. 2000-059688 discloses a technique for expanding the dynamic range by performing photoelectric conversion in both a PD and a floating diffusion (hereinafter referred to as “FD”).

Japanese Patent Application Laid-Open No. 2001-177775 discloses a technique for expanding the dynamic range by transferring charges generated in a PD to a FD a plurality of times.

In an image pickup device, when signal charges accumulated in the PD exceeds a saturation signal amount, the signal charges may leak into an area having a low potential barrier. The leaking state is shown inFIGS. 18A and 18B. InFIGS. 18A and 18B, the upper part illustrates a cross-sectional constitution of each unit, and the lower part illustrates a potential distribution of the each unit. The vertical axis of the potential distribution indicates the surface potential of a semiconductor substrate under an insulation layer. A lateral position in the potential distribution illustrated in the lower part corresponds to the lateral position in the cross-sectional constitution illustrated in the upper part.FIGS. 3A to 3D,FIGS. 5A to 5DandFIGS. 19A to 19Das will be described below are also similarly illustrated. As illustrated inFIG. 18A, signal charges generated in the PD is accumulated in a parasitic capacitance of the PD. When the signal charges accumulated in the parasitic capacitance of the PD exceeds a saturation signal amount, the signal charges exceeds the potential barrier of a transfer switch TX to leak into the FD, as illustrated inFIG. 18B.

Japanese Patent Application Laid-Open No. 2003-087665 discloses a technique for expanding the dynamic range by adding the signal charges leaking into the FD to signal charges generated in the PD by utilizing the phenomenon as illustrated inFIGS. 18A and 18B.

Further, in an image pickup device in recent years, in order to reduce the size of a pixel, attempts have been made to share a circuit (signal reading circuit) arranged in the pixel and used to read out signals of a PD, among a plurality of PDs to narrow the pixel pitch.

However, in the pixel structure sharing the signal reading circuit, it is very difficult to apply the method in which the dynamic range is expanded by trapping in the FD the excess charges generated in the PD. In the following, such a situation will be described with reference toFIGS. 19A to 19DandFIG. 20.

FIGS. 19A to 19Dare figures in which the upper part illustrates a cross-sectional constitution of each unit of the pixel structure, and the lower part illustrates the potential distribution of the each unit.FIG. 20is a figure illustrating a relation between an accumulated charge amount of each unit and an exposure time. InFIG. 20, the horizontal axis represents time and the vertical axis represents the charge amount.FIG. 19Acorresponds to a state of the pixel at time point t51inFIG. 20.FIG. 19Bcorresponds to a state of the pixel at time point t52inFIG. 20.FIG. 19Ccorresponds to a state of the pixel at time point t53inFIG. 20.FIG. 19Dcorresponds to a state of the pixel at time point t54inFIG. 20. Further,FIG. 20respectively illustrates a relation between an accumulated charge amount and an exposure time in a PD1, a relation between an accumulated charge amount and an exposure time in PD2, and a relation between an accumulated charge amount and an exposure time in an FD. Here, among the two PDs sharing a signal reading circuit, a PD which is read first is referred to as the PD1, and a PD which is subsequently read is referred to as the PD2. Further, it is assumed that the PD1is a pixel provided with a color filter having a higher sensitivity as compared with the PD2.

At time point t51(corresponding toFIG. 19A), the PD1, the PD2and the FD are reset so that signal charges start to be accumulated.

At time point t52(corresponding toFIG. 19B), the signal charges are accumulated in the PD1and the PD2.

At time point t53(corresponding toFIG. 19C), the accumulated charge amount of the PD1exceeds a saturation charge amount. In the PD1, the potential barrier on the side of the transfer switch TX1is lowest, and hence the signal charges generated in the PD1start to leak into the FD.

At time point t54(corresponding toFIG. 19D), the accumulated charge amount exceeds the saturation charge amount in the PD1and the PD2. The potential barrier on the side of a transfer switch TX2is lowest in the PD2. Therefore, the excessive charges exceeding the saturation charge amount in the PD1and the excessive charges exceeding the saturation charge amount in the PD2both start to leak into the FD, and the signal charges of the PD1and the PD2are intermixed in the FD.

As described above, when the signal charges leak into the FD from the plurality of pixels sharing the FD, the signal charges leaking from the PDs of the both pixels are intermixed. Therefore, in an image pickup device in which color filters are regularly arranged, information on an adjacent pixel having another color filter is intermixed, so that a signal which is different from the signal to be originally obtained, is generated. As a result, in the image pickup apparatus having the pixel structure sharing the FD, it has been very difficult to trap in the FD the excess charges generated in the PD, and to thereby expand the dynamic range.

The present invention has been made in view of the above described problem. An aspect of the present invention is to expand a dynamic range in a pixel structure in which an accumulating unit for temporarily accumulating charges accumulated in photoelectric conversion devices is shared by a plurality of pixels.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, a photoelectric conversion apparatus comprises pixels arranged two-dimensionally, each of which includes a first photoelectric conversion device and a second photoelectric conversion device respectively adapted to photo-electrically convert light to charges, an accumulating unit adapted to temporarily accumulate the charges accumulated in the first and second photoelectric conversion devices, and an output unit adapted to convert the charges transferred from the first and second photoelectric conversion devices to the accumulating unit to a voltage and to output the voltage, and a control unit adapted, when an accumulated charge amount of the first photoelectric conversion device exceeds a saturation charge amount, to effect control in accordance with a first operation for discharging excess charges in the first photoelectric conversion device to the accumulating unit, and adapted, when an accumulated charge amount of the second photoelectric conversion device exceeds a saturation charge amount, to effect control in accordance with a second operation for discharging excess charges in the second photoelectric conversion device to a charge discharge area.

According to another aspect of the present invention, a method for controlling a photoelectric conversion apparatus including pixels arranged two-dimensionally, each of which includes a first photoelectric conversion device and a second photoelectric conversion device respectively adapted to photo-electrically convert light to charges, an accumulating unit adapted to temporarily accumulate the charges accumulated in the first and second photoelectric conversion devices, and an output unit adapted to convert the charges transferred from the first and second photoelectric conversion devices to the accumulating unit to a voltage and to output the voltage, comprises effecting control in accordance with a first operation for discharging excess charges in the first photographic conversion device to the accumulating unit, when an accumulated charge amount of the first photoelectric conversion device exceeds a saturation charge amount, and effecting control in accordance with a second operation for discharging excess charges in the second photoelectric conversion device to a charge discharge area, when an accumulated charge amount of the second photoelectric conversion device exceeds a saturation charge amount.

According to another aspect of the present invention, a photoelectric conversion apparatus comprises: pixel units arranged two-dimensionally, each of which includes a first photoelectric conversion device and a second photoelectric conversion device respectively adapted to photo-electrically convert light to charges, an accumulating unit adapted to temporarily accumulate the charges accumulated in the first and second photoelectric conversion devices, a first transfer switch arranged between the first photoelectric conversion device and the accumulating unit, a second transfer switch arranged between the second photoelectric conversion device and the accumulating unit, and an output unit adapted to convert the charges transferred from the first and second photoelectric conversion devices to the accumulating unit to a voltage and to output the voltage; a charge discharge area; a first charge discharge switch arranged between the first photoelectric conversion device and the charge discharge area; and a second charge discharge switch arranged between the second photoelectric conversion device and the accumulating unit, wherein while charges are accumulated in the first and second photoelectric conversion devices, a first voltage is supplied to the first charge discharge switch and the second transfer switch, and a second voltage different from the first voltage is supplied to the second charge discharge switch and the first transfer switch.

According to still another aspect of the present invention, a method for controlling a photoelectric conversion apparatus including pixel units arranged teo-dimensionally, each of which includes a first photoelectric conversion device and a second photoelectric conversion device respectively adapted to photo-electrically convert light to charges, an accumulating unit adapted to temporarily accumulate the charges accumulated in the first and second photoelectric conversion devices, a first transfer switch arranged between the first photoelectric conversion device and the accumulating unit, a second transfer switch arranged between the second photoelectric conversion device and the accumulating unit, and an output unit adapted to convert the charges transferred from the first and second photoelectric conversion devices to the accumulating unit to a voltage and to output the voltage, comprises supplying a first voltage to the first charge discharge switch arranged between the first photoelectric conversion device and a charge discharge area, and the second transfer switch, and supplying a second voltage different from the first voltage to the second charge discharge switch arranged between the second photoelectric conversion device and the accumulating unit, and the first transfer switch, while charges are being accumulated in the first and second photoelectric conversion devices.

DESCRIPTION OF THE EMBODIMENTS

In the following, a first embodiment according to the present invention will be described in detail with reference to the drawings.

FIG. 1is a figure illustrating a constitution of an image pickup unit100according to a first embodiment of the present invention. In a photoelectric conversion apparatus101, a plurality of pixels is arranged in the vertical direction and the horizontal direction on an image pickup surface. A vertical selection unit102outputs a control pulse for reading out electric signals from pixels for each of row selection lines VSEL1to VSELm. The electric signals of the respective pixels selected by each of the row selection lines VSEL1to VSELm, are selected by a control pulse of column selection lines VSIG1to VSIGm applied by a horizontal selection unit104. The outputs of the pixels selected by the control pulse of the column selection lines VSIG1to VSIGm, are subjected to CDS (Correlated Double Sampling) processing by a CDS circuit103. The signals processed by the CDS circuit103are sequentially output from an output line105.

FIG. 2is a figure illustrating a detail of the pixels arranged in the photoelectric conversion apparatus101inFIG. 1. Reference numeral201denotes an area which includes two pixels sharing a signal reading circuit. Photodiodes PD1and PD2(hereinafter referred to as “PDs”) convert light to charges. Transfer switches TX1and TX2are driven by transfer pulses φTX1and φTX2and transfer the charges generated in the PD1and the PD2to an FD, respectively.

The FD is an accumulating unit (floating diffusion) for temporarily accumulating the charges. An amplifying MOS amplifier207functions as a source follower. A selection switch208selects the pixel by a vertical selection pulse φSEL. A reset switch209removes the charges accumulated in the FD by a reset pulse φRES. A floating diffusion amplifier (hereinafter referred to as “FDA”) is constituted of the FD, the amplifying MOS amplifier207, and a constant current source (not shown) connected to a signal reading circuit. The signal charges of the pixel selected by the selection switch208are converted to a voltage, which is output to the CDS circuit103inFIG. 1through a read-out output line210. The PD1and the PD2are independently connected to the transfer switches TX1and TX2, but shares the signal reading circuit constituted of the FD, the amplifying MOS amplifier207, the selecting switch208, and the reset switch209.

A state of charges in the pixels sharing the FD is described with reference toFIGS. 3A to 3DandFIG. 4.FIGS. 3A to 3Dare figures in which the upper part illustrates a cross-sectional constitution of each unit, and the lower part illustrates the potential distribution in the each unit.FIG. 4is a figure illustrating a relation between an accumulated charge amount and an exposure time of each unit. InFIG. 4, the horizontal axis represents time and the vertical axis represents the charge amount.FIG. 3Acorresponds to a state of the pixels at time point t71inFIG. 4.FIG. 3Bcorresponds to a state of the pixels at time point t72inFIG. 4.FIG. 3Ccorresponds to a state of the pixels at time point t73inFIG. 4.FIG. 3Dcorresponds to a state of the pixels at time point t74inFIG. 4. Further, inFIG. 4, there are respectively illustrated a relation between the accumulated charge amount of the PD1and the exposure time, a relation between the accumulated charge amount of the PD2and the exposure time, and a relation between the accumulated charge amount of the FD and the exposure time. Here, it is assumed that the PD1is a pixel having a color filter more sensitive as compared with the PD2.

At time point t71(corresponding toFIG. 3A), the PD1, the PD2and the FD are reset, and signal charges start to be accumulated.

At time point t72(corresponding toFIG. 3B), signal charges are accumulated in the PD1and the PD2.

At time point t73(corresponding toFIG. 3C), the accumulated charge amount of the PD1exceeds a saturation charge amount. In the PD1, a potential barrier on the side of the transfer switch TX1is lowest, so that signal charges generated in the PD1start to leak into the FD. Referring to the potential distribution illustrated on the lower side inFIG. 3C, the potential barrier of semiconductor area (p-type semiconductor) on the left-hand side of the PD1is sufficiently high, so that such potential distribution can be easily obtained unless a φTX1of a large negative potential is applied to the gate electrode of the TX1.

At time point t74(corresponding toFIG. 3D), the accumulated charge amount exceeds the saturation charge amount in the PD1and the PD2. At this time, in the PD2, a potential barrier on the side of an overflow drain (OFD) is lower than the potential barrier on the side of the transfer switch TX2. Thereby, excess charges accumulated in the PD2and exceeding the saturation charge amount start to leak into the OFD but not into the FD. When it is assumed that the constitution of the semiconductor area (channel) between the FD and the PD2is substantially the same as the constitution of the semiconductor area between the OFD and the PD2, such potential distribution can be obtained by applying a φTX2of a negative potential to the gate electrode of the TX2. This is because the potential below the transfer switch TX2is lowered and the potential barrier below the transfer switch TX2is raised, by applying the negative potential to the gate electrode of the TX2. In this way, at time point t74, the excess charges accumulated in the PD1and exceeding the saturation charge amount start to leak into the FD. The excess charges accumulated in PD2and exceeding the saturation charge amount start to leak into the OFD but not into the FD.

Next, with reference toFIG. 4andFIGS. 5A to 5D, there are described states of charges in the pixels in which both the PD1and the PD2are provided with the OFD.FIGS. 5A to 5Dare figures in which the upper part illustrates a cross-sectional constitution of each unit, and the lower part illustrates the potential distribution in the each unit.FIG. 5Acorresponds to a state of the pixels at time point t71.FIG. 5Bcorresponds to a state of the pixels at time point t72.FIG. 5Ccorresponds to a state of the pixels at time point t73.FIG. 5Dcorresponds to a state of the pixels at time point t74.

At time point t71(corresponding toFIG. 5A), the PD1, the PD2and the FD are reset and signal charges start to be accumulated.

At time point t72(corresponding toFIG. 5B), signal charges are accumulated in the PD1and the PD2.

At time point t73(corresponding toFIG. 5C), the accumulated charge amount of the PD1exceeds the saturation charge amount. At this time, in the PD1, the potential barrier on the side of the transfer switch TX1is lower than the potential barrier on the side of the OFD. Thereby, the excess charges accumulated in the PD1and exceeding the saturation charge amount start to leak into the FD but not into the OFD. When it is assumed that the constitution of the semiconductor area (channel) between the OFD and the PD1is substantially the same as the constitution of the semiconductor area between the FD and the PD1, such potential distribution can be obtained by applying the φTX1of a positive potential to the gate electrode of the TX1. This is because the potential below the transfer switch TX1is raised and the potential barrier below the transfer switch TX1is lowered, by applying the positive potential to the gate electrode of TX1. Note that depending upon the constitution of the semiconductor area (channel) between the OFD and the PD1, it is also possible to obtain the potential distribution as illustrated inFIG. 5Cby applying the φTX1of a zero potential or a low negative potential to the gate electrode of the TX1. In this way, at time point t73, the signal charges generated in the PD1starts to leak into the FD but not into the OFD.

At time point t74(corresponding toFIG. 5D), the accumulated charge amount exceeds the saturation charge amount in the PD1and the PD2. At this time, in the PD2, the potential barrier on the side of the OFD is lower than the potential barrier on the side of the transfer switch TX2. Thereby, the excess charges accumulated in the PD2and exceeding the saturation charge amount start to leak into the OFD but not into the FD. When it is assumed that the constitution of the semiconductor area (channel) between the FD and the PD2is substantially the same as the constitution of the semiconductor area between the OFD and the PD2, such potential distribution can be obtained by applying the φTX2of a negative potential to the gate electrode of the TX2. This is because the potential below the transfer switch TX2is lowered and the potential barrier below the transfer switch TX2is raised, by applying the negative potential to the gate electrode of the TX2. In this way, at time point t74, the excess charges accumulated in the PD1and exceeding the saturation charge amount start to leak into the FD. The excess charges accumulated in PD2and exceeding the saturation charge amount start to leak into the OFD but not into the FD.

In this way, even when the OFD is arranged only in the PD2as illustrated inFIGS. 3A to 3D, and even when the OFD is arranged in both the PD1and the PD2as illustrated inFIGS. 5A to 5DandFIGS. 6A to 6D, the excess charges in the PD1start to leak into the FD, and the excess charges in PD2start to leak into the OFD.

The charges which have leaked into the FD are read out and then added to the signals of the PD1, thereby enabling the expansion of the dynamic range by the same method as that for the conventional constitution in which the signal reading circuit is provided for one pixel.

For the purpose of description, two PDs share an FD, but the present invention is not limited to this. The number of PDs may be, for example, three or more. When two PDs shares an FD, the dynamic range can be more effectively expanded by the method as will be described below.

In the structure illustrated inFIGS. 5A to 5D, whether the charges are discharged to the OFD side or the FD side is determined by the voltages of the φTX1and the φTX2applied to the TX1and the TX2, respectively. However, a constitution as illustrated inFIGS. 6A to 6Dmay also be adopted.

FIGS. 6A to 6Dillustrate a constitution having charge discharge switches in addition to the transfer switches.FIG. 6Acorresponds to a state of pixels at time point t71inFIG. 4.FIG. 6Bcorresponds to a state of the pixels at the time point t72inFIG. 4.FIG. 6Ccorresponds to a state of the pixels at the time point t73inFIG. 4.FIG. 6Dcorresponds to a state of the pixels at the time point t74inFIG. 4.

At time point t71(corresponding toFIG. 6A), the PD1, the PD2and the FD are reset, and signal charges start to be accumulated.

At time point t72(corresponding toFIG. 6B), signal charges are accumulated in the PD1and the PD2.

At time point t73(corresponding toFIG. 6C), the accumulated charge amount of the PD1exceeds the saturation charge amount. In the PD1, the potential barrier on the side of the transfer switch TX1is lower than the potential barrier on the side of a charge discharge switch TOFD1. Thereby, the excess charges accumulated in the PD1and exceeding the saturation charge amount start to leak into the FD but not into the OFD. When it is assumed that the constitution of the semiconductor area (channel) between the FD and the PD1is substantially the same as the constitution of the semiconductor area between the OFD and the PD1, such potential distribution can be obtained by making the potential of the φTX1higher than the potential of a φTOFD1. This is because when the potential of the φTX1is made higher than the potential of the φTOFD1, the potential below the transfer switch TX1becomes relatively higher than the potential below the charge discharge switch TOFD1, and thereby the potential barrier below the transfer switch TX1becomes relatively lower than the potential barrier below the charge discharge switch TOFD1.

At time point t74(corresponding toFIG. 6D), the accumulated charge amount exceeds the saturation charge amount in both the PD1and the PD2. At this time, in the PD2, the potential barrier on the side of a charge discharge switch TOFD2is lower than the potential barrier on the side of the transfer switch TX2. Thereby, the excess charges accumulated in the PD2and exceeding the saturation charge amount start to leak into the OFD but not into the FD. When it is assumed that the constitution of the semiconductor area (channel) between the FD and the PD2is substantially the same as the constitution of the semiconductor area between the OFD and the PD2, such potential distribution can be obtained by making the potential of the φTX2lower than the potential of a φTOFD2. This is because when the potential of the φTX2is made lower than the potential of the φTOFD2, the potential below the transfer switch TX2becomes relatively lower than the potential below the charge discharge switch TOFD2, and thereby the potential barrier below the transfer switch TX2becomes relatively higher than the potential barrier below the charge discharge switch TOFD2. In this way, at time point t74, the excess charges accumulated in the PD1and exceeding the saturation charge amount start to leak into the FD. The excess charges accumulated in PD2and exceeding the saturation charge amount start to leak into the OFD but not into the FD.

FIG. 7is a figure illustrating a detail of the pixel illustrated inFIGS. 6A to 6D. The same units as those inFIG. 2are denoted by the same reference numerals and characters. The overflow drain OFD discharges excess charges and is connected to the PD1and the PD2via the charge discharge switches TOFD1and TOFD2, respectively. The charge discharge switches TOFD1and TOFD2are driven by charge discharge pulses φTOFD1and φTOFD2, respectively. Then, the excess charges in the PD1and the PD2are distributed to one of the OFD and the FD by the voltage values of the φTX1, the φTX2, the φTOFD1and the φTOFD2.

Next, a temporal relation between a read operation in the FD and a read operation in the PD is described with reference toFIG. 8which illustrates a temporal relation in driving the image pickup device. InFIG. 8, the horizontal axis represents time. For the purpose of description, the number of PDs sharing the FD is assumed to be four. First, the image pickup device resets the PD and the FD prior to photographing operation. Then, the image pickup device starts to accumulate charges so that signal charges are generated in the PD. The signal charges in the PD operating in the above described mode of PD1are discharged to the FD, when exceeding the saturation charge amount. The signal charges in the PD operating in the above described mode of PD2are discharged to the OFD, when exceeding the saturation charge amount. Which of PDa, PDb, PDc and PDd is set to the mode of PD1is determined by a color decision unit as will be described below. The PDs other than the mode of PD1operate as the mode of PD2. When the charge accumulation is ended, the signal of the FD is read out. Then, the FD is reset once, and the PDa set as the first PD is read out. Then, the FD is reset again, and the PDb set as the second PD is read out. This operation is repeated until the PDd set as the fourth PD is read. The signal of the FD is added to the read signal of one of PDa to PDd which is operated in the mode of the PD1, by a color decision operation unit as will be described below, so that the resultant signal is output as an image in which the saturation of one of the four PDs is expanded.

It is determined by the color decision unit which one of the PDs respectively provided with color filters performs the operation of PD1in which the signal amount is large. The way of determination is described with reference toFIGS. 9A and 9B. As illustrated by1201inFIG. 9A, the color decision unit performs a preliminary photographing operation in an image pickup device used for main photographing just before a photographer performs the main photographing operation. The picked up image is sent to a color decision operation unit. The color decision operation unit determines a color in which the signal amount is largest, based on an average value of a specific region and the like, and makes the PD having the color filter of the color operate in the above described mode of PD1. Then, in order to operate the remaining PDs in the above described mode of PD2, as illustrated by1202inFIG. 9A, the color decision operation unit sends to the image pickup device or an image pickup device driving unit (not shown) the information on the PD to be operated in the mode of PD1or the information to be operated in the mode of PD2. The image pickup device or the image pickup device driving unit (not shown) determines a drive pattern and performs the main photographing operation based on the information sent from the color decision operation unit. A state where the signal of the FD and the signals of the plurality of PDs are output as described above in the main photographing operation is illustrated by1203inFIG. 9A. As for the output signals, as illustrated by1204inFIG. 9A, the signal of the FD is added by a signal processing circuit to the signal of the PD operated in the mode of PD1among the signals of the plurality of PDs, based on the information sent from the color decision operation unit. The signals of the other PDs are set as the signal of the PD operated in the mode of PD2and processed without being added with the signal of the FD.

Further, as illustrated inFIG. 9B, in addition to the image pickup device used for the main photographing operation, the image pickup apparatus includes an apparatus for color determination separately, and estimates which one of PDs respectively provided with color filters generates signal amount which is most increased in an image to be photographed, based on color information from the apparatus for color determination. Then, the image pickup apparatus enables the estimated PD to operate in the mode of PD1. In this case, the exchange of data (1206to1208) except1205inFIG. 9Bis performed similarly to that inFIG. 9A.

The pixels each of which has a color filter are regularly arranged in the vertical direction and the horizontal direction, as illustrated inFIGS. 10A and 10B.FIG. 10Aillustrates an arrangement in which three kinds of color filters are used, andFIG. 10Billustrates an arrangement in which four kinds of color filters are used. Here, two pixels sharing the FD are assumed to be vertically arranged. As illustrated inFIG. 10B, when four kinds of color filters are used, a color filter having a highest sensitivity is arranged as the above described PD1. Since two pixels sharing the FD are vertically arranged side by side, a pixel arranged horizontally adjacent to the PD1is also the PD1. Thereby, it is possible to expand the dynamic range in the two colors. In the case as illustrated inFIG. 10B, from a viewpoint of expanding the dynamic range, it is more preferred to arrange two colors of color filters having a high sensitivity in a row in which the dynamic range is to be expanded.

When three kinds of color filters as illustrated inFIG. 10Aare used, two color filters having a highest sensitivity are often provided in the case of normal photographing. Thereby, it is possible to increase the charge amount handled by the PDs of colors other than that of the PD provided with the color filter having a lowest sensitivity, by setting the color filter having the lowest sensitivity in the row of PD2. As a result, it is possible to obtain an image in which the dynamic range of the pixels other than the pixel provided with the color filter having the lowest sensitivity is expanded.

Further, when the pixel provided with the color filter having the lowest sensitivity is saturated, it may be adapted such that the pixels at the periphery of the saturated pixel are also regarded to be saturated, and the signals are changed by signal processing.

Next, there is described an image pickup apparatus in which the image pickup unit100according to the present embodiment is incorporated.FIG. 11is a figure illustrating the image pickup apparatus according to the present embodiment.

A light beam is made incident on the image pickup unit100through an optical system1having an iris mechanism and lenses. A mechanical shutter2is arranged between the optical system1and the image pickup unit100or in the optical system1. The optical system1, the mechanical shutter2, and the image pickup unit100are driven by a drive circuit7. An A/D converter5converts an analog signal processed in the CDS circuit (CDS circuit103inFIG. 1) in the image pickup unit100to a digital signal. A timing signal generation circuit6generates a timing signal supplied to the image pickup unit100, the CDS circuit103, and the A/D converter5. A signal processing circuit8performs various signal processings to the A/D converted image data in addition to the above described signal processing. An image memory9is used to temporarily store the digital image signal being subjected to the signal processing and to store the image data which is the digital image signal subjected to the signal processing. A recording circuit11records the image data subjected to the signal processing in a recording medium10. A display circuit13supplies the image data subjected to the signal processing to an image display apparatus12, and to make the image display apparatus12display the image data.

A ROM15such as a nonvolatile memory stores a control program, control data such as a parameter and a table used in executing the control program, and correction data such as a defect address. The program, the control data and the correction data which are stored in the ROM15are transferred to a RAM16, so as to be used by a control unit14which performs control of the whole image pickup apparatus.

Prior to photographing operation, at a time when the control unit14starts to be operated, such as when the power source of the image pickup apparatus is turned on, a necessary program, control data and correction data are transferred from the ROM15to the RAM16. According to a control signal sent from the control unit14, the optical system1drives the iris and the lens, to enable an object image set to proper brightness to be formed on the image pickup device100. Next, according to the control signal sent from the control unit14, the mechanical shutter2is driven so as to shade the image pickup unit100in correspondence with the operation of the image pickup device. The image pickup unit100, which is driven by a driving pulse generated in the drive circuit7based on an operation pulse generated by the timing signal generation circuit6controlled by the control unit14, converts the object image to an electric signal by photoelectric conversion, and outputs the electric signal as an analog image signal. The analog image signal output from the image pickup unit100, whose clock synchronous noise is removed in the CDS circuit103based on the operation pulse generated by the timing signal generation circuit6controlled by the control unit14, is converted to a digital image signal by the A/D converter5. Next, in the signal processing circuit8controlled by the control unit14, image processing such as color conversion, white balance and gamma correction, resolution conversion processing, image compression processing, and the like are performed to the digital image signal. The image data subjected to the signal processing in the signal processing circuit8, and the image data stored in the image memory9are converted in the recording circuit11to data (for example, file system data having a hierarchical structure) suitable for the image recording medium10. Then, the data is stored in the recording medium10or subjected to the resolution conversion processing in the signal processing circuit8. The data subjected to the resolution conversion processing is thereafter converted in the display circuit13to a signal (for example, analog signal of the NTSC system or the like) suitable for the image display apparatus12, so as to be displayed in the image display apparatus12.

Here, in the signal processing circuit8, the digital signal may not be subjected to the signal processing but may be output as it is to the image memory9or the recording circuit11. Further, when there is a request from the control unit14, the signal processing circuit8outputs to the control unit14information on the digital image signal and the image data which are generated in the process of the signal processing. The information includes, for example, information on the spatial frequency of the image, the average value of a specified region, the amount of data of a compressed image and the like, or information extracted from these. Further, when there is a request from the control unit14, the recording circuit11outputs to the control unit14information on a type, a free space and the like of the image recording medium10.

FIG. 12is a figure illustrating a constitution of an image pickup unit100according to a second embodiment of the present invention. First, a plurality of pixels is arranged in the vertical direction and the horizontal direction in a photoelectric conversion apparatus101. A vertical selection unit102outputs a control pulse for reading electric signals from pixels for each of row selection lines VSEL1to VSELm. An electric signal of the respective pixels selected by the row selection lines VSEL1to VSELm is selected by a control pulse of the column selection lines VSIG1to VSIGm applied by the horizontal selection unit104. The output signal of the pixel selected by the control pulse of the column selection lines VSIG1to VSIGm, is subjected to the CDS (Correlated Double Sampling) processing by the CDS circuit103. The signal processed in the CDS circuit103is sequentially output from an output line105.

FIG. 13illustrates a detail of the pixel in the photoelectric conversion apparatus101inFIG. 12. Reference numeral601denotes a pixel area (pixel structure), wherein two pixels sharing a signal reading circuit are arranged. Photoelectric conversion devices PD1and PD2convert light into electric charges. Transfer switches TX1and TX2are driven by transfer pulses φTX1and φTX2, respectively, and transfer the charges generated in the PD1and the PD2to an FD, respectively.

The FD is an accumulating unit which temporarily accumulates the charges. An amplifying MOS amplifier207functions as a source follower. A selection switch208selects a pixel by a vertical selection pulse φSEL. A reset switch209removes the charges accumulated in the FD by a reset pulse φRES. An FDA is constituted of the FD, the amplifying MOS amplifier207, and a constant current source (not shown) connected to the signal reading circuit. The signal charge of the pixel selected by the selection switch208is converted to a voltage so as to be output to the CDS circuit103inFIG. 12via a read-out output line210.

An overflow drain OFD discharges excess charges. The OFD is connected to the PD1and the PD2via charge discharge switches TOFD1and TOFD2, respectively. The charge discharge switches TOFD1and TOFD2are driven by charge discharge pulses φTOFD1and φTOFD2, respectively. The charge discharge threshold values of the charge discharge switches TOFD1and TOFD2are determined by the voltage values of the charge discharge pulses φTOFD1and φTOFD2, respectively. In the exemplary embodiment according to the present invention, as will be described below, a common voltage is supplied to a part of the wirings to which the transfer pulses φTX1and φTX2and the charge discharge pulses φTOFD1and φTOFD2are applied, so that the voltages applied to the wirings are changed during a period when the signal charges are accumulated in the PD1and the PD2. Thereby, the excess charges exceeding the saturation charge amount of the PD1and the PD2can be accumulated in the FD and the OFD, respectively.

FIGS. 14A and 14Bare figures for describing the voltages applied to the charge discharge switches TOFD1and TOFD2and the transfer switches TX1and TX2, and the potential distribution of the respective units. The wiring connected to the TOFD1and the wiring connected to the transfer switch TX2are connected in common, and a common voltage V1is applied to the wirings. Further, the wiring connected to the gate of the TOFD2and the wiring connected to the gate of the transfer switch TX1are connected in common, and a common voltage V2is applied to the wirings. Note that inFIGS. 14A and 14B, for convenience of illustration, two OFDs are illustrated, but as shown inFIG. 13, one OFD is actually provided.

FIG. 14Aillustrates a case where V2is set to a voltage higher than V1. In this case, the potential barriers of the charge discharge switch TOFD1and the transfer switch TX2become higher than the potential barriers of the transfer switch TX1and the charge discharge switch TOFD2. The potential barrier in this case represents the potential barrier seen from an electron. Therefore, the excess charges exceeding the saturation charge amount of the PD1exceed the potential barrier of the transfer switch TX1to flow into the FD. The excess charge exceeding the saturation charge amount of the PD2exceeds the potential barrier of the charge discharge switch TOFD2to be discharged into the OFD.

FIG. 14Billustrates the case where V1is set to a voltage higher than V2. In this case, the potential barriers of the charge discharge switch TOFD1and the transfer switch TX2become lower than the potential barriers of the transfer switch TX1and the charge discharge switch TOFD2. As a result, the excess charges exceeding the saturation charge amount of the PD1exceed the potential barrier of charge discharge switch TOFD1to be discharged to the OFD. Further, the excess charges exceeding the saturation charge amount of the PD2exceed the potential barrier of transfer switch TX2to flow into the FD.

In this way, it is possible to accumulate the excess charges of the PD1and the PD2are respectively distributed and accumulated in the FD and the OFD so that the excess charges of the PD1and the PD2exceeding the respective saturation charge amounts are not intermixed in the FD and the OFD.

FIG. 15is a figure for describing in more detail the commonalization of the wirings as illustrated inFIG. 13andFIGS. 14A and 14B. The same constitutions as those inFIG. 13andFIGS. 14A and 14Bare denoted by the same reference numeral and characters. InFIG. 15, the circuit subsequent to the FD is omitted for convenience of illustration. As illustrated inFIG. 15, four kinds of color filters R, Gr, Gb and B are laminated on the respective PDs. The color filters Gr and B are laminated on the PD1and the PD2arranged in the pixel area601, respectively. The PD1and the PD2share one FD.

While the charges are accumulated in the PD1and the PD2, the voltage V1and the voltage V2higher than V1are applied. In this case, as illustrated inFIG. 14A, the excess charges exceeding the saturation charge amount of the PD1(R1) are accumulated in the FD, and the excess charges exceeding the saturation charge amount of PD2(Gb1) sharing the FD with the PD1(R1) are discharged to the OFD. Similarly, the excess charges exceeding the saturation charge amount of PD1(Gr1) are accumulated in the FD, and the excess charges exceeding the saturation charge amount of the PD2(B1) sharing the FD with the PD1(Gr1) are discharged to the OFD. Further, the excess charges exceeding the saturation charge amount of PD1(R3) are accumulated in the FD, and the excess charges exceeding the saturation charge amount of the PD2(Gb3) sharing the FD with the PD1(R3) are discharged to the OFD. Similarly, the excess charges exceeding the saturation charge amount of the PD1(Gr3) are accumulated in the FD, and the excess charges exceeding the saturation charge amount of the PD2(B3) sharing the FD with the PD1(Gr3) are discharged to the OFD.

Further, during the period when the charges are accumulated in the PD1and the PD2, the voltage V1and the voltage V2lower than V1may also be applied. In this case, as shown inFIG. 14B, the excess charges exceeding the saturation charge amount of the PD1(R1) are discharged to the OFD, and the excess charges exceeding the saturation charge amount of the PD2(Gb1) sharing the FD with the PD1(R1) are accumulated in the FD. Similarly, the excess charges exceeding the saturation charge amount of the PD1(Gr1) are discharged to the OFD, and the excess charges exceeding the saturation charge amount of the PD2(B1) sharing the FD with the PD1(Gr1) are accumulated in the FD. Further, the excess charges exceeding the saturation charge amount of the PD1(R3) are discharged to the OFD, and the excess charges exceeding the saturation charge amount of the PD2(Gb3) sharing the FD with the PD1(R3) are accumulated in the FD. Similarly, the excess charges exceeding the saturation charge amount of the PD1(Gr3) are discharged to the OFD, and the excess charges exceeding the saturation charge amount of the PD2(B3) sharing the FD with the PD1(Gr3) are accumulated in the FD.

Therefore, according to the constitution as illustrated inFIG. 15, it is possible to acquire the excess charges exceeding the saturation charge amount of the PD corresponding to either of the two color filters, and to expand the dynamic range of the PD.

Next, there is described a temporal relation between the read operation of FD and the read operation of PD with reference toFIG. 16which illustrates a temporal relation in driving the image pickup device.

InFIG. 16, the horizontal axis represents time. For the purposes of description, two PDs of the PD1and the PD2, which share the FD, are assumed to be provided. First, prior to photographing operation, the photoelectric conversion apparatus101resets the PDs and the FD. Then, signal charges are generated in the PD1and the PD2, and start to be accumulated in the PD1and the PD2. As described above, when exceeding the saturation charge amount, the signal charges in the specified PD (here, PD1) are accumulated in the FD. When the accumulation of the signal charges is ended, the signal charges accumulated in the FD are read out first. Then, the FD is reset once, and the signal charges accumulated in the PD1connected in common to the FD are read out. Then, the FD is reset again, and the signal charges accumulated in the PD2are read out. This operation is repeated so as to acquire the signals for one screen. The signals accumulated in the FD and based on the excess charges exceeding the saturation charge amounts of the PD1and the PD2are subjected to a known interpolation processing and the like, and thereafter synthesized with the signals from the PD1and the PD2. As a result, the image signals for one frame can be obtained.

Next, there is described an image pickup apparatus to which the image pickup unit100according to the present embodiment is applied.FIG. 17is a figure illustrating an image pickup apparatus, such as a digital camera, according to the present embodiment.

A light beam is made incident on the image pickup unit100through an optical system1having an iris mechanism and lenses. A mechanical shutter2is arranged between the optical system1and the image pickup unit100or in the optical system1. The optical system1, the mechanical shutter2, and the image pickup unit100are driven by a drive circuit7. An A/D converter5converts an analog signal processed in a CDS circuit (CDS circuit103inFIG. 1) in the image pickup unit100to a digital signal. A timing signal generation circuit6generates a timing signal supplied to the image pickup unit100, the CDS circuit103inFIG. 12, and the A/D converter5. A signal processing circuit8performs various signal processings to the A/D converted image data in addition to the above described signal processing. An image memory9stores the image data subjected to the signal processing. A recording circuit11records in a recording medium10the image data subjected to the signal processing. A display circuit13supplies the image data subjected to the signal processing to an image display apparatus12, and to make the image display apparatus12display the image data. A ROM15such as a nonvolatile memory stores a control program, control data such as a parameter and a table used in executing the control program, and correction data such as a defect address. The program, the control data and the correction data which are stored in the ROM15are transferred to a RAM16, so as to be used by a control unit14which performs control of the whole image pickup apparatus. Prior to photographing operation, at a time when the control unit14starts to be operated, such as when the power source of the image pickup apparatus is turned on, a necessary program, control data, and correction data are transferred to the RAM16from the ROM15.

According to a control signal sent from the control unit14, the optical system1drives the iris and the lens, to enable an object image set to a proper brightness to be formed on the photoelectric conversion apparatus (photoelectric conversion apparatus101inFIG. 12) in the image pickup unit100. Next, according to the control signal sent from the control unit14, the mechanical shutter2is driven so as to shade the image pickup unit100in correspondence with the operation of the photoelectric conversion apparatus (photoelectric conversion apparatus101inFIG. 12) in the image pickup unit100. At this time, when the image pickup unit100has an electronic shutter function, necessary exposure time may be secured by using the shutter function together with the mechanical shutter2. The image pickup unit100, which is driven by a driving pulse generated in the drive circuit7based on an operation pulse generated by the timing signal generation circuit6controlled by the control unit14, converts the object image to an electric signal by photoelectric conversion, and outputs the signal as an analog image signal. The analog image signal output from the image pickup unit100, whose clock synchronous noise is removed in the CDS circuit103inFIG. 12based on the operation pulse generated by the timing signal generation circuit6controlled by the control unit14, is converted to a digital image signal by the A/D converter5. Next, in the signal processing circuit8controlled by the control unit14, image processing such as color conversion, white balance and gamma correction, resolution conversion processing, image compression processing, and the like are performed to the digital image signal. The image memory9is used to temporarily store the digital image signal being subjected to the signal processing or to store the image data which is the digital image signal subjected to the signal processing. The image data subjected to the signal processing in the signal processing circuit8, and the image data stored in the image memory9are converted in the recording circuit11to data (for example, file system data having a hierarchical structure) suitable for the image recording medium10. Then, the data is stored in the recording medium10or subjected to the resolution conversion processing in the signal processing circuit8. The data subjected to the resolution conversion processing is thereafter converted in the display circuit13to a signal (for example, analog signal of the NTSC system or the like) suitable for the image display apparatus12, so as to be displayed in the image display apparatus12.

Here, in the signal processing circuit8, the digital image signal may not be subjected to the signal processing, but may be output as it is as image data to the image memory9or the recording circuit11. Further, when there is a request from the control unit14, the signal processing circuit8outputs to the control unit14information on the digital image signal and the image data, which is generated in the process of the signal processing. The information includes, for example, information on the spatial frequency of the image, the average value of a specified region, the amount of data of a compressed image and the like, or information extracted from these. Further, when there is a request from the control unit14, the recording circuit11outputs information on a type, a free space and the like of the image recording medium10to the control unit14.

Further, there is described a reproducing operation when an image data is recorded in the recording medium10. The recording circuit11reads the image data from the recording medium10according to a control signal from the control unit14. Similarly, when the image data is a compressed image, the signal processing circuit8performs image expansion processing of the image data according to the control signal from the control unit14, and stores the processed data in the image memory9. The image data stored in the image memory9is subjected to the resolution conversion processing in the signal processing circuit8, and thereafter converted in the display circuit13to a signal suitable for the image display apparatus12, so as to be displayed in the image display apparatus12.

This application claims the benefit of Japanese Patent Application Laid-Open No. 2006-273416 filed on Oct. 4, 2006, and No. 2006-273417 filed on Oct. 4, 2006, which are hereby incorporated by reference herein in their entirety.