Image sensor and control method thereof, and image capturing apparatus

An image sensor comprises: a plurality of pixels; a plurality of column output lines; and a control unit configured to control a signal to be output from pixels selected from the plurality of pixels to the plurality of column output lines, and each of the plurality of pixels includes: a photoelectric conversion portion; a floating diffusion portion for holding charge transferred from the photoelectric conversion portion; and an addition portion to add capacitance to the floating diffusion portion. The control unit controls to add the capacitance to the floating diffusion portion in a case where signals are simultaneously output to the same column line from the selected pixels.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to a technique of mixing signals read out from a plurality of pixels from an image sensor used in an image capturing apparatus, such as a camera, and outputting a mixed signal.

Description of the Related Art

Conventionally, an image capturing apparatus capable of shooting an image at high frame rate by mixing pixel signals and reading out the mixed signals at high speed from an image sensor has been realized. Further, various counter measures to cope with problems that arises when mixing pixel signals have been proposed.

For example, Japanese Patent Laid-Open No. 2010-259027 proposes that, in a case where signals from a plurality of pixels arranged in the same column are simultaneously read out to a vertical output line to mix the pixel signals, a current value on the vertical output line is controlled to an optimum value in accordance with the number of pixels to be mixed. More specifically, the current on the vertical output line is increased in a case where the pixel signals are mixed comparing to a case where the pixel signals are not mixed. Further, as the number of pixels to be mixed is increased, the current on the vertical output line is increased, thereby expanding the optimum range for mixing pixel signals. The reason for changing the current in this manner is that, in a case where an amplitude of a pixel signal, in other words, of a signal on the vertical output line is large and a difference between pixel signals to be mixed is large, circuit current of the pixel producing a larger signal decreases, which prevents the pixel signals from being mixed correctly, and thus it is necessary to compensate for the shortage of current in the vertical output line.

However, in order to optimally mix pixel signals as disclosed in Japanese Patent Laid-Open No. 2010-259027, energy consumption greatly increases.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the above situation, and is to optimally mix pixel signals on a column output line without increasing energy consumption of an image sensor.

Further, the present invention is to mix pixel signals on the column output line without deteriorating an S/N ratio.

According to the present invention, provided is an image sensor comprising: a plurality of pixels; a plurality of column output lines; and a control unit configured to control a signal to be output from pixels selected from the plurality of pixels to the plurality of column output lines, wherein each of the plurality of pixels includes: a photoelectric conversion portion; a floating diffusion portion for holding charge transferred from the photoelectric conversion portion; and an addition portion to add capacitance to the floating diffusion portion, and wherein the control unit controls to add the capacitance to the floating diffusion portion in a case where signals are simultaneously output to the same column line from the selected pixels.

Further, according to the present invention, provided is the image capturing apparatus comprising: an image sensor disclosed above; and a processing unit configured to process a signal output from the image sensor.

Furthermore, according to the present invention, provided is a control method of an image sensor having a plurality of pixels, a plurality of column output lines, and a control unit configured to control a signal to be output from pixels selected from the plurality of pixels to the plurality of column output lines, wherein each of the plurality of pixels includes a photoelectric conversion portion, a floating diffusion portion for holding charge transferred from the photoelectric conversion portion, and an addition portion to add capacitance to the floating diffusion portion, the method comprising: adding the capacitance to the floating diffusion portion by the addition portion in a case where signals are simultaneously output to the same column output line from the selected pixels.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described in detail in accordance with the accompanying drawings.

FIG. 1is a diagram schematically illustrating the configuration of the image sensor used in an image capturing apparatus according to an embodiment of the present invention. InFIG. 1, an image sensor100includes a pixel portion101, a vertical scanning unit102, a readout unit103, a horizontal scanning unit104, and a differential output unit110. The pixel portion101has a plurality of pixels200, each having a configuration that will be explained later, arranged in a matrix, and receives an optical image formed by an optical system (not shown).

The image sensor100may be provided with a timing generator or the like, for providing a timing signal to each circuit described above.

FIG. 2is a diagram illustrating a configuration of pixels of two rows in one column out of the pixels200forming the pixel portion101, and the readout unit103for a column.

A photodiode (PD)201generates charge (referred to as “light charge”, hereinafter) by performing photoelectric conversion on light entering via the optical system (not shown). A transfer MOS transistor202is controlled by a Ptx(n) signal (n is a variable indicating a row number), and transfers the charge generated by the PD201to the floating diffusion portion (FD)208when it is on. The transferred light charge is temporarily held in the FD208.

A reset MOS transistor203is controlled by a Pres(n) signal, and when it is turned on, the FD208is reset by a source voltage VDD. Further, by truing on the reset MOS transistor and the transfer MOS transistor202simultaneously, it is possible to reset the PD201.

The charge held in the FD208is output to the column output line210via an amplification MOS transistor204when a selection MOS transistor205controlled by a signal Psel(n) is turned on. The amplification MOS transistor204functions as a source follower amp when connected to a vertical output line load211via the selection MOS transistor205.

Further, a capacitor206is connected to the FD208via a FD addition switch207driven by a Pfd(n) signal. By turning on the FD addition switch207, the capacitor206is added and the capacitance of a floating diffusion portion is increased; as a result, it is possible to reduce an amplitude of a signal output from the amplification MOS transistor204.

A signal output to the column output line210is input to the readout unit103, and amplified by an amplifier212. At this time, an output (noise signal) from the pixels200, read out as will be explained later, right after the reset operation is ended is stored in a capacitor216via a MOS switch214which is controlled by a PTN pulse. Further, a signal obtained as a result of photoelectric conversion of an incident light (referred to as “light signal”, hereinafter) is held in a capacitor215via a MOS switch213which is controlled by a PTS pulse. The noise signal and the light signal read out to the capacitors216and215, respectively, of the readout unit103as described above are sequentially selected by a ph signal which is controlled by the horizontal scanning unit104on a column-by-column basis, and a signal obtained by taking the difference between the noise signal and the light signal by the differential output unit110is output.

The image capturing apparatus applies known processes, including image data generation for recording, to the signal output from the image sensor having the above configuration in a signal processing unit (not shown).

Next, driving method of the image sensor100having the above configuration will be explained with reference to timing charts shown inFIGS. 3A to 3C. Here, a case where the pixels200in the ith row and the i+1th row are read out from among the pixels200arranged in a matrix will be explained. InFIGS. 3A to 3C, the abscissa indicates time. Further, the row number to which each signal is applied is shown in parentheses after each signal name.

FIG. 3Ashows an operation for independently reading out signals from the pixels in the ith row and the i+1th row. As shown inFIG. 3A, a HD signal changes from a low level to a high level each time the signals are read out from pixels200of one row (i.e., for 1 horizontal period). Further, it is assumed that when each control signal is in a high level, a corresponding transistor or transistors turn on. For the sake of simplicity of explanation, it is assumed that charge accumulation is performed in the PD201of each pixel200before time T1, and the explanation after time T1is given below.

In the control shown inFIG. 3A, the Pfd(i) and Pfd(i+1) signals are always in a low level, and thus the FD addition switches207are kept off so as not to increase the capacitance of the floating diffusion (FD capacitance is kept to a normal capacitance).

First, at time T2, the HD signal changes from a low level to a high level and the Psel(i) signal becomes a high level, thereby the ith row is selected and the pixels200in the ith row are connected to the column output lines210. At this time, the Pres(i) is in a high level, and the FDs208of the pixels200in the ith row are reset to the source voltage VDD.

Next, during a period between times T3and T4, the PTS pulse and the PTN pulse become a high level, which turns on the MOS switches213and214, and the capacitors215and216are reset. At time T5, the Pres(i) signal becomes a low level, thereby the reset operation of the FDs208ends.

Thereafter, during a period between times T6and T7, the PTN pulse becomes a high level again and the MOS switches214are turned on, and an output (noise signal) from the pixels200in the ith row after the reset operation is ended is held in the capacitors216. Next, during a period between times T8and T9, the PTX(i) signal becomes a high level, and light change is transferred from the PDs201to the FDs208in all the pixels200in the selected row. After that, during a period between times T9and T10, the PTS pulse becomes a high level again, which turns on the MOS switches213, and an output (light signal+noise signal) from the pixels200in the ith row is held in the capacitors215.

Thereafter, during a period between times T10and T11, the light signal+noise signal held in the capacitors215and the noise signal held in the capacitors216are transferred to the differential output unit110provided in the downstream on a column-by-column basis by driving the ph pulse by the horizontal scanning unit104, and a light signal, which is a difference between the transferred signals, is output.

At time T11, the Pres(i) signal becomes a high level, and at T12, the HD signal changes from a low level to a high level again. At time T12and on, the Psel(i) signal becomes a low level and the Psel(i+1) signal becomes a high level so as to select the pixels200in the next row (here, the i+1th row), and signals of the pixels200in the i+1th row are read out in the same manner as the pixels200in the ith row. By repeating the aforesaid readout operation on a row-by-row basis, signals are read out from all the pixels200of one frame, and thus pixel signals of one frame representing a shot image of a subject can be obtained.

FIG. 3Bshows a driving method for mixing signals from the two pixels200in the vertical direction (column direction) by simultaneously connecting the pixels200of two rows, the ith and i+1th rows, to the column output lines210, and outputting the resultant signals.

The HD signal changes from a low level to a high level each time signals from the pixels200in the two rows (here, the ith row and the i+1th row) to be mixed are read out (i.e., for 1 horizontal period). Further, it is assumed that when each control signal is in a high level, a corresponding transistor or transistors turn on. Here, similarly toFIG. 3A, it is assumed that charge accumulation is performed in the PD201of each pixel200before time T1, for the sake of simplicity of explanation, and the explanation after time T1is given below.

It should be noted that in the control inFIG. 3B, the Pfd(i) and Pfd(i+1) signals are also always in a low level, and thus the FD addition switches207are kept off so as not to increase the FD capacitance (i.e., kept to a normal capacitance).

First at time T2, the HD signal changes from a low level to a high level and the Psel(i) and Psel(i+1) signals become a high level, thereby the pixels200in the ith and i+1th rows are connected to the column output lines210. At this time, the Pres(i) and Pres(i+1) signals are in a high level, and thus the FDs208of the pixels200in the ith and i+1th rows are reset to the source voltage VDD.

Next, during a period between times T3and T4, the PTS pulse and the PTN pulse become a high level, which turns on the MOS switches213and214, and the capacitors215and216are reset. At time T5, the Pres(i) and Pres(i+1) signals become a low level, thereby the reset operation of the FDs208ends.

Thereafter, during a period between times T6and T7, the PTN pulse becomes a high level again and the MOS switches214are turned on, and an output (noise signal) from the pixels200in the ith and i+1th rows after the reset operation is ended is held in the capacitors216. Next, during a period between times T8and T9, the Ptx(i) and Ptx(i+1) signals become a high level, and light charge is transferred from the PDs201to the FDs208in all the pixels200in the selected rows (here, the ith and i+1th rows). After that, during a period between times T9and T10, the PTS pulse becomes a high level again, which turns on the MOS switches213, an output (light signal and noise signal) from the pixels200in the ith and i+1th rows is held in the capacitors215.

Thereafter, during a period between times T10and T11, the light signal+noise signal held in the capacitors215and the noise signal held in the capacitors216are transferred to the differential output unit110provided in the downstream on a column-by-column basis by operating the Ph pulse by the horizontal scanning unit104, and a light signal, which is a difference between the transferred signals, is output.

At time T11, the Pres(i) and Pres(i+1) signals become a high level, and at time T12, the HD signal changes from a low level to a high level again. After time T12and on, the Psel(i) and Psel(i+1) signals become a low level, and the Psel(i+2) and Psel(1+3) signals become a high level so as to select the pixels200in the next two rows (here, the i+2th and i+3th rows). Then, signals of the pixels200in the next two rows are readout in the same manner as the pixels200in the ith and i+1th rows. By repeating the mixing of the pixel signals and reading of the mixed signals, signals are read out from all of the pixels200of one frame, and thus it is possible to obtain a frame of pixel signals each made from signals of two pixels mixed in the vertical direction (column direction) representing a shot image of a subject.

Similarly toFIG. 3B,FIG. 3Cshows a driving method for mixing signals from the two pixels200in the vertical direction (column direction) by simultaneously connecting the pixels200of two rows, the ith and i+1th rows, to the column output lines210, and outputting the resultant signals. A difference betweenFIGS. 3B and 3Cis that the Pfd(i) signal and the Pfd(i+1) signal are kept in a high level, namely, the FD addition switches207are on, thereby the FD capacitance is increased. Operation timing of other signals are the same as those shown inFIG. 3B, and thus the explanation thereof is omitted.

FIGS. 4A and 4Bshow graphs of voltage Vo of a column output line210with respect to voltage (FD voltage) Vfd of a floating diffusion portion. The abscissa indicates the FD voltage Vfd, and the ordinate indicates the voltage Vo of the column output line210. InFIGS. 4A and 4B, an explanation will be given of under assumption that in a case where signals are mixed and read out by two pixels200when the pixels200receive light from a high-contrast subject, one of the two pixels200receives light and the other pixel200does not receive light at all.

A line401shows relationship between the FD voltage Vfd and the voltage Vo of the column output line210in a case where only the pixel200which receives light is connected to the column output line210. A curve402shows relationship between the FD voltage Vfd of the pixel200which receives light and the voltage Vo of the column output line210in a case where the pixel200which receives light and the pixel200which does not receive light are simultaneously connected to the column output line210to mix the pixel signals.

FIG. 4Ashows a case in which signals of two pixels200in the vertical direction (column direction) are mixed and outputted by the operation explained with reference toFIG. 3B. As shown by the line401, in the pixel200that receives light, the voltage Vo of the column output line210linearly changes within a range (ΔV1) between Vfdres which is a reset voltage of the FD voltage Vfd and Vpdsatl which is the FD voltage Vfd when the PD is saturated and charge is transferred.

By contrast, the curve402showing a case in which pixel signals are mixed is ideally supposed to be a straight line having ½ of the tilt of the line401if signals from the two pixels200are properly mixed. However, the curve402starts to bent around a point where the FD voltage Vfd of the pixel200which receives light changes by ΔV2from the FD reset voltage Vfdres as an infection point, and eventually stops to change as described in Japanese Patent Application Laid-Open No. 2010-259027. In other words, in the bent part of the curve402, the signals are not mixed properly.

Accordingly, in order to properly mix signals from the two pixels200, it is desirable to drive within a range, as shown inFIG. 4B, where linearly is sufficiently secured. More specifically, the range is between the FD reset voltage Vfdres and a FD voltage when charge is transferred from a saturated PD, where the FD voltage Vfd of the pixel200changes by Vpdsat3(ΔV3).

As explained with reference toFIG. 3C, in a case where two pixel signals are to be mixed, the Pfd(n) signal is set to a high level to keep the FD addition switch207on, thereby it is possible to increase the FD capacitance. By doing so, it is possible to reduce a changing range of the FD voltage to a range of ΔV3shown inFIG. 4B.

Meanwhile, an image capturing apparatus such as a regular camera or the like generally has a function for changing a gain (ISO sensitivity setting) of an amplification unit provided downstream in accordance with a luminance of a subject to be shot. For example, a control is made such that, in a case of shooting a bright subject, a low sensitivity, such as ISO100, is set and the gain is decreased, whereas, in a case of shooting a dark subject, a high sensitivity, such as ISO1600, is set and the gain is increased. If the output range of a voltage after amplified by the gain in the amplification unit is constant independent of the ISO sensitivity, the voltage range (FD voltage range) of a floating diffusion portion is large when a low sensitivity is set, and the FD voltage range is small when a high sensitivity is set.

In a case where the ISO sensitivity is changed when the signals of two pixels200in the vertical direction (column direction) are mixed by the operation shown inFIG. 3C, the FD voltage range is as shown inFIG. 4B. For example, the FD voltage ranges for ISO100, ISO200 and ISO400 are ΔV3, ΔV4and ΔV5, respectively, where ΔV3=2×ΔV4=4×ΔV5. It is known that the light charge transferred from the PD decreases as the FD capacitance increases, and an S/N ratio deteriorates especially when high ISO sensitivity is set.

Therefore, it is possible to prevent the S/N ratio from deteriorating when high ISO sensitivity is set by limiting the operation for increasing the FD capacitance as shown inFIG. 3Cto be performed to when low ISO sensitivity is set, and by performing the operation for not increasing the FD capacitance as shown inFIG. 3Bwhen other ISO sensitivity is set. Although the FD voltage range is widened when the high ISO sensitivity is set, it is possible to properly mix signals output from the two pixels in the vertical direction (column direction) within a range where linearity of the voltage Vo of the column output line210can be secured.

Examples of combinations of the ISO sensitivity, FD capacitance, and gain of the amplification unit arranged downstream are shown inFIGS. 5A and 5B. InFIGS. 5A and 5B, “ISO” indicates ISO sensitivity, “Tfd” indicates on/off of the FD addition switch207driven by the Pfd signal, and “Cfd” indicates a ratio of total FD capacitance. In a case where the FD addition switch207is off (i.e., Tfd=OFF), the capacitor206is not connected and thus the total FD capacitance is ×1, and in a case where the FD addition switch207is on (i.e., Tfd=ON), the capacitor206is connected and thus the total FD capacitance becomes ×2. “Vfdr” indicates a ratio of the FD voltage range at the time of shooting a subject, and “gain” indicates a gain of the amplification unit arranged downstream. Here, the capacitance of the FD208and the capacitance of the capacitor206are assumed to be the same.

For all of the combinations shown inFIG. 5A, the FD addition switch207is off (Tfd=OFF) for every ISO sensitivity. These settings are used in an image shooting operation in a case where it is determined that the pixel signals are not mixed since the FD voltage range especially in the low ISO sensitivity is large (for example, a moving image shooting at a low frame rate, a still image shooting, and so forth). For example, the ratio Vfdr of the FD voltage range in ISO100 is 1 and the gain at that time is 1, by contrast, the ratio Vfdr of the FD voltage range in ISO200 is ½ and the gain at that time is 2.

In the operation shown inFIG. 5B, the FD addition switch207is turned on (Tfd=ON) only in a case where ISO100 is set in order to double the FD capacitance and halve the FD voltage range. In other ISO sensitivities, the FD addition switch207is set to off (Tfd=OFF). These settings are used in an image shooting operation in a case where it is determined that the pixel signals are to be mixed (for example, a moving image shooting at a high frame rate in which all pixel readout is not possible, and so forth). The ratio Vfdr of the FD voltage range in ISO100 is set to ½ and the gain at that time is set to ×2. For other ISO sensitivities, the same operation as those ofFIG. 5Ais performed.

In a case where pixel signals are to be mixed, as shown inFIG. 5B, the set ISO sensitivity is determined, and in the low ISO sensitivity, it is possible to optimally mix pixel signals by increasing the FD capacitance and amplifying a signal corresponding to a decreased portion of the FD voltage range. Further, since the FD capacitance is not increased in the other ISO sensitivities (high ISO sensitivities), it is possible to obtain pixel signals of a high S/N ratio.

FIG. 6is a diagram illustrating an equivalent circuit diagram of a pixel300of the image sensor according to a modification of the embodiment of the present invention. InFIG. 6, the same constituents as those shown inFIG. 2are referred to by the same reference numerals. The difference between the pixel300shownFIG. 6and the pixel200shown inFIG. 2is that a second capacitor310is connected to the FD208via a second FD addition switch309which is driven by a Pfd2signal. By turning on the second FD addition switch309, it is possible to increase a FD capacitance, and to decrease an amplitude of a signal output from the pixel300. Note that the Pfd2signal is also driven by the vertical scanning unit102shown inFIG. 1.

The operating method of the image sensor100comprising the pixels300having the structure as shown inFIG. 6is explained for each ISO sensitivity with reference toFIGS. 7A and 7B. In the table shown inFIGS. 7A and 7B, “Tfd2” for indicating ON/OFF of the second FD addition switch309driven by the Pfd2signal is added. Here, the capacitance of the second capacitor310is twice as large as the capacitance of the FD208and the capacitor206.

When both of the FD addition switch207and the second FD addition switch309are off (Tfd=OFF, Tfd2=OFF), the capacitors206and310are not connected, and the total FD capacitance is ×1. When the FD addition switch207is on (Tfd=ON) and the second FD addition switch309is off (Tfd2=OFF), the capacitor206is connected but the capacitor310is not connected, therefore the total FD capacitance becomes ×2. Further, when both of the FD addition switch207and the second FD addition switch309are on (Tfd=ON, Tfd2=ON), the capacitors206and310are connected, and the total FD capacitance becomes ×4. “Vfdr” indicates a ratio of the FD voltage range when shooting a subject, and “gain” indicates a gain of the amplification unit arranged downstream.

In the combinations shown inFIG. 7A, the FD addition switch207and the second FD addition switch309are off (Tfd=OFF, Tfd2=OFF) for every ISO sensitivity. This operation is used in an image shooting in a case where it is determined that pixel signals are not mixed since the FD voltage range is large especially in a low ISO sensitivity (for example, a moving image shooting at low frame rate, a still image shooting, and so forth). For example, in ISO100, the ratio Vfdr of the FD voltage range is 1 and the gain at that time is ×1, and in ISO200, the ratio Vfdr of the FD voltage range is ½ and the gain at that time is ×2.

In the operation shown inFIG. 7B, both of the FD addition switch207and the second FD addition switch309are turned on (Tfd=ON, Tfd2=ON) in a case of ISO100, thereby the FD capacitance is increased by ×4, and the ratio Vfdr of the FD voltage range is set to ¼. Further, in a case of ISO200, the FD addition switch207is turned on (Tfd=ON) and the second FD addition switch309is turned off (Tfd2=OFF), thereby the FD capacitance is increased by ×2, and the ratio Vfdr of the FD voltage range is set to ¼. In the other ISO sensitivities, both of the FD addition switch207and the second FD addition switch309are turned off (Tfd=OFF, Tfd2=OFF).

The operation as shown inFIG. 7Bis performed in an image shooting in a case where it is determined that pixel signals are to be mixed (for example, a moving image sensing at a high frame rate in which all pixel readout is not possible, and so forth). Especially, this operation is very effective in an image sensor having an especially small FD capacitance in a case where an extra FD capacitance is not added. In this example, in ISO100, the ratio Vfdr of the FD voltage range is ¼ and the gain at that time is ×4. Further, in ISO200, the ratio Vfdr of the FD voltage range is ¼ and the gain at that time is ×4, and in other ISO sensitivities, the same operation as that ofFIG. 7Ais performed.

By operating an image sensor100as shown inFIG. 7B, in a case where pixel signals are mixed in the image sensor100having a particularly small FD capacitance, the FD capacitance is increased when a low ISO sensitivity is set and a portion corresponding to a reduced amount of the FD voltage range is amplified in downstream, it is possible to optimally mix pixel signals. Further, since the FD capacitance is not increased in other ISO sensitivities (high ISO sensitivities), it is possible to obtain a pixel signal of a high S/N ratio.

Note that in the above embodiment and modification, pixels whose pixel signals are added are two successive pixels as shown inFIG. 2, however, the present invention is not limited thereto. For example, the present invention is applicable to a case where signals from two or more same color pixels in the Bayer arrangement are mixed.

Further, the selection MOS transistor205is used inFIGS. 2 and 6, however, the present invention is not limited thereto. For example, other circuit structures capable of activating the amplification MOS transistor204to output to the column output line210may be used.

Furthermore, inFIGS. 2 and 6, the sources of addition switch207and the second FD addition switch309are connected to the capacitors206and310whose other terminals are grounded, however, parasitic capacitance may be used instead if the parasitic capacitance is large enough to realize the present invention.

Further, as shown inFIG. 6for explaining the modification, by further adding a FD addition switch and a capacitor and appropriately controlling on/off of a FD addition switch, it is possible to mix pixel signals more appropriately.

Further, in the above embodiment and modification, it is assumed that pixel signals are mixed and read out, and the FD capacitance is increased in a case of low ISO sensitivity (a predetermined capacitance or less). However, it is possible to configure the pixel200or300such that the FD capacitance is increased in a case where pixel signals are to be mixed and read out regardless of the ISO sensitivity.

Further, in the above embodiment and modification, whether or not to increase the FD capacitance is controlled by taking the ISO sensitivity as a condition for the operation, however, the present invention is not limited to the ISO sensitivity, and can be controlled on the basis of the brightness of a subject. In this case, when the photometry value is larger than a predetermined value and the variation of voltage across the FD208does not fall within the range ΔV3inFIG. 4B, it may be considered to control the FD capacitance being increased.

This application claims the benefit of Japanese Patent Application No. 2014-109430, filed on May 27, 2014, which is hereby incorporated by reference herein in its entirety.