Imaging device

In an imaging device, a first digital signal corresponding to a first analog signal read out from a pixel, and a second digital signal corresponding to the amount of charge accumulated during a first exposure period following a period for reading the first analog signal. A difference is acquired between a first difference and a second difference wherein the first difference is a difference between a third digital signal and the second digital signal, the third digital signal is a digital signal corresponding to the amount of charge cumulatively accumulated during the first exposure period and a following second exposure period, and the second difference is a difference between the second digital signal and the first digital signal. At least one of the first exposure period or the second exposure period includes a period during which the first light source is in the on-state.

BACKGROUND

1. Technical Field

The present disclosure relates to an imaging device.

2. Description of the Related Art

Japanese Unexamined Patent Application Publication No. 2011-228648 (hereinafter, for simplicity, referred to as PTL 1) discloses an imaging device having an organic photoelectric conversion layer supported on a semiconductor substrate via an insulating layer. A configuration in which a photoelectric converter having a plurality of pixel electrodes is disposed above a semiconductor substrate instead of an embedded photodiode as described PTL 1 is called a “stacked type”. In such a configuration, the semiconductor substrate supporting the photoelectric converter has a plurality of readout circuits corresponding to respective pixels each having a pixel electrode. As described in FIG. 1 of PTL 1, the pixel electrode of each pixel is connected to a corresponding one of the readout circuits through a via formed in the insulating layer.

The imaging device can be used not only for acquiring a still image or a moving image of a person, a scene, or the like, but also can be used for acquiring information in terms of intensity of reflected light for calculating a distance to a subject located in front of the imaging device. For example, International Publication No. 2016/157593 discloses a technique for acquiring a distance image indicating a distance to a subject by irradiating the subject with a two-dimensional light pattern and capturing an image of the subject.

In the field of imaging devices, there is a demand for noise reduction. In particular, there is a demand for reducing kTC noise which is generated when a charge generated by a photoelectric conversion is reset. The kTC noise is also called reset noise. Japanese Unexamined Patent Application Publication No. 2008-028517 discloses a technique in which data of a reset level is stored in a frame memory, and a difference between a signal level and the reset level is acquired. In this technique, the reset level is subtracted from the signal level via digital processing thereby cancelling an influence of shot noise caused by a dark current generated in memory means in each pixel.

SUMMARY

In a so-called stacked configuration, a pixel electrode and a readout circuit on a semiconductor substrate are electrically connected through a via which is typically formed of metal. In this configuration, it is fundamentally difficult to completely transfer a signal charge to the readout circuit by providing a structure using an impurity region or the like in the semiconductor substrate for temporarily accumulating the signal charge and transferring the accumulated signal charge to the readout circuit via a transistor as disclosed, for example, in Japanese Unexamined Patent Application Publication No. 2008-028517 (hereafter for simplicity referred to PTL 3). That is, it is difficult to simply apply a noise cancellation technique conventionally used for an imaging device having an embedded photodiode such as that described PTL 3. In particular, in ranging, random noise typified by the reset noise may cause a reduction in accuracy of a calculated distance.

In one general aspect, the techniques disclosed here feature an imaging device including a plurality of pixels each of which includes a charge accumulation region and a readout circuit and accumulates a charge depending on an amount of exposure, a first frame memory, a second frame memory, an image processing circuit, and a first light source that emits infrared light having a peak at a first wavelength, wherein the first frame memory temporarily stores a first digital signal corresponding to a first analog signal read through the readout circuit, the second frame memory temporarily stores a second digital signal corresponding to a second analog signal depending on an amount of charge accumulated in the pixel during a first exposure period following a reading period in which the first analog signal is read out, the image processing circuit outputs a difference between a first difference and a second difference, the first difference being a difference between a third digital signal and the second digital signal, the third digital signal corresponding to a third analog signal corresponding to an amount of charge cumulatively accumulated in the pixel during the first exposure period and a second exposure period following the first exposure period, the second difference being a difference between the second digital signal and the first digital signal, and at least one of the first exposure period or the second exposure period includes a period for which the first light source is in an on-state.

General or specific embodiments may be implemented by a component, a device, a system, an integrated circuit, a method, or a computer program. Furthermore, the general or specific embodiments may be realized by any combination of a component, a device, an apparatus, a system, an integrated circuit, a method, and a computer program.

According to an embodiment of the present disclosure, it is possible to realize an imaging device capable of obtaining an image based on light with a specific wavelength in a form in which reset noise is suppressed.

DETAILED DESCRIPTION

An overview of aspects of the present disclosure is described below.

Aspect 1: An imaging device according to Aspect 1 includes

a plurality of pixels each of which includes a charge accumulation region and a readout circuit and accumulates a charge depending on an amount of exposure,

a first frame memory,

a second frame memory,

an image processing circuit, and

a first light source that emits infrared light having a peak at a first wavelength,

wherein the first frame memory temporarily stores a first digital signal corresponding to a first analog signal read through the readout circuit,

the second frame memory temporarily stores a second digital signal corresponding to a second analog signal depending on the amount of charge accumulated in the pixel during a first exposure period following a reading period in which the first analog signal is read out,

the image processing circuit outputs a difference between a first difference and a second difference, the first difference being a difference between a third digital signal and the second digital signal, the third digital signal corresponding to a third analog signal corresponding to an amount of charge cumulatively accumulated in the pixel during the first exposure period and a second exposure period following the first exposure period, the second difference being a difference between the second digital signal and the first digital signal, and

at least one of the first exposure period or the second exposure period includes a period for which the first light source is in an on-state.

According to Aspect 1, it is possible to obtain an image based on specific light with which a subject is intentionally irradiated during one exposure period such that the influence of charge generated by photoelectric conversion of, for example, ambient light is substantially cancelled while canceling random noise.

Aspect 2: In the imaging device according to Aspect 2 based on Aspect 1, the first analog signal is a reset signal representing a reset level of the pixel.

Aspect 3: In the imaging device according to Aspect 3 based on Aspect 1 or 2, the first light source is set to be in the on-state during the second exposure period of the first exposure period and the second exposure period.

According to Aspect 3, the digital signal or pixel value that substantially represents an image obtained by shooting performed under only the light emitted from the first light source is obtained in a form in which the influence of the reset noise is cancelled.

Aspect 4: The imaging device according to Aspect 4 based on Aspect 3 further includes a second light source that emits light having a peak at a second wavelength different from the first wavelength, wherein the first exposure period includes a period during which the second light source is set to be in the on-state.

Aspect 5: In the imaging device according Aspect 5 based on one of Aspects 1 to 4,

each pixel includes

a semiconductor substrate on which the readout circuit is disposed, and

a photoelectric converter which is located above the semiconductor substrate and is electrically connected to the readout circuit.

Embodiments of the present disclosure are described below in detail with reference to the drawings. Note that any of the embodiments described below illustrate a general or a specific example. In the following embodiments of the present disclosure, values, shapes, materials, constituent elements, locations of the constituent elements and manners of connecting the constituent elements, steps, the order of steps, and the like are described by way of example but not limitation. Various aspects described herein can be combined with each other as long as there is no conflict. Among constituent elements described in the following embodiments, those constituent elements that are not described in independent claims indicating highest-level concepts of the present disclosure are optional. In the following description, constituent elements having substantially the same function are denoted by common reference numerals, and a duplicated description thereof may be omitted. Some constituent elements may be omitted in figures in order to avoid overly complicated drawings.

Embodiments of Imaging Device

FIG.1schematically illustrates an example of a configuration of an imaging device according to an embodiment of the present disclosure. The imaging device100shown inFIG.1includes a plurality of pixels Px each having, as one of its elements, a photoelectric converter supported on a semiconductor substrate110. That is, the imaging device100according to this embodiment of the present disclosure is formed by way of example in a so-called stacked configuration.

The plurality of pixels Px are arranged, for example, in a two-dimensional array on the semiconductor substrate110thereby forming an imaging region. The number and arrangement of pixels Px are not limited to those shown inFIG.1, but the number and arrangement of pixels Px are arbitrary. For example, a plurality of pixel Px may be one-dimensionally arranged thereby realizing the imaging device100functioning as a line sensor. As will be described in detail below with reference to the drawings, the semiconductor substrate110has a plurality of readout circuits each formed corresponding to one pixel Px.

The imaging device100includes a plurality of row signal lines Riand a plurality of output signal lines Sj. InFIG.1, a plurality of pixels Px are arranged in an array with m rows and n columns, and the plurality of row signal lines Riinclude m row signal lines Ri(i=0, 1, 2, . . . , m−2, m−1) arranged corresponding to respective rows of pixels Px. Similarly, the plurality of output signal lines Sjinclude n output signal lines Sj(j=0, 1, 2, . . . , n−2, n−1) arranged corresponding to respective columns of pixels Px. Note that m and n each take independently an integer equal to or greater than of 1.

Each of the row signal lines Riis electrically connected to one or more pixels Px belonging to the same row. These row signal lines Riare connected to a row scanning circuit130. Note that two or more signal lines may be provided for each row of pixels Px. Similarly, each of the output signal lines Sjis electrically connected to readout circuits of one or more pixels Px belonging to the same column. The plurality of output signal lines Sjare connected to an analog-to-digital conversion circuit140and further to a digital output interface160. The digital output interface160outputs signals read from readout circuits of pixels Px in each column. Hereinafter, for simplicity, the analog-to-digital conversion circuit140will be referred to simply as the “A/D conversion circuit140”, and the digital output interface160will be referred to simply as the “interface160”.

As shown inFIG.1, the imaging device100further includes a digital memory150connected between the A/D conversion circuit140and the interface160. The digital memory150temporarily stores signals read from a plurality of pixels Px of one row. By interposing the digital memory150between the A/D conversion circuit140and the interface160, it becomes possible to output the digital signals in units of rows at a higher speed.

The interface is connected to an image processing circuit170. The image processing circuit170performs processes such as a gamma correction, a color interpolation process, a spatial interpolation process, an auto white balance process on the digital signal output from the interface160as necessary. The image processing circuit170may be implemented, for example, by a digital signal processor (DSP), an image signal processor (ISP), a field-programmable gate array (FPGA), or the like. In the configuration illustrated by way of example inFIG.1, the imaging device100further includes a display device180, such as a liquid crystal display or an organic EL display, connected to the image processing circuit170. The display device180displays an image based on a digital signal obtained by imaging a subject thereby presenting the image to a user of the imaging device100.

In the configuration illustrated by way of example inFIG.1, the image processing circuit170includes a first frame memory172and a second frame memory174. Each of the first frame memory172and the second frame memory174temporarily stores digital data corresponding to one frame output from the interface160. Note that the first frame memory172and the second frame memory174each store digital data of one frame such that frames stored therein are different between the first and second frame memories. As described later, the image processing circuit170executes a process of determining a difference between the digital signal stored in the first frame memory172and the digital signal stored in the second frame memory174.

In this example, the imaging device100further includes a light source200. In an exemplary embodiment of the present disclosure, the light source200is an infrared light source that emits infrared light. An infrared laser may be used as the light source200. In particular, a light source called an eye-safe later, which emits light in a wavelength range of about 1.4 micrometers, may be advantageously used for the light source200. In this specification, for convenience, general electromagnetic waves including infrared light and ultraviolet light may be referred to as “light”.

Turning on/off of the light source200is controlled by a light source control apparatus210. The image processing circuit170receives, for example, command data, and a clock, or the like from the outside of the imaging device100, and supplies a light emission timing signal, synchronized with the reading of a signal from the pixel Px, to the light source control apparatus210. That is, in the embodiment of the present disclosure, the light source200is configured to operate in synchronization with reading signals from a plurality of pixels Px. The light source control apparatus210includes a switching element which is turned on/off according to the light emission timing signal supplied from the image processing circuit170.

In this example, a control circuit250is electrically connected to the image processing circuit170. The image processing circuit170provides control signals such as a vertical synchronization signal and a horizontal synchronization signal to the control circuit250. The row scanning circuit130and the A/D conversion circuit140are connected to the control circuit250. The control circuit250controls the entire imaging device100. The control circuit250may be implemented, for example, by a microcontroller including one or more processors, and typically includes a timing generator. The control circuit250supplies drive signals to the row scanning circuit130and the A/D conversion circuit140. InFIG.1, an arrow extending toward the control circuit250and arrows extending from the control circuit250schematically represent an input signal to the control circuit250and output signals from the control circuit250, respectively. The control circuit250may include one or more memories.

As will be described in detail later with reference to drawings, in a typical embodiment of the present disclosure, a difference between a first digital signal output from interface160and a second digital signal representing an image for a frame period that includes a first exposure period as a part thereof is calculated via digital processing, where the first digital signal is, for example, a digital signal corresponding to a reset signal read from each pixel after a reset operation performed before the start of the first exposure period. The first digital signal includes reset noise, and pixel exposure is performed following reading of the reset signal, and thus the second digital signal also includes reset noise similar to that included in the first digital signal. Thus, the difference between the first and second digital signals substantially does not include reset noise.

In the typical embodiment of the present disclosure, furthermore, a difference between a third digital signal and the second digital signal is calculated via digital processing, where the third digital signal represents an image for a frame period following the above-described frame period and includes a second exposure period as a part thereof. Note that the reset operation of the pixel Px is not performed between the two adjacent frame periods described above. Therefore, signals read out from the respective pixels in a reading period in a subsequent frame period are signals corresponding to the total amount of charge cumulatively accumulated during the first exposure period and the second exposure period. The third digital signal also includes reset noise as with the second digital signal. The influence of the reset noise is substantially canceled by taking a difference between the third digital signal and the second digital signal.

One of the first exposure period or the second exposure period includes a period for which the light source200is in the on-state, and the light source200is in the off-state for the other of the first and second exposure periods. That is, one of the second digital signal or the third digital signal is a signal based on light reflected from a subject under ambient light such as sunlight or light from a luminaire, and the other is a signal based on light reflected from the subject under the condition in which infrared light or the like is intentionally superimposed on the ambient light. Therefore, as will be described in further detail later, by further calculating the first difference and the second difference where the first difference is a difference between the first digital signal and the second digital signal, and the second difference is a difference between the second digital signal and the third digital signal, it is possible to obtain image data substantially based on reflected light from the subject wherein the reflected light originates from light which is emitted from the light source to intentionally strike the subject during one of the exposure periods. Note that the influence of the reset noise is substantially eliminated in the difference between the first difference and the second difference. That is, according to the embodiment of the present disclosure, it is possible to obtain an image based on specific light which is emitted so as to intentionally strike the subject during one of the exposure periods in a form in which random noise and an effect of a charge generated by photoelectric conversion of ambient light are both cancelled.

Example of Circuit Configuration of Pixel Px

FIG.2illustrates an example of a circuit configuration of the imaging device100. For the sake of simplicity,FIG.2schematically shows four pixels Px included in the imaging region shown inFIG.1. These four pixels Px include a first pixel Px1, a second pixel Px2, a third pixel Px3, and a fourth pixel Px4arranged in an array of two rows and two columns. Of these pixels, the first pixel Px1and the second pixel Px2are located in one row, while the third pixel Px3and the fourth pixel Px4are located in another row different from the row in which the first pixel Px1and the second pixel Px2are located. The basic circuit configuration of the pixels is common for these pixels Px1to Px4, and thus, an example of a configuration of a pixel is described below with reference to the first pixel Px1.

The first pixel Px includes a photoelectric converter10and a readout circuit20electrically connected to the photoelectric converter10. As will be described later, the photoelectric converter10includes a pixel electrode, a counter electrode, and a photoelectric conversion layer disposed between these electrodes. The photoelectric converter10of each pixel includes an electrical connection with a voltage line192connected to a voltage supply circuit190, thereby allowing it to apply a particular voltage between the pixel electrode and the counter electrode in an operation of the imaging device100. The voltage supply circuit190is not limited to a particular power supply circuit as long as the particular voltage can be applied to the photoelectric converter10of each pixel in the operation of the imaging device100. The voltage supply circuit190may be a circuit that generates the particular voltage, or a circuit that converts a voltage supplied from another power supply to the particular voltage. The voltage supply circuit190may be a part of the row scanning circuit130.

In the example of the configuration illustrated inFIG.2, the readout circuit20includes a signal detection transistor22, an address transistor24, and a reset transistor26. The signal detection transistor22, the address transistor24, and the reset transistor26are each typically a field effect transistors formed on the semiconductor substrate110. The following description is given, by way of example, for a case where N-channel MOSFETs are used for these transistors.

A gate of the signal detection transistor22is connected to the pixel electrode of the photoelectric converter10. A source of the signal detection transistor22is connected to a corresponding output signal line Sjvia the address transistor24. The first pixel Px1and the fourth pixel Px4belong to the same column, and the sources of the signal detection transistors22included in the readout circuits20of these pixels are both electrically connected to the same output signal line Sj. As schematically shown inFIG.2, the A/D conversion circuit140may include a plurality of elements such as column signal processing circuits143each of which is provided for corresponding one of output signal lines Sj. Each of the elements is connected to a corresponding one of the output signal lines. On the other hand, a drain of the signal detection transistor22is connected to a power supply line194. The power supply line194functions as a source-follower power supply when a power supply voltage VDD of about 3.3 V is applied in operation of the imaging device100.

A row signal line Riis connected to the gate of the address transistor24. The row scanning circuit130switches the state of the address transistor24between the on-state and the off-state by controlling the voltage level applied to the row signal line Ri. By performing this switching operation, the row scanning circuit130can read signals from the pixels Px belonging to the selected row to the corresponding output signal line.

In this example, the readout circuit20includes a reset transistor26. One of the drain or the source of the reset transistor26is connected to a node FD via which the photoelectric converter10is electrically connected to the gate of the signal detection transistor22. The other of the drain and the source of the reset transistor26is connected to a reset voltage line196. The reset voltage line196is connected to a reset voltage supply circuit198. In operation of the imaging device100, a particular reset voltage VRSTis applied from the reset voltage supply circuit198to the reset voltage line196. As the reset voltage VRST, for example, a voltage of 0 V or a voltage about 0 V is used. The reset voltage supply circuit198, as with the voltage supply circuit190, is not limited to a particular power supply circuit as long as it is capable of applying the particular reset voltage to each pixel in operation of the imaging device100. The reset voltage supply circuit198may be a circuit independent of the voltage supply circuit190. One of the reset voltage supply circuit198or the voltage supply circuit190may be a part of the other.

A plurality of reset signal lines Qiare provided corresponding to a plurality of pixels Pw. As shown inFIG.2, typically, one reset signal line Qiis connected in common to gates of reset transistors26of pixels Px belonging to one same row. In this example, each reset signal line Qihas a connection with the row scanning circuit130. Therefore, by controlling the voltage level applied to the reset signal lines Qi, the row scanning circuit130can turn on a plurality of reset transistors Px26at a time on a row by row basis thereby resetting, to VRST, the voltage of the nodes FD of the pixels Px whose reset transistors26are turned on.

Device Structure of Pixel Px

FIG.3schematically illustrates a device structure of the first pixel Px1. The first pixel Px1includes mainly the semiconductor substrate110on which the readout circuit20is formed, and the photoelectric converter10supported on the semiconductor substrate110. As shown inFIG.3, typically, an insulating layer50covering the readout circuit20is disposed between the semiconductor substrate110and the photoelectric converter10.

The photoelectric converter10includes a pixel electrode11supported on the insulating layer50, a translucent counter electrode13, and a photoelectric conversion layer12disposed between the pixel electrode11and the counter electrode13. The pixel electrode11is located closer to the semiconductor substrate110than to the photoelectric conversion layer12. The pixel electrode11may be formed of a metal such as aluminum or copper, a metal nitride, or polysilicon doped with impurities so as to be conductive. As shown inFIG.3, the pixel electrode11is spatially separated from other pixel electrodes11of other adjacent pixels, thereby being electrically separated from them.

The counter electrode13is located on a side where light from the subject is incident. The counter electrode13is a translucent electrode formed of a conductive material such as ITO. Note the term “translucent” is used in this specification to indicate that the counter electrode13is capable of transmitting light with at least a part of the wavelengths that can be absorbed by the photoelectric conversion layer12, and it is not essential that light is transmitted over the entire visible wavelength range. On a principal surface of the counter electrode13opposite to the photoelectric conversion layer12, an optical filter such as a color filter, a microlens, or the like may be disposed.

The counter electrode13is typically provided in the form of a single electrode layer continuous across a plurality of pixels. The voltage line192described above is connected to the counter electrode13of the photoelectric converter10.FIG.2indicates that one voltage line192is connected to photoelectric converters10of each group of pixels. However, typically, the counter electrode13of each pixel is part of the single translucent electrode that is continuous over the plurality of pixels, and therefore, the counter electrode13of the pixels is basically equipotential, and thus it is not essential that the voltage line192is branched into a plurality of lines.

The photoelectric conversion layer12is formed of an organic material or an inorganic material such as amorphous silicon such that the photoelectric conversion layer12generates a pair of charges in response to receiving incident light transmitted through the counter electrode13. Like the counter electrode13, the photoelectric conversion layer12is typically provided in the form of a single photoelectric conversion structure that is continuous over a plurality of pixels. That is, the photoelectric conversion layer12in each pixel may be part of the photoelectric conversion layer that is continuously formed over the plurality of pixels.

By forming the photoelectric conversion layer12with a photoelectric conversion material containing one or more suitable materials, it becomes possible to achieve the photoelectric conversion layer12which is sensitive, for example, to both visible light region and infrared light. Examples of such materials are described in detail, for example, in International Publication No. 2018/025544. The entire disclosure of International Publication No. 2018/025544 is incorporated herein by reference. The photoelectric conversion layer12may be formed by quantum dots and/or nanotubes. Alternatively, the photoelectric conversion layer12may include quantum dots and/or nanotubes functioning as a photoelectric conversion material. The photoelectric conversion layer12may include a layer formed of an organic material and a layer formed of an inorganic material.

The insulating layer50located between the semiconductor substrate110and the photoelectric converter10includes a plurality of insulating layers each formed of, for example, silicon dioxide. As schematically shown inFIG.3, a multilayer wiring including at least a conductive structure52whose one end is connected to the pixel electrode11of the photoelectric converter10is provided inside the insulating layer50. The conductive structure52may include a via and wiring formed of a metal such as copper, a plug formed of polysilicon, and the like. In the example illustrated, the other end of the conductive structure52is electrically connected to an impurity region111formed in the semiconductor substrate110.

The semiconductor substrate110includes, in addition to the impurity region111, impurity regions112,113,114, and115. The semiconductor substrate110further includes an element isolation region116for electrically isolating the readout circuits20provided for the respective pixels Px between the pixels Px. In the following description, a P-type silicon substrate is used by way of example as the semiconductor substrate110. The semiconductor substrate110may be an insulating substrate on which a semiconductor layer is provided.

Each of the impurity regions111,112,113,114and115is typically an N-type diffusion region. Among these impurity regions, the impurity region111is connected to the conductive structure52and functions as one of the source region and the drain region of the reset transistor26. The reset transistor26further includes an impurity region112functioning as the other of the source region and the drain region, a gate insulating layer26gon the semiconductor substrate110, and a gate electrode26eon the gate insulating layer26g. Although not shown inFIG.3, the above-described reset voltage line196is connected to the impurity region112.

The signal detection transistor22includes an impurity region113and an impurity region114, a gate insulating layer22gon the semiconductor substrate110, and a gate electrode22eon the gate insulating layer22g. The impurity region113functions as a drain region of the signal detection transistor22, and the impurity region114functions as a source region of the signal detection transistor22. The impurity region113is connected to the power supply line194described above. As schematically shown inFIG.3, an element isolation region116is also provided between the signal detection transistor22and the reset transistor26.

The address transistor24includes an impurity region114and an impurity region115, a gate insulating layer24gon the semiconductor substrate110, and a gate electrode24eon the gate insulating layer24g. The impurity regions114and115respectively function as a drain region and a source region of the address transistor24. In the configuration illustrated by way of example inFIG.3, the address transistor24shares the impurity region114with the signal detection transistor22. The impurity region115is connected to corresponding one of the plurality of output signal lines Sj.

The insulating layer50covers the signal detection transistor22, the address transistor24, and the reset transistor26. As schematically shown inFIG.3, the conductive structure52in the insulating layer50also has an electrical connection with the gate electrode22eof the signal detection transistor22. That is, the conductive structure52in each pixel has a function of electrically connecting the pixel electrode11of the photoelectric converter unit10to the readout circuit20including the signal detection transistor22etc. formed on the semiconductor substrate110.

The conductive structure52also functions as part of a charge accumulation region that temporarily accumulates the charge collected by the pixel electrode11, that is, the signal charge. As described above with reference toFIG.2, the voltage supply circuit190applies a particular voltage to the photoelectric converter10of each pixel via the voltage line192. For example, by applying a voltage to the counter electrode13of the photoelectric converter10, a particular voltage difference ΔV can be applied between the counter electrode13and the pixel electrode11for the exposure period. For example, by applying a voltage to the counter electrode11such that the potential of the counter electrode13is higher than that of the pixel electrode11with reference to the pixel electrode13, it is possible to collect a charge by the pixel electrode11. More specifically, positive and negative charges are generated by incident light in the photoelectric conversion layer12, and the positive charge (holes) is collected as the signal charge by the pixel electrode11. The signal charge is temporarily accumulated in the charge accumulation region including, as part thereof, the conductive structure52. As with the conductive structure52, the impurity region111formed in the semiconductor substrate110, the pixel electrode11of the photoelectric converter10, and the gate electrode22eof the signal detection transistor22also function as part of a charge accumulation region for temporarily accumulating signal charges. Example of method of driving imaging device100

FIG.4is a diagram for explaining an example of a method of driving an imaging device according to an embodiment of the present disclosure. InFIG.4, a chart on the top shows pulses of the vertical synchronization signal VD. A rising edge of a pulse of the vertical synchronizing signal VD indicates a start of a frame period that includes, as part thereof, an exposure period for accumulating a signal charge. InFIG.4, a chart in a second row from the top shows pulses of the horizontal synchronization signal HD. A period from a rise of one pulse to a rise of a next pulse corresponds to one horizontal scanning period 1H. InFIG.4, a chart in a third row from the top shows on/off timing of the light source200controlled by the light source control apparatus210.

InFIG.4, a plurality of blocks included in the imaging region are also shown to indicate operations thereof. For the sake of simplicity, it is assumed here that the plurality of pixels Px are arranged in 6 rows from a 0th row R0to a 5th row R5, and operations of the pixels Px are schematically indicated by a plurality of rectangular blocks. InFIG.4, for example, open rectangular blocks schematically represent exposure periods in frame periods. Rectangular blocks hatched with vertical lines indicate periods for reading reset levels corresponding to signal levels in a dark state. Rectangular blocks hatched with diagonal lines indicate periods for reading pixel signals representing an image of a subject.

FIG.4illustrates an example of an operation based on a so-called rolling shutter in which the exposure and signal reading are executed in units of rows of pixels. In the following explanation, first, the 0th row R0is taken for explanation from the 0th row R0to the 5th row R5. In acquiring an image, first, resetting of the charge accumulation region of each pixel Px is performed. In the example shown inFIG.4, resetting of a plurality of pixels belonging to a 0th row R0is started at time t0according to the vertical synchronization signal VD.

More specifically, the reset transistor26is turned on thereby setting the potential of the node FD so as to be equal to the potential of the reset voltage line196. That is, the voltage of the pixel electrode11of the photoelectric converter10is set to be equal to the reset voltage VRST. As can be seen fromFIGS.2and3, when the gate electrode22eof the signal detection transistor22of the readout circuit20is electrically connected to the pixel electrode11via the conductive structure52, the signal detection transistor22outputs a signal corresponding to the potential of the pixel electrode11. That is, the readout circuit20outputs an analog signal corresponding to the potential of the pixel electrode11to the corresponding output signal line Sjvia a source follower including the signal detection transistor22.

Thereafter, the reset transistor26is turned off and then the address transistor24is turned on. As a result, a signal corresponding to the reset voltage VRSTapplied to the gate electrode22eof the signal detection transistor22is output to the output signal line Sj. The signal output to the output signal line Sjin this situation is an analog signal representing a reset level, which usually includes reset noise generated when the reset transistor26is turned off. Hereinafter, for convenience, the analog signal representing the reset level is referred to simply as a reset signal.

The reset signal read to the output signal line Sjis converted into a digital signal by the A/D conversion circuit140. After the reset signal is read out, the address transistor24is turned off. As schematically shown inFIG.4, in synchronization with the horizontal synchronization signal HD, the reading operation described above is sequentially executed on a row by row basis. The pulse interval of the horizontal synchronizing signal HD, that is, 1H period, represents a period from a selection of one row to a selection of a next row. In this example, pixels belonging to the 0th row R0are reset and signals are read out from these pixels in a period from time t0to time t1, and pixels belonging to the 1st row R1are reset and signals are read out from these pixels in a period from time t1to time t2. The operation is performed sequentially in a similar manner for the 2nd row R2and subsequent rows. As can be seen from the above description, the reset level reading period may include a reset period in which the potential of the charge accumulation region of the pixel is reset. InFIG.4, the digital signal corresponding to the reset signal read out between time t0and time t6is stored in the first frame memory172. That is, at a point of time when the reset level reading period is ended, the first frame memory172is in a state in which image data corresponding to one frame of digital signal is stored.

Referring again to pixels belong to the 0th row R0, after the reset signal is read out, an exposure period is started. In the present example, for the 0th row R0, a period from time t1to time t8is an exposure period in a kth frame period for the 0th row R0, where k is an integer equal to or greater than 0. The exposure period is a period for accumulating the signal charge corresponding to the amount of exposure to the pixel in the charge accumulation region. The length of the exposure period for the plurality of pixels Px in each row is in the range of, for example, 1/60 seconds to 1/16000 seconds.

The counter electrode13of the photoelectric converter10of each pixel Px is supplied with a particular voltage V1from the voltage supply circuit190via the voltage line192such that the counter electrode13is at a potential higher than that of the pixel electrode11. In a state immediately after the resetting is performed, the potential of the pixel electrode11is determined by the reset voltage VRSTdescribed above, and a bias voltage equal to V1−VRSTis applied between the pixel electrode11and the counter electrode13.

Since the potential of the counter electrode13is relatively higher than that of the pixel electrode11, positive charge of charge pairs generated via the photoelectric conversion is collected by the pixel electrode11. A PN junction formed in the semiconductor substrate110, which is formed as a result of the formation of the impurity region111, provides a junction capacitance for temporarily storing the positive charge collected by the pixel electrode11. In the case where holes are used as signal charges, the potential of the impurity region111functioning as the charge accumulation region increases as the signal charge is accumulated in the impurity region111. In exemplary embodiments of the present disclosure, V1−VRST>0. However, a voltage may be applied to the counter electrode11such that the potential of the counter electrode13is lower than that of the pixel electrode13, and electrons may be used as signal charges.

After a predetermined time elapses, the pixel signal reading process is performed. In the present example, reading of signals from pixels belonging to the 0th row R0is started at time t8according to the vertical synchronization signal VD. More specifically, the address transistors24in the readout circuit20of the pixels in the 0th row R0are turned on. As described above, the readout circuit20outputs an analog signal corresponding to the potential of the pixel electrode11to the corresponding output signal line Sj. The signal read out from the pixel of the 0th row R0is an analog signal corresponding to the amount of charge accumulated in the charge accumulation region in the exposure period for the 0th row R0, and this analog signal represents an image of a subject based on environmental light such as sunlight. Hereinafter, for convenience of description, the analog signal corresponding to the amount of charge accumulated in the exposure period in a kth frame period is referred to as a first pixel signal. The first pixel signal includes reset noise generated when a reset operation is performed before the exposure period. After the first pixel signal is read out, the address transistor24is again turned off.

The first pixel signal read to the output signal line Sjvia the readout circuit20is converted to a digital signal by the A/D conversion circuit140. In this embodiment of the present disclosure, the digital signal generated by the A/D conversion circuit140by the A/D conversion of the first pixel signal is temporarily stored in the second frame memory174.

As for the 1st row R1to the 5th row R5, the above-described exposure operation and the signal reading operation are sequentially performed on a row by row basis in a period from time t9to time t14. When the reading of the first pixel signals for the 0th row R0to the 5th row R5is completed, the kth frame period ends. At this point of time, the second frame memory174is in a state in which image data corresponding to one frame of a digital signal is stored.

Next, acquiring of image data of a (k+1)th frame period following the kth frame period is performed. In this acquisition process, the process of accumulation a signal charge in each pixel Px and the process of reading a signal are basically the same as those in the kth frame period. However, pixels Px are not reset between the kth frame period and the (k+1)th frame period.

In the example shown inFIG.4, for the case of the 0th row R0, the exposure period in the (k+1)th frame period starts at time t8. In this embodiment of the present disclosure, the resetting of pixels Px is not performed between the exposure period in the kth frame period and the following exposure period, that is, the exposure period in the (k+1)th frame period. Therefore, in the reading of the first pixel signal via the readout circuit20, flowing of the signal charge out of the charge accumulation region and further flowing of the charge into the charge accumulation region basically do not occur. That is, the reading of the first pixel signal via the readout circuit20is performed nondestructively. Therefore, when the exposure period in the (k+1)th frame period starts, signal charges generated by photoelectric conversion in the exposure period in the (k+1)th frame period are accumulated in the charge accumulation regions of the respective pixels Px such that the signal charges are added to the signal charges accumulated in the previous exposure period.

In the case of the 0th row R0, in this example, the period from time t9to time t16at which reading of the analog signal via the readout circuit20is started corresponds to the exposure period of the (k+1)th frame period. Here, in the present embodiment of the disclosure, one of the exposure period in the kth frame period or the exposure period in the (k+1)th frame period includes a period for which the light source200is in the on-state. In the example shown inFIG.4, the light source is in the on-state for a period from time t14to time t16.

For the period in which the light source200is in the on-state, the subject is irradiated with light, for example, infrared light emitted from the light source200such that the light is superimposed on environmental light. As a result, signal charges corresponding to the intensity of light including the infrared light reflected from the subject are accumulated in the charge accumulation regions of the respective pixels Px. That is, the increase in the potential of the charge accumulation region caused by cumulative accumulation of signal charges during a period starting from the exposure period of the kth frame period includes an increase caused by the irradiation of infrared light. This increase in potential reflects the amount of signal charge obtained when the subject is irradiated only with the infrared light. In particular, in this example, as schematically shown inFIG.4, the length of the on-period of the light source200included in the exposure period is common among the rows of pixels Px. Therefore, the pixel signal corresponding to the increase in the signal charge caused by turning on the light source200represents the image of the subject based on the infrared light emitted from the light source200.

As for the light emitted from the light source200, infrared light having a peak at a first wavelength in the infrared region may be used. It may be advantageous to use infrared having the first wavelength peak in a range, for example, from 1300 nm to 1500 nm, because a wavelength that is not included in the spectrum of sunlight can be effectively utilized. By selecting a wavelength which is not included in the spectrum of sunlight as the first wavelength, it is possible to perform imaging while suppressing the influence of disturbance of light. In applications such as distance measurement using the light from the light source200, the light source200emits light having a pattern such as stripes or random dots such that a reflected light pattern changes depending on the unevenness of the subject surface.

After the exposure period ends, in the same manner as the kth frame period, the readout of the pixel signals is sequentially executed for each row of the plurality of pixels Px. In this example, the reading of a plurality of pixels belonging to the 0th row R0is started from the time t16according to the vertical synchronization signal VD. An analog signal read from each pixel via the readout circuit20in this process is a signal corresponding to an amount of charge cumulatively accumulated in the pixel during the exposure period in the kth frame period and the exposure period in the (k+1)th frame period. Hereinafter, this analog signal is referred to as a second pixel signal. When the reading of the second pixel signal from the plurality of pixels belonging to the 5th row R5is completed, the (k+1)th frame period ends. In this example, the (k+1)th frame period ends at time t22.

The second pixel signal read to the output signal line Sjvia the readout circuit20is converted into a digital signal by the A/D conversion circuit140and output to the image processing circuit170in the same manner as the first pixel signal. The image processing circuit170calculates the difference the digital signal corresponding to the second pixel signal, for example, of the (k+1)th frame period and the digital signal corresponding to the first pixel signal of the kth frame period temporary stored in the second frame memory174. Hereinafter, for convenience, this difference is referred to as the “first difference”.

During the exposure period of the (k+1)th frame period, signal charges corresponding to the intensity of light reflected from a subject are cumulatively accumulated in the charge accumulation regions of the respective pixels, wherein the light reflected from the subject includes infrared light originating from the light source200. That is, the signal charge accumulated during the exposure period of the (k+1)th frame period includes a charge generated by the photoelectric conversion of the ambient light reflected by the subject, and a charge generated by the photoelectric conversion of light which is emitted from the light source200and reflected by the subject. As for the amount of the former charge, the amount of charge accumulated in the charge accumulation region of the pixel during the exposure period of the (k+1)th frame period has substantially no difference from that accumulated during the exposure period of the kth frame period if the length of the exposure period is substantially the same between the (k+1)th frame period and the kth frame period. Therefore, the first difference calculated as a difference between the digital signal corresponding to the second pixel signal and the digital signal corresponding to the first pixel signal corresponds to the sum of the amount of signal charge reflecting the intensity of light reflected from the subject under ambient light such as sunlight or light from a lighting fixture and the amount of signal charge obtained when the subject is irradiated only with pure infrared light.

Here, since the first pixel signal includes reset noise, the digital signal stored in the second frame memory174includes a signal component corresponding to the reset noise. However, in the embodiment of the present disclosure, the pixels are not reset between two adjacent frame periods, and thus the signal component corresponding to the reset noise is also included in the digital signal corresponding to the second pixel signal of the (k+1)th frame period. Therefore, the signal component corresponding to the reset noise disappears in the first difference obtained by digitally determining the difference between the digital signal stored in the second frame memory174and the digital signal corresponding to the second pixel signal.

The image processing circuit170calculates the difference between the digital signal temporarily stored in the second frame memory174and the digital signal corresponding to the reset signal temporarily stored in the first frame memory172. Hereinafter, for convenience, this difference is referred to as the “second difference”. As described above, the digital signal temporarily stored in the first frame memory172includes the signal component corresponding to the reset noise. However, the signal component corresponding to the reset noise is also removed via the process of calculating the second difference. That is, the second difference corresponds to the amount of signal charge that reflects the intensity of light reflected from a subject under ambient light such as sunlight or light emitted from a luminaire.

The image processing circuit170outputs the difference between the first difference and the second difference. Here, the image processing circuit170outputs, as a final signal, a digital signal obtained by subtracting the second difference from the first difference. The first difference corresponds to the sum of the amount of signal charge that reflects the intensity of light reflected from the subject under ambient light such as sunlight or light emitted from the luminaire and the amount of signal charge obtained when the subject is irradiated only with infrared light, while the second difference corresponds to the former of these. Therefore, the difference between the first difference and the second difference represents an image obtained when shooting is performed under only the light emitted from the light source200. That is, it is possible to obtain image data based on the charge generated by the photoelectric conversion of the component of the light emitted from the light source200and reflected by the subject. Note that the output obtained here does not include the signal component corresponding to the reset noise. Therefore, it is possible to construct an image relating to the first wavelength in a form in which the influence of the reset noise is canceled, and thus it is possible to achieve an improvement in the accuracy of measurement in applications of distance measurement.

As described above, according to the embodiment of the present disclosure, it is possible to obtain image data relating to a specific wavelength in a form in which the influence of reset noise is canceled. In the above description, it is assumed by way of example that the image processing circuit170calculates the first difference and the second difference, and further calculates the difference between the first and second differences. However, it is not essential that the intermediate value corresponding to the first difference and the intermediate value corresponding to the second difference are stored in the memory or the like as long as the digital signal or the pixel values substantially representing the image obtained when shooting is performed under only the light emitted from the light source200.

Alternatively, the difference between the second difference and the first difference may be determined as follows. First, the second difference is determined by calculating the difference between digital signal corresponding to the first pixel signal temporarily stored in the second frame memory174and the digital signal corresponding to the reset signal temporarily stored in the first frame memory172, and the resultant second difference is overwritten in the first frame memory172. After that, the first difference is determined by calculating the difference between the digital signal corresponding to the second pixel signal and the digital signal temporarily stored in the second frame memory174, and the result is overwritten in the second frame memory174. Then, the difference is calculated between the result of the calculation of the second difference stored in the first frame memory172and the result of the calculation of the first difference stored in the second frame memory174.

In the process described above, the light source200is set to be in the on-state during the exposure period of the (k+1)th frame period following the kth frame period which are frame periods adjacent to each other. However, the light source200may be set to be in the on-state selectively during the exposure period of the kth frame period preceding the (k+1)th frame period. In this case, contrary to the previous example, the first difference represents the intensity of the light reflected from the subject under ambient light such as sunlight or light emitted from the luminaire, and the second difference represents the sum of this intensity and the intensity related to the infrared light emitted from the light source200. Therefore, by subtracting the first difference from the second difference, it is possible to obtain image data based on the charge generated by the photoelectric conversion of a component of light which is part of the light emitted from the light source200and reflected from the subject. Also in this case, the effect of reset noise is canceled via the process of determining the difference between the first difference and the second difference.

As described above, in the embodiment of the present disclosure, in two adjacent frame periods including a prior frame period and a subsequent frame period, the pixel signal corresponding to the amount of signal charge accumulated in an exposure period of a prior frame period is read out nondestructively, and the pixel signal generated by the exposure during the exposure period of the subsequent frame period is cumulatively accumulated and the pixel signal is read out. Furthermore, the difference between these pixel signals is acquired in the form of a digital signal. As a result, it is possible to obtain image data corresponding to the sum of the amount of signal charge accumulated during the exposure period of the prior frame period and the amount of signal charge accumulated in addition to the above amount of signal charge. Note that the component corresponding to the reset noise is removed when the difference is calculated.

In the embodiment of the present disclosure, the difference between the pixel signal corresponding to the amount of signal charge accumulated in the exposure period of the prior frame period and the analog signal read from the pixels before the start of this exposure period is also acquired in the form of a digital signal. As a result, the component corresponding to the reset noise can be removed, and image data corresponding to the pure amount of signal charge accumulated in the exposure period of the prior frame period can be obtained. By calculating the difference between these image data, it is possible to obtain image data based on the amount of signal charge cumulatively accumulated in the exposure period of the subsequent frame period in a form in which the component corresponding to the reset noise is removed.

Furthermore, in the embodiment of the present disclosure, since the light source200is set to be in the on-state during part of the exposure period of one of the prior frame period or the subsequent frame period, it is possible to obtain an image substantially based on only the light emitted from the light source200by acquiring the difference between the first difference and the second difference. Since the pixels are not reset every frame period, it is possible to also achieve an improvement in the frame rate.

In the example described above with reference toFIG.4, the digital signal stored in the first frame memory172is given, by way of example, by a digital signal obtained via an analog-to-digital conversion from the reset signal. However, the digital signal stored in the first frame memory172is not limited to the digital signal corresponding to the reset signal. Instead of the digital signal corresponding to the reset signal, for example, the pixel signal corresponding to the amount of signal charge accumulated in the pixel during the exposure period included in the frame period immediately previous to the two adjacent frame periods may be read out, and the digital signal corresponding to this pixel signal may be stored in the first frame memory172. Also in this case, by calculating the difference between the first difference and the second difference, it is possible to obtain the image substantially based only on the infrared light from the light source200, as in the example described above with reference toFIG.4, in a form in which the component corresponding to the reset noise is removed. Also in this case, the pixel reset operation is not performed between the two adjacent frame periods. Another example of operation of the imaging device100

FIG.5is a diagram for explaining another example of an operation of the imaging device100. In the example shown inFIG.5, the light source200is in the on-state for a period from time t11to t20in the exposure period in the (k+1)th frame period. In the embodiment of the present disclosure, it is not essential to irradiate an entire subject at a time with light emitted from the light source200. As described below, for example, light emitted from the light source200may be scanned over the subject by using an optical system such as a MEMS mirror.

FIG.6schematically illustrate a manner in which an area of a subject is illuminated with light emitted from the light source200and the area illuminated with light moves over the subject during a period in which the light source200is in the on-state. In the example shown inFIG.6, the subject is scanned in the vertical direction by the light from the light source200. InFIG.6, among six vertically aligned rectangular blocks are shown for each 1H period, open blocks represent areas on the surface of the subject illuminated by the light from the light source200. In this example, as schematically shown inFIG.6, the area illuminated by the light from the light source200moves from top to bottom on the paper surface.

Reference symbols such as “R0” put in open blocks inFIG.6indicate which pixel in rows of pixels Px receives the reflected light from a region corresponding to the block. As can be seen fromFIG.6, the length of signal charge accumulation time during which the signal charge is accumulated in response to light from the light source200is equally four times the 1H period for any of the six rows.

FIGS.7and8each show an example in which light from the light source200is scanned in a faster manner.FIGS.7and8are diagrams respectively corresponding toFIGS.5and6. In the example shown inFIG.7, in contrast to the example shown inFIG.5, a 1V period, which is the pulse interval of the vertical synchronization signal VD, is shortened from 8 times 1H to 6 times 1H.

In this example, the period for which the light source200is in the on-state is shortened to a period from time t8to time t14, and the subject is scanned from its upper end to its lower end by the light emitted from the light source200in the period which is 6 times as long as 1H. As shown schematically inFIG.8, a plurality of pixels Px are scanned such that the row of a pixel that receives light that is emitted from the light source200and reflected by the subject is transitioned every 1H. That is, in this example, the period in which the signal charge is accumulated in response to the light emitted from the light source200has a length equal to 1H which is common to all six blocks aligned vertically. Thus, the frame rate can be further increased by shortening the 1V period and increasing the speed of scanning of light from the light source200in the above-described manner.

In the example shown inFIG.7, after the reading of the second pixel signal is completed, the reset operation in the next frame period is started at time t18in synchronization with a pulse of the vertical synchronization signal VD. In the operation of pixels Px in the 0th row R0to the 5th row R5, as indicated schematically by dot shaded rectangles inFIG.7, a period in which accumulating of signal charge is basically unnecessary exists between reading the second pixel signal in the (k+1)th frame period and the start of reading the reset signal in the next (k+2)th frame period.

If the voltage V1supplied from the voltage supply circuit190to the counter electrode13via the voltage line192is adjusted such that the bias voltage applied between the pixel electrode11and the counter electrode13is equal to V1−VRST=0, then the signal charge is not hardly collected by the pixel electrode11even when the charge is generated by the photoelectric conversion. In other words, it is possible to achieve the state similar to that which occurs when the mechanical shutter is closed. By setting the bias voltage applied between the pixel electrode11and the counter electrode13to be substantially equal to 0 V for the period in which it is not necessary to accumulate signal charges, it is possible to suppress a dark current to occur in the photoelectric conversion layer12.

By setting the bias voltage applied between the pixel electrode11and the counter electrode13to be substantially equal to 0 V for all rows of pixels Px, it is possible to provide a signal charge accumulation period in common to all pixels, that is, it is possible to realize a function of a so-called global shutter. See, for example, International Publication No. 2017/094229 for further detailed information on the global shutter realized by controlling the bias voltage between the pixel electrode11and the counter electrode13. The entire contents of International Publication No. 2017/094229 are incorporated herein by reference.

FIG.9schematically illustrates still another example of an operation of the imaging device100. In the example shown inFIG.9, the voltage V1supplied from the voltage supply circuit190to the counter electrode13is set to a high level selectively in a period from time t6to time t8in an exposure period included in the kth frame period. For the plurality of pixels Px in each row, the voltage V1is lowered to a low level in periods in the exposure period other than the period from time t6to time t8. InFIG.9, “LOW” indicates that the voltage output from the voltage supply circuit190is adjusted such that the bias voltage applied between the pixel electrode11and the counter electrode13is equal to V1−VRST=0.

In this period, even when light is incident on the photoelectric converter10, substantially no signal charge is accumulated. In other words, the effective period for accumulating the signal charge in the kth frame period is limited to the period from time t6to time t8. Note that the period in which the accumulation of signal charges actually occurs is equal for all rows from the 0th row R0to the 5th row R5. That is, the acquisition of the pixel signals by the global shutter is executed.

In the case of the (k+1)th frame period, for example, the voltage V1supplied from the voltage supply circuit190to the counter electrode13is set to the high level selectively during a period from time t4to time t6, which is a part of the exposure period. In this example, the light source200is in the on-state in a period from time t14to time t16equally for all rows. Thus, the global shutter is also applied to the acquisition of the second pixel signal. By acquiring the difference between the digital signal corresponding to the second pixel signal and the digital signal corresponding to the first pixel signal, it is possible to obtain an image based on infrared light with no distortion even when the subject is moving at high speed.

In each example described above, the pixels Px are reset before the accumulation of the signal charge in the exposure period included in the prior frame period of the two successive frame periods. In other words, in each of the above described examples, the resetting of the pixels Px is performed in units of two frame periods. However, the cycle of the resetting of pixels Px is not limited to twice the frame period. The signal charge may be accumulated cumulatively over a plurality of frame periods as long as the amount of accumulated charge does not exceed the capacity of the charge accumulation region.

FIG.10schematically illustrates still another example of an operation the imaging device according to an embodiment of the present disclosure.FIG.10also includes a graph schematically illustrating a temporal change in the amount of signal charge accumulated in the charge accumulation region for one pixel belonging to a 0th row R0of a plurality of pixels Px included in the imaging region. In the graph shown at the bottom ofFIG.10, the horizontal axis represents time T. The vertical axis of the graph represents the amount of signal charge C accumulated in the charge accumulation region of a pixel of interest, and the reading of the vertical axis of the graph corresponds to the potential value of the node FD. A broken line in the graph shown at the bottom ofFIG.10represents a saturation amount of charge Ct, which is the maximum possible value of the amount of charge accumulated in the charge accumulation region of the pixel.

In the example shown inFIG.10, before the start of the exposure period included in the kth frame period, pixels in each row are reset and reset signals are read. After that, as in the example described above with reference toFIG.4, reading of the first pixel signal in the kth frame period (from time t8to time t14) and the second pixel signal in the following (k+1)th frame period (from time t16to time t22) are performed.

Here, the graph shown at the bottom ofFIG.10indicates a change in the amount of charge accumulated in the charge accumulation region which is the greatest among the plurality of pixels Px included in the imaging region at a point of time at which the reading of the second pixel signal is completed. In this example, at the end of the (k+1)th frame period (at time t22), the total amount of signal charge accumulated in the charge accumulation region of the pixel is less than the saturation amount of charge Ct described above. Therefore, at the end of the (k+1)th frame period, each of the plurality of pixels Px included in the imaging region is in a state where a further signal charge can be cumulatively accumulated in the charge accumulation region.

In a case where the charge accumulation region of each pixel is capable of accepting an additional signal charge at the end of two consecutive frame periods, the exposure operation in the next frame period may be started without performing resetting of pixels. In this example, following the reading of the second pixel signal, the (k+2)th frame period is started and the signal charge is accumulated. Here, the total amount of signal charge accumulated in the charge accumulation region of each pixel is less than the saturation amount of charge Ct at the end of the (k+2)th frame period. Therefore, a third pixel signal read from each pixel after the completion of the exposure period of the (k+2)th frame period can also be effectively used. Note that the “third pixel signal” inFIG.10indicates an analog signal corresponding to the amount of charge cumulatively accumulated in each pixel during the exposure periods included in the kth, (k+1)th, and (k+2)th frame periods.

The operation such as that illustrated inFIG.10is effective particularly when the infrared light is emitted from the light source such that the infrared light strikes the same target subject a plurality of times. For example, as shown inFIG.10, the light source200may be in the on-state for a part of the exposure period in the (k+2)th frame period. When the operation is performed in the above-described manner, the infrared light reflected from the subject can be received a plurality of times and the signal charge can be cumulatively accumulated, it is possible to achieve a higher signal level which allows an improvement in the SN ratio.

The determination as to whether or not to start the exposure for the next frame period without performing the pixel reset operation following two consecutive frame periods may be made depending on whether the total amount of signal charge accumulated in the charge accumulation region exceeds the saturation amount of charge Ct.FIG.11schematically illustrates an example in which the pixel reset operation is executed when it is determined that the total amount of signal charge accumulated in the charge accumulation region exceeds the saturation amount of charge Ct. The graph at the bottom ofFIG.11includes, in addition to a broken line indicating the saturation amount of charge Ct, a broken line indicating a threshold value that provides a criterion for determining whether or not to execute the pixel reset operation. The threshold value Th is appropriately set, in advance, to value smaller than the saturation amount of charge Ct.

In the graph shown at the bottom ofFIG.11, a solid line represents an example of a change in amount of signal charge in one of pixels belonging to one of rows (for example, 0th row R0) in the imaging region, while a dotted line represents an example of a change in amount of signal charge in one of pixels belonging to another one of rows (for example, 2nd row R2). In this example, the amount of signal charge accumulated in the charge accumulation region of a pixel in the 2nd row R2is smaller than the above-described threshold value Th even at the end of the (k+1)th frame period (at time t22). On the other hand, the amount of signal charge accumulated in the charge accumulation region of the pixel in the 0th row R0exceeds the threshold value Th before the end of the (k+1)th frame period, due to the exposure is performed following the reading of the second pixel signal.

Therefore, in this example, after the end of the (k+1)th frame period, the pixel reset operation is executed again before the start of the next frame period (from time t22to time t28). As a result, the potential of the charge accumulation region of each pixel is reset, which makes it possible to prevent the acquisition of an image deteriorated due to an overflow of the signal charge. In this way, the amount of signal charge accumulated in the charge accumulation region of the pixel is monitored, the determination is performed as to whether or not the total amount of signal charge exceeds a predetermined threshold value Th, and the pixel reset operation is performed depending on the result of the determination. The amount of signal charge accumulated in the charge accumulation region can be known, for example, by monitoring the analog voltage read to the output signal line Sjvia the readout circuit20or the digital signal output from the A/D conversion circuit140.

FIG.12schematically illustrates an example of a configuration of an imaging device according to another embodiment of the present disclosure. In addition to the components of the imaging device100described above with reference toFIG.1, the imaging device100A shown inFIG.12further includes a second light source220and a light source control apparatus230connected to the light source220. The light source control apparatus230is connected to the image processing circuit170, and the light source control apparatus230controls the operation of the light source220according to a control signal supplied from the image processing circuit170. The light source220is in the on-state, for example, for exposure periods in the kth and (k+1)th frame periods, different from the exposure periods for which the light source200is in the on-state.

The light source220emits light having a peak at a second wavelength different from the first wavelength described above. The light source220may be an infrared light source that emits infrared light like the light source200. However, as for the peak wavelength of light emitted from the light source220, a second wavelength different from the above-described first wavelength is selected. Example of method of driving imaging device100A

FIG.13is a diagram for explaining an example of a method of driving the imaging device100A shown inFIG.12. Here, a light source that emits infrared light whose wavelength peak is not located at any wavelength near 1450 nm or 1940 nm is used as the first light source200, while a light source that emits infrared light whose wavelength peak is located near 1450 nm is used as the second light source220. The example of the operation shown inFIG.12is useful when the illuminance of the ambient light in the visible wavelength region is low, such as in the outdoors at night, or when the photoelectric conversion layer12of the photoelectric converter10is not sensitive to light in the visible wavelength range.

In the example shown inFIG.13, the reset signal is read out during a period from time t0to time t6, and, after that, an exposure operation in the kth frame and an exposure operation in the (k+1)th frame are executed. Furthermore, the light source220is set to be in the on-state selectively during a period from time t6to time t8in the kth frame period, and the light source200is set to be in the on-state selectively during a period from time t14to time t16in the (k+1)th frame period.

In a case where the light source220is set to be in the on-state selectively during part of the exposure period in an environment where there is almost no sunlight or artificial lighting in the visible wavelength range, the light emitted from the light source220and reflected by the subject occupies most of the light reaching the photoelectric converter10of each pixel. That is, the amount of signal charge accumulated in the charge accumulation region of each pixel reflects the intensity of the light emitted from the light source220and reflected by the subject. The same applies when the light source200is set to be in the on-state selectively during part of the exposure period.

For example, at night, by turning on the light sources200and220during at least part of the exposure period as shown inFIG.13, it is possible to acquire an image based on substantially only infrared light. In the example shown inFIG.13, the first pixel signal acquired during the period from time t8to time t14substantially represents an image based only on the infrared light emitted from the light source220. The first pixel signal is converted into a digital signal by the A/D conversion circuit140, and is temporarily stored in the second frame memory174in the form of a digital signal corresponding to the first pixel signal.

The image processing circuit170calculates the first difference which is a difference between a digital signal corresponding to the second pixel signal in the (k+1)th frame period and a digital signal corresponding to the first pixel signal in the kth frame period temporarily stored in the second frame memory174. By performing this process of acquiring the first difference, as in the previous examples, one frame of image data is obtained in a form in which the influence of the reset noise is substantially canceled. The first difference in the example shown inFIG.13corresponds to the increment of the second pixel signal acquired during the period from time t16to time t22from the signal level of first pixel signal. Thus, the first difference substantially represents an image based on only the infrared light emitted from the light source200.

Also in this state, a digital signal corresponding to the reset signal read out during a period from time t0and time t6inFIG.13is held in the first frame memory172. The image processing circuit170further calculates the second difference which is different between the digital signal temporarily stored in the second frame memory174and a digital signal corresponding to the reset signal temporarily stored in the first frame memory172. The second difference in the example shown inFIG.14substantially represents an image based on only the infrared light emitted from the light source220.

The image processing circuit170calculates the difference between the first difference and the second difference. Here, let it be assumed that there is no significant difference in output between the light source200and the light source220. In this situation, there is a possibility that the second difference is close to 0 even though a relatively large value is obtained as the difference between the second difference and the first difference. One of two possible reasons for an occurrence of such a situation is that a subject irradiated with the infrared light emitted from the light source220is located far away from the light source220. The other one of possible reasons is that most of the infrared light emitted from the light source220is absorbed by moisture in the atmosphere.

Here, let it be assumed that a wavelength that is easily absorbed by water is selected as the second wavelength, and a wavelength that is not easily absorbed by water is selected as the first wavelength. If the second difference is close to 0, and the difference between the second difference and the first difference is relatively large, this means that the target irradiated with the infrared light is located not far from the light sources200and220, and that the light with the first wavelength is mostly absorbed by the moisture in the atmosphere. That is, it is possible to determine whether the reduction in the signal level based on the light with the first wavelength is caused by the too large distance to the subject or caused by the large absorption by water. In a case where a distance is measured using light with the first wavelength, when the second difference between the digital signal corresponding to the first pixel signal and the digital signal corresponding to the reset signal has a value close to 0 for a pixel of a plurality of pixels, it is determined that an error has occurred in a pixel value of that pixel, and a correction may be made by extrapolation using the second difference value related to neighboring pixels. Instead of selecting different wavelengths between the light source220and the light source220, an equal wavelength may be selected for both the light source220and the light source220, and a difference may be introduced in other parameters such as the light emission intensity, the irradiation region, and/or the like between the light source220and the light source220.

In a case where it is difficult to remove ambient light as in the case where shooting is performed in the daytime, as in the example described above with reference toFIG.10, the first pixel signal may be acquired in a state in which the light sources200and220are set in the off-state during the exposure period of the kth frame period, and the second pixel signal may be acquired in a state in which one of the light source200or the light source200is set in the on-state during the exposure period of the (k+1)th frame period. In this case, by acquiring the difference between the second difference and the first difference, it is possible to obtain an image substantially similar to an image obtained when shooting is performed only under the light from the light source set to be in the on-state during the exposure period of the (k+1)th frame period.

Note that the pixel signal may be obtained such that the pixel reset operation is not executed between the (k+1)th frame period and the subsequent (k+2)th frame period, and the other one of the light source200and the light source220is set to be in the on-state selectively during the exposure period of the (k+2)th frame period. Let this pixel signal be referred to as the third pixel signal. By acquiring the difference between the digital signal corresponding to the third pixel signal and the digital signal corresponding to the second pixel signal, it is possible to obtain an image substantially similar to an image obtained when shooting is performed only under the light from the light source set to be in the on-state during the exposure period of the (k+2)th frame period.

As the second light source220, a general strobe light source having a peak in the visible wavelength range may be used. In this case, first, the second light source220may be set to be in the on-state during a part of the exposure period in a prior frame period, and the light source200that emits infrared light may be set in the on-state during a part of the exposure period in a subsequent frame period following the prior frame period. Alternatively, the light source200may be set to be in the on-state in the prior frame period, and the light source220may be set to be in the on-state in the subsequent frame period. However, in the case where the light source200that emits infrared light is set to be in the on-state during a part of the exposure period in the prior frame period, a greater amount of signal charge is accumulated in the charge accumulation region of the pixel, and, as a result, the amount of charge that can be further cumulatively accumulated decreases, which may cause the saturation to easily occur. Note that the larger the amount of charge accumulated in the charge accumulation region of the pixel, the greater the increase in the potential of the impurity region111(seeFIG.3) is, and thus more likely a dark current is to occur in the impurity region111. Therefore, as illustrated inFIG.13, it is generally advantageous to set the light source200, which emits infrared light, to be in the on-state during part of the exposure period in the subsequent frame period.

FIG.14schematically illustrates an example of a configuration of an imaging device according to still another embodiment of the present disclosure. The imaging device100B shown inFIG.14includes a set of two output signal lines for each column of pixels Px. In the example shown inFIG.14, for example, in a jth column of pixels Px, an output signal line Sjand an output signal line Tjare provided. Although some components are not shown inFIG.14for the sake of simplicity, the imaging device100B also includes a light source200as in the previous examples.

In the present embodiment, one of the two output signal lines provided for each column of pixels Px is connected to, for example, pixels Px in even-numbered rows, and the other one of the two output signal lines is connected to pixels Px in odd-numbered rows. In the example of the configuration illustrated inFIG.14, the output signal line Sjis connected to the readout circuit20of the first pixel Px1located in the ith row. On the other hand, the output signal line Tjis connected to the readout circuit20of the fourth pixel Px4located in the (i+1)th row.

Furthermore, in this example, the imaging device100B includes a first A/D conversion circuit141and a second A/D conversion circuit wherein the first A/D conversion circuit141is electrically connected to pixels in even-numbered rows via one of two output signal lines provided for each column of pixels Px, and the second A/D conversion circuit141is electrically connected to pixels in odd-numbered rows via the other one of the two output signal lines. As shown inFIG.14, the A/D conversion circuit141is connected to a first interface161, and the A/D conversion circuit142is connected to a second interface162. The output of the first interface161and the output of the second interface162are supplied to an image processing circuit170having a first frame memory172and a second frame memory174(seeFIG.2).

Example of Method of Driving Imaging Device100B

By providing output signal lines independently for pixels Px in even-numbered rows and pixels in odd-numbered rows, it becomes possible to read signals independently and in parallel from the pixels Px in the even-numbered rows and the pixels Px in the odd-numbered rows.FIG.15is a diagram for explaining an example of a method of driving the imaging device100B shown inFIG.14.

In this example, the reset signal is read out before the exposure period of the kth frame period. The reading of the reset signal is executed via one of the two output signal lines provided for each column of pixels Px. For example, in a period from time t4to time t5, reading of the first pixel signal from the pixels in the 0th row R0is performed in parallel with reading the reset signal of the pixels in the 3rd row R3vie the output signal line Tj. The reading of the first pixel signal is performed via the output signal lines Sj.

By employing the configuration in which two output signal lines are provided for each column of pixels Px, it becomes possible to read different signals independently from pixels Px belonging to different rows at a time. In other words, it becomes possible to allow two reading periods for reading different signals to overlap each other. More specifically, in the present example, in the kth frame, the reading period for reading the reset signal is overlapped with the reading period for reading the first pixel signal. Furthermore, in the present example, the reading period for reading the first pixel signal in the kth frame is overlapped with the reading period for reading the second pixel signal in the (k+1)th frame. As described above, by employing the configuration in which two output signal lines are provided for each column of pixels Px, it also becomes possible to overlap signal reading periods between a plurality of frame periods, which allows an increase in the frame rate.

Furthermore, when the frame rate is increased, the exposure time may be reduced thereby reducing the influence of the dark current in the photoelectric conversion layer12and/or the influence of the dark current in the impurity region111. This effect is achieved because the dark current increases in proportion to the exposure time. In particular, in each row of pixels Px, the period in which the light source200and/or the light source220are in the on-state may be provided as close as possible to the exposure period thereby making it possible to achieve a high SN ratio while suppressing the dark current.

In this example, the light emitted from the light source200is scanned over a subject in a period from time t6to time t12in a similar manner as in the example described above with reference toFIG.7. The pixel Px that receives light emitted from the light source200and reflected by the subject is transitioned from one row to another row may be transitioned in a similar to the manner in the example shown inFIG.8.

As described above, according to the embodiment of the present disclosure, the digital signal corresponding to the first analog signal is read out via the readout circuit20of each pixel, and stored in the first frame memory172. The digital signal corresponding to the second analog signal corresponding to the amount of charge accumulated in the pixel during the first exposure period following the reading period for the first analog signal is stored in the second frame memory174. By calculating the difference between the signals store in the first frame memory172and the second frame memory174, the first difference is obtained which is a signal in which the reset noise is substantially removed. Furthermore, for example, in the second exposure period occurring subsequent to the first exposure period, the light source200is set to be in the on-state, and the signal charge is cumulatively accumulated such that the signal charge is added to the signal charge accumulated in the first exposure period. Furthermore, the difference of the digital signal corresponding to the third analog signal corresponding to the amount of charge cumulatively accumulated in the pixel from the digital signal stored in the second frame memory174is acquired, and the difference between this difference and the above-described first difference is calculated. As a result, it is possible to obtain image data based on substantially only the light from the light source200in a form in which the reset noise is removed.

In the examples of configurations shown inFIGS.1and12, the row scanning circuit130, the control circuit250, the A/D conversion circuit140, the digital memory150, and the interface160are disposed on the semiconductor substrate110on which the plurality of pixels Px are formed. That is, the semiconductor substrate110on which the plurality of pixels Px are formed, the row scanning circuit130, the control circuit250, the A/D conversion circuit140, the digital memory150, and the interface160may be in the form of a package in which those are integrated. Part or all of these circuits may be integrally formed on the semiconductor substrate110in addition to the readout circuits20of the respective pixels Px. That is, these circuits may be formed on the semiconductor substrate110by applying the same process as the process of forming the readout circuits20of the respective pixels Px. For example, the control circuit250may be implemented by an integrated circuit formed on the semiconductor substrate110. However, it is not essential that all of these circuits are integrally formed on the semiconductor substrate110together with the pixels Px. Part or all of these circuits may be disposed on a substrate different from the semiconductor substrate110on which the pixels Px are formed.

The functions of the control circuit250and the functions of the image processing circuit170described above may be realized by a combination of a general-purpose processing circuit and software, or may be realized by hardware specialized for such processing. The control circuit250may receive a setting related to the exposure time from the image processing circuit170according to the result of the processing performed by the image processing circuit170, and may supply drive signals, according to the setting related to the exposure time, to the row scanning circuit130, the A/D conversion circuit140, and the like.

The image processing circuit170may be provided in the imaging device in the form of a chip or a package separate from the circuit group formed on the semiconductor substrate110. The first frame memory172and/or the second frame memory174may be disposed in the imaging device in the form of a chip or package separate from the image processing circuit170. Alternatively, the image processing circuits170and/or the light source control apparatuses210and230may be disposed on the semiconductor substrate110. The image processing circuit170may be part of the control circuit250. The image processing circuit170or the control circuit250may be configured to execute processing such as a distance measurement calculation, a wavelength information separation process, etc.

The imaging device according to the embodiment of the present disclosure may be provided in the form of a package in which the semiconductor substrate110on which the plurality of pixels Px are formed and the image processing circuit170are integrated. The imaging device according to the embodiment of the present disclosure may be in the form of an image sensor chip or in the form of a camera.

The embodiments of the present disclosure may be applied to various cameras and camera systems such as a medical camera, a security camera, a camera mounted on a vehicle, a distance measuring camera, a microscope camera, a camera installed on an unmanned aerial vehicle called a drone, and a robot camera, etc. The in-vehicle camera may be used, for example, to input information to a control apparatus for a vehicle to run safely. The in-vehicle camera may be used to assist an operator to drive a vehicle safely.