Patent Description:
In recent years, various kinds of semiconductor apparatuses have been used. As the operation period of a semiconductor apparatus increases, a loss of the function of the semiconductor apparatus is more likely to occur.

For example, as for a technique disclosed in the document <CIT>, there is a description of a semiconductor apparatus that includes a lifetime estimating unit that obtains the degradation degree of a functional unit of the semiconductor apparatus and that estimates the lifetime of the semiconductor apparatus from the degradation degree. According to the document <CIT>, when a threshold voltage increases due to the degradation of a semiconductor device, a power supply voltage Vdd is increased. This enables the lifetime of the functional unit to be increased.

In the technique disclosed in the document <CIT>, drive for increasing the power supply voltage is carried out to increase the lifetime of the semiconductor apparatus. In some cases, however, an increase in the power supply voltage results in a decrease in the lifetime of the semiconductor apparatus, and the drive is not optimal for increasing the lifetime of the semiconductor apparatus. In addition, estimation of the lifetime of an image capturing apparatus is not considered at all.

Further background art is known as follows:
The document <CIT> discloses an apparatus and a method for a user configurable reliability control loop. Therein, it is taught that a processor comprises: a reliability meter to track accumulated stress on components of the processor based on measured processor operating conditions; and a controller to receive stress rate limit information from a user or manufacturer and to responsively specify a set of N operating limits on the processor in accordance with the accumulated stress and the stress rate limit information; and performance selection logic to output one or more actual operating conditions for the processor based on the N operating limits specified by the controller.

The document <CIT> discloses a semiconductor device, a monitoring system, and a lifetime prediction system/method enabling it to notify a user that a semiconductor device is likely to become faulty, before the semiconductor device becomes faulty. Therein, it is taught that a semiconductor device includes functional units and a lifetime prediction circuit, wherein the lifetime prediction circuit acquires a deterioration degree indicating a degree of how each functional unit deteriorates, using a signal that is output from each functional unit, and the lifetime prediction circuit executes processing to make a notification that the semiconductor device is close to its lifetime, if the deterioration degree is more than a first threshold.

Further documents of relevant background art are e.g. <CIT>, <CIT> and <CIT>.

According to the present invention, drive for increasing the remaining lifetime of an image capturing apparatus is considered.

The present invention has been accomplished in view of the above problem. Accordingly, the present invention provides an image capturing apparatus, an image capturing system, and a moving object, as defined by the claims.

According to the present invention, the remaining lifetime of an image capturing apparatus can be increased.

Also, according to the present invention, an image capturing apparatus that can estimate remaining lifetime after an image capturing operation is actually performed is provided.

Embodiments of the present disclosure will hereinafter be described with reference to the drawings. The first embodiment is useful for understanding the invention, but falls outside the scope of the claims.

In the following description, a transistor is an N-type transistor, provided that there is no notification. However, the embodiments described below are not limited to an N-type transistor, a P-type transistor may be appropriately used. In this case, the voltage of a gate, a source, and a drain of the transistor can be appropriately changed from those in the description according to the embodiments. For example, in the case of a transistor that operates as a switch, the low level and the high level of a voltage that is applied to the gate are inverted in comparison with the description according to the embodiments.

<FIG> illustrates an example of the structure of a semiconductor apparatus according to the present embodiment. A semiconductor apparatus <NUM> includes a semiconductor device <NUM> and a lifetime estimating unit <NUM>. The semiconductor device <NUM> is a circuit that has many functions such as light reception, light emission, AD conversion processing, signal processing, communication processing, a storage unit, and a detection unit. The structure and the operation of the semiconductor device <NUM> will now be described.

A first circuit block <NUM>, a second circuit block <NUM>, and a N-th circuit block <NUM> are circuits that have many functions such as light reception, light emission, AD conversion processing, signal processing, communication processing, a storage unit, and a detection unit.

A voltage control unit <NUM> supplies, to the first circuit block <NUM>, the second circuit block <NUM>, and the N-th circuit block <NUM>, a voltage value that is set by using a signal from a driving mode selecting unit <NUM> that corresponds to a driving control unit.

When the voltage value that is supplied by the voltage control unit <NUM> is zero, all or a part of the first circuit block <NUM>, the second circuit block <NUM>, and the N-th circuit block <NUM> are turned off.

By using the signal from the driving mode selecting unit <NUM>, a frequency control unit <NUM> changes a period in which a signal generating unit <NUM> generates control signals that are to be transmitted to the first circuit block <NUM>, the second circuit block <NUM>, and the N-th circuit block <NUM>.

The driving mode selecting unit <NUM> receives remaining lifetime information from the lifetime estimating unit <NUM> that corresponds to a lifetime obtaining unit described later via a lifetime information receiving unit <NUM>. The voltage control unit <NUM> and the frequency control unit <NUM> are controlled depending on the information. The driving mode selecting unit <NUM> can generate a control signal that enables all or a part of the first circuit block <NUM>, the second circuit block <NUM>, and the N-th circuit block <NUM> to be stopped.

The lifetime estimating unit <NUM> estimates the lifetime of the semiconductor device <NUM>, based on information that represents the latest state of the operation of the semiconductor device <NUM> or history information in relation to the state of the operation and obtains the remaining lifetime information on the semiconductor device <NUM>. That is, the lifetime estimating unit <NUM> corresponds to the lifetime obtaining unit that obtains the lifetime information. The state of the operation means a state in which the semiconductor device <NUM> operates such as an operation voltage or an operating frequency at which the semiconductor device <NUM> operates or a state caused as a result of the operation such as a temperature. The information that represents the latest state of the operation of the semiconductor device <NUM> and the history information in relation to the state of the operation are operation information that represents the state of the operation of the semiconductor device <NUM>.

An example of a method of estimating the lifetime of the semiconductor device <NUM> that is performed by the lifetime estimating unit <NUM> will now be described. The lifetime of the semiconductor apparatus is typically limited because the performance of a transistor, a wiring line, and an oxide film that are included in the semiconductor apparatus impairs over time. Main mechanisms of the impairment in the performance of the semiconductor apparatus include electromigration (EM) and time dependent dielectric breakdown (TDDB). Other mechanisms of the impairment in the performance include variations in characteristics due to hot carrier injection (HC) and negative bias temperature instability (NBTI).

A remaining lifetime LT (EM) when a factor in the impairment in the performance is EM is calculated by using the following expression (<NUM>): <MAT>.

Here, A is a coefficient that depends on a manufacturing process, Ea is an activation energy, J is the current density of the wiring line, n is a current acceleration factor, k is the Boltzmann constant, and T is the temperature of the wiring line.

A remaining lifetime LT (TDDB) when the factor in the impairment in the performance is TDDB is calculated by using the following expression (<NUM>): <MAT>.

Here, Vg is a voltage that is applied to the gate of the transistor, and yVg is a voltage acceleration factor.

A remaining lifetime LT (HC) when the factor in the impairment in the performance is HC is calculated by using the following expression (<NUM>): <MAT>.

Here, Isub is the maximum substrate current that flows through a semiconductor substrate, and m is a coefficient that depends on the substrate current.

A remaining lifetime LT (NBTI) when the factor in the impairment in the performance of the semiconductor device <NUM> is NBTI is calculated by using the following expression (<NUM>): <MAT>.

The lifetime estimating unit <NUM> measures all or a part of coefficients in expression (<NUM>) to expression (<NUM>) and calculates the remaining lifetime of the semiconductor device <NUM>. The lifetime estimating unit <NUM> may be located inside the semiconductor device <NUM>.

Examples of operations according to the present invention will be described below with reference to a flowchart in <FIG> and timing charts in <FIG>. An example of an operation corresponding to a timing chart in <FIG> will now be described. <FIG> illustrates a relationship between the operation time of the semiconductor device <NUM> and the remaining lifetime of the semiconductor device <NUM> by a solid line.

At S200, the driving mode selecting unit <NUM> obtains a first lifetime threshold L1 and a second lifetime threshold L2 of the semiconductor device <NUM>. The first lifetime threshold L1 and the second lifetime threshold L2 are determined as thresholds for the remaining lifetime of the semiconductor device <NUM>, and L1 > L2 > <NUM> is satisfied.

At S201, the lifetime estimating unit <NUM> calculates the remaining lifetime Ti of the semiconductor device <NUM> at time i.

At S202, the driving mode selecting unit <NUM> compares the first lifetime threshold L1 and the remaining lifetime Ti at time i. If Ti > L1 is satisfied, processing proceeds to S203. If Ti ≤ L1 is satisfied, the processing proceeds to S204.

<FIG> illustrates the case where Ti > L1 is satisfied.

At S203, a process of continuing the operation of the semiconductor device <NUM> for a freely determined period Δt1 is performed, and subsequently, the processing returns to S201.

In the above flow, the remaining lifetime of the semiconductor device <NUM> is longer than the first lifetime threshold L1. The semiconductor device <NUM> has an enough remaining lifetime, and the driving state of the semiconductor device <NUM> is not changed.

The case in <FIG> when operation time elapses after that in <FIG> will now be described.

<FIG> illustrates the relationship between the operation time of the semiconductor device <NUM> and the remaining lifetime of the semiconductor device <NUM> by a solid line. A dashed line represents an estimated remaining lifetime of the semiconductor device <NUM> that is estimated by the lifetime estimating unit <NUM> at time i with respect to the operation time of the semiconductor device <NUM>.

<FIG> illustrates the case where Ti ≤ L1 is satisfied.

At S204, the lifetime estimating unit <NUM> estimates the remaining lifetime Lt when predetermined operation time t of the semiconductor device <NUM> elapses. For example, the predetermined operation time is time until the semiconductor device <NUM> spends a lifetime that is estimated in advance. After the operation time exceeds the time, the operation of the semiconductor device <NUM> is not guaranteed.

At S205, whether the remaining lifetime Lt when the predetermined operation time t of the semiconductor device <NUM> elapses is more than <NUM> is determined. If Lt > <NUM> is satisfied, the processing proceeds to S203. If Lt ≤ <NUM> is satisfied, the processing proceeds to S206. <FIG> illustrates the case where Lt > <NUM> is satisfied.

The case where Lt > <NUM> is satisfied means the case where the semiconductor device <NUM> does not spend the lifetime even after a normal driving mode continues until the predetermined operation time t. Accordingly, the driving mode selecting unit <NUM> does not change the driving mode of the semiconductor device <NUM>.

The processing proceeds to S203, and a process of continuing the operation of the semiconductor device <NUM> in the normal driving mode for a freely determined period Δt1 is performed. Subsequently, the processing returns to S201.

In the case in <FIG>, the normal driving mode continues until the predetermined operation time t of the driving mode of the semiconductor device <NUM> as described above.

The case in <FIG> will now be described.

The case in <FIG> corresponds to an example in which the semiconductor device <NUM> spends the lifetime before the predetermined operation time t when the operation in the normal driving mode continues after time i. That is, the result of determination whether Lt > <NUM> is satisfied is No in a process at S205.

<FIG> illustrates the relationship between the operation time of the semiconductor device <NUM> and the remaining lifetime of the semiconductor device <NUM> by a solid line. A dashed line represents an estimated remaining lifetime of the semiconductor device <NUM> that is estimated by the lifetime estimating unit <NUM> at time i with respect to the operation time of the semiconductor device <NUM>. A one-dot chain line represents a relationship between the operation time of the semiconductor device <NUM> and the remaining lifetime of the semiconductor device <NUM> in a changed driving state after the driving state of the semiconductor device <NUM> is changed at time i.

At S205 in <FIG>, it is determined to be No in the case in <FIG>. Accordingly, the processing proceeds to S206.

At S206 in <FIG>, the driving mode selecting unit <NUM> controls the operation of both of the voltage control unit <NUM> and the frequency control unit <NUM> or either of them and selects a life-prolonging driving mode in which the remaining lifetime of the semiconductor device <NUM> is longer than the remaining lifetime at time i. The life-prolonging driving mode corresponds to a condition in which the throughput of signal processing is less than that in the case where the semiconductor device <NUM> is driven in the normal driving mode. In the driving mode, the remaining lifetime of the semiconductor device <NUM> is longer than that in the case where the semiconductor device <NUM> is driven in the normal driving mode. That is, the normal driving mode corresponds to a first condition in which the driving control unit drives the semiconductor device <NUM>. The life-prolonging driving mode corresponds to a second condition in which the throughput of the signal processing is less than that in the case where the semiconductor device <NUM> is driven in the first condition and the remaining lifetime of the semiconductor device is longer than that in the case where the semiconductor device <NUM> is driven in the first condition.

An example of a method of selecting the life-prolonging driving mode of the semiconductor device <NUM> by the driving mode selecting unit <NUM> will now be described. As defined as expression (<NUM>) to expression (<NUM>) according to the first embodiment, the lifetime of the semiconductor apparatus is limited by the current density J of the wiring line, the temperature T of the wiring line, the voltage Vg that is applied to the gate of the transistor, and the maximum substrate current Isub that flows through the semiconductor substrate. The driving mode selecting unit <NUM> controls both of the voltage control unit <NUM> and the frequency control unit <NUM> or either of them such that the remaining lifetime of the semiconductor device <NUM> increases. Specifically, the state of the operation is changed such that the values of all or a part of the current density J, the temperature T of the wiring line, the voltage Vg that is applied to the gate of the transistor, and the maximum substrate current Isub that flows through the semiconductor substrate are lower than the normal driving mode. The change in the state of the operation can be said to be a change for reducing a drive ability through which the semiconductor device <NUM> is driven. For example, the absolute values of the power supply voltages that are supplied to the first circuit block <NUM>, the second circuit block <NUM>, and the N-th circuit block <NUM> by the voltage control unit <NUM> are decreased (typically, the power supply voltages are decreased). This enables the current density J, the temperature T of the wiring line, and the voltage Vg to be decreased. The temperature T of the wiring line and the substrate current Isub can be decreased by increasing a period in which the signal generating unit <NUM> generates a signal by using the frequency control unit <NUM>. At this time, the driving mode selecting unit <NUM> controls the voltage control unit <NUM> and the frequency control unit <NUM> such that the operation of the semiconductor device <NUM> can continue.

In the normal driving mode, the semiconductor device <NUM> is driven in the first condition as described above. In the life-prolonging driving mode, the semiconductor device <NUM> is driven in the second condition in which the lifetime of the semiconductor device <NUM> is longer than that in the case where the semiconductor device <NUM> continue to be driven in the normal driving mode as described above.

In some cases where the mode is changed into the life-prolonging driving mode, the amount of the signal processing of the semiconductor device <NUM> is smaller than that in the normal driving mode, for example, the functions are partly stopped, and the operation speed of the semiconductor device <NUM> is decreased. In the life-prolonging driving mode, however, the operation of the semiconductor device <NUM> can continue, and the remaining lifetime can be increased to a lifetime longer than that in the case where the normal driving mode continues in a manner in which a load that is applied to the circuit of the semiconductor device <NUM> is decreased to a level lower than that in the normal driving mode.

In some cases, the throughput of the signal processing of the semiconductor device <NUM> in the life-prolonging driving mode is less than that in the normal driving mode. For example, it can be said that the throughput of the signal processing corresponds to the quantity of signals on which the semiconductor device <NUM> performs the signal processing per unit time. In this example, one or a plurality of the amount of electric current supply from an electric current source, the drive voltage (power supply voltage), and the operating frequency of the semiconductor device <NUM> in the life-prolonging driving mode is less than in the normal driving mode. In another example, in the normal driving mode, the signal processing is performed for a first period. The life-prolonging driving mode comprises an intermittent drive in which the signal processing is performed for a second period shorter than the first period and signal processing is not performed for a period that is included in the first period and that is not included in the second period, or a drive for reducing the throughput of the signal processing to a level lower than that in the second period. In an example in <FIG>, the remaining lifetime of the semiconductor device <NUM> that operates in the life-prolonging driving mode is longer than the second lifetime threshold L2, and subsequent operation in the normal driving mode is possible.

At S207, the lifetime estimating unit <NUM> calculates the remaining lifetime Lj of the semiconductor device <NUM> at time j.

At S208, the driving mode selecting unit <NUM> compares the second lifetime threshold L2 and the remaining lifetime Lj at time j. If Lj > L2 is satisfied, the processing proceeds to S209. If Lj ≤ L2 is satisfied, the processing proceeds to S210. <FIG> illustrates the case where Lj > L2 is satisfied.

In the state of Lj > L2, the remaining lifetime of the semiconductor device <NUM> that operates in the life-prolonging driving mode is shorter than the first lifetime threshold L1 and is longer than the second lifetime threshold L2. Accordingly, the semiconductor device <NUM> has an enough remaining lifetime in the life-prolonging driving mode relative to a predetermined operation continuation period. Accordingly, even when the operation continues for a period not in the life-prolonging driving mode but in the normal driving mode, the semiconductor device <NUM> can continue the operation until the predetermined operation continuation period. Accordingly, the driving state of the semiconductor device <NUM> is restored to the normal driving mode before the mode is changed into the life-prolonging driving mode. The processing proceeds to S203, and the operation of the semiconductor device <NUM> continues in the normal driving mode for a freely determined period Δt1. Subsequently, the processing returns to S201.

An example in <FIG> will now be described.

In <FIG>, a dashed line represents an estimated remaining lifetime of the semiconductor device <NUM> that is estimated by the lifetime estimating unit <NUM> at time i with respect to the operation time of the semiconductor device <NUM>. A one-dot chain line represents a relationship between the operation time of the semiconductor device <NUM> and the remaining lifetime of the semiconductor device <NUM> in a changed driving state after the driving state of the semiconductor device <NUM> is changed at time i.

At S208, the driving mode selecting unit <NUM> compares the second lifetime threshold L2 and the remaining lifetime Lj at time j. <FIG> illustrates the case where Li ≤ L2 is satisfied.

At S210, the lifetime estimating unit <NUM> calculates the remaining lifetime Lk of the semiconductor device <NUM> at time k.

At S211, the driving mode selecting unit <NUM> determines whether the remaining lifetime Lk at time k is more than <NUM>. If Lk > <NUM> is satisfied, the processing proceeds to S212. If Lk ≤ <NUM> is satisfied, the processing proceeds to S213. According to the present embodiment, the case where the processing proceeds to S212 when Lk > <NUM> is satisfied will be described.

In the state of Lk > <NUM>, the remaining lifetime of the semiconductor device <NUM> that operates in the life-prolonging driving mode is shorter than the first lifetime threshold L2. The remaining lifetime of the semiconductor device <NUM> is close to <NUM> but is more than <NUM>. Accordingly, the processing proceeds to S212, and a process of continuing the operation of the semiconductor device <NUM> for a freely determined period Δt2 is performed. The processing returns to S210. At this time, since the remaining lifetime of the semiconductor device <NUM> is shorter than the second lifetime threshold L2, the period Δt2 that corresponds to an interval in which the remaining lifetime Lk is obtained is set shorter than the period Δt1 that corresponds to an interval in which the remaining lifetime Li is obtained in the case where the remaining lifetime is longer than the first lifetime threshold L1. This enables a time during which a normal operation of the semiconductor device <NUM> is not guaranteed to be detected with precision.

<FIG> illustrates the case where Lk ≤ <NUM> is satisfied.

In the state of Lk ≤ <NUM>, the remaining lifetime Lk of the semiconductor device <NUM> is equal to or less than <NUM>, and the operation is not guaranteed. The processing proceeds to S213, and it is reported to the outside that the operation of the semiconductor device <NUM> is not guaranteed. An example of a reporting method is to notify the user of the semiconductor device <NUM> by using, for example, a display not illustrated in <FIG>. In another example, the report is made to a monitoring system that monitors the semiconductor device <NUM> via a communication line. Subsequently, the processing proceeds to S214, and the operation of the semiconductor device <NUM> is stopped.

In the case in <FIG>, the remaining lifetime of the semiconductor device <NUM> is equal to or less than <NUM> at time k, and the operation of the semiconductor device <NUM> is not guaranteed. For this reason, it is reported to a location outside the semiconductor device <NUM> that the operation of the semiconductor device <NUM> is not guaranteed. The operation of the semiconductor device <NUM> is stopped.

Accordingly, the operation is prevented from continuing while the operation is not guaranteed even though the remaining lifetime of the semiconductor device <NUM> is equal to or less than <NUM>.

In this way, the semiconductor apparatus according to the present embodiment changes the driving state, based on the remaining lifetime information. This enable the semiconductor device <NUM> to be driven depending on the remaining lifetime. In the case where the remaining lifetime of the semiconductor device <NUM> is less than the first lifetime threshold, the mode is changed from the normal driving mode into the life-prolonging driving mode. This enables the lifetime of the semiconductor device <NUM> to be prolonged while the operation of the semiconductor device <NUM> continues. In the case where the remaining lifetime of the semiconductor device <NUM> is less than the second lifetime threshold, the interval in which the remaining lifetime information is obtained is shortened to a level lower than that in the case where the remaining lifetime is more than the first lifetime threshold. This enables the remaining lifetime of the semiconductor device <NUM> to be obtained with precision. In the case where the remaining lifetime of the semiconductor device <NUM> is equal to or less than <NUM>, this is reported to a location outside the semiconductor device <NUM>. This prevents the operation from continuing while the operation of the semiconductor device <NUM> is not guaranteed.

According to the present embodiment, the driving mode of the semiconductor device <NUM> is changed depending on whether the remaining lifetime exceeds the first lifetime threshold. This example is not a limitation. In the case where the remaining lifetime corresponds to the first length, the mode may be changed into the normal driving mode. In the case where the remaining lifetime corresponds to the second length shorter than the first length, the mode may be changed into the life-prolonging driving mode.

In an example described according to the present embodiment, the semiconductor apparatus includes the lifetime estimating unit <NUM> but is not limited to this example. According to an embodiment, the semiconductor apparatus may include a time obtaining unit that obtains time information that represents the elapsed time of the operation of the semiconductor device <NUM>. In this case and in the case where the time information corresponds to the first length, the semiconductor device <NUM> is driven in the first condition. In the case where the time information corresponds to the second length longer than the first length, the semiconductor device <NUM> is driven in the second condition. In the second condition, the throughput of the signal processing is less than that in the case where the semiconductor device is driven in the first condition, and the remaining lifetime of the semiconductor device is longer than that in the case where the semiconductor device is driven in the first condition.

In the description according to the present embodiment, an image sensor is used as the semiconductor device <NUM> according to the first embodiment.

An image capturing apparatus <NUM> illustrated in <FIG> includes the image sensor and the lifetime estimating unit <NUM>. An image sensor <NUM> is a CMOS sensor. A pixel array <NUM> that includes pixels <NUM> that are arranged in rows and in columns, a vertical scanning circuit <NUM>, vertical output lines <NUM>, and column circuits <NUM> is provided. The image sensor <NUM> includes a reference signal generating circuit <NUM>, storage units <NUM>, a counter circuit <NUM>, a horizontal scanning circuit <NUM>, a signal processing circuit <NUM>, an image capturing mode control unit <NUM>, and the lifetime estimating unit <NUM>.

The structure of each pixel <NUM> will now be described with reference to <FIG>.

Each pixel <NUM> includes photodiodes 501a and 501b. On the photodiodes 501a and 501b of the single pixel <NUM>, light that passes through a single microlens, not illustrated, is incident. In other words, the photodiodes 501a and 501b receive light from different regions of an exit pupil of an optical system that guides the light to the CMOS sensor. For this reason, the use of a signal based on electric charges that are obtained by photoelectric conversion carried out by the photodiodes 501a and 501b enables focus detection in a phase difference method.

An image can be generated by using a signal corresponding to an electric charge that is obtained by adding the electric charge that is obtained by the photoelectric conversion carried out by the photodiode 501a and the electric charge that is obtained by the photoelectric conversion carried out by the photodiode 501b.

The photodiodes 501a and 501b accumulate the electric charges depending on the incident light.

The vertical scanning circuit <NUM> illustrated in <FIG> performs vertical scanning in which signals are read in a unit of the rows from the pixels <NUM> that are arranged in the rows. The electric charge that is accumulated by the photodiode 501a is transferred to a floating diffusion unit (FD unit) <NUM> by setting a signal txa that is outputted to a transfer gate 502a at a high level.

The electric charge that is accumulated by the photodiode 501b is transferred to the FD unit <NUM> in a manner in which the vertical scanning circuit <NUM> sets a signal txb that is outputted to a transfer gate 502b at a high level.

The FD unit <NUM> converts the electric charges that are transferred from the photodiodes 501a and 501b into a voltage, based on the capacity of the FD unit <NUM>. The FD unit <NUM> is connected to a gate of an amplifier transistor <NUM>. The amplifier transistor <NUM> is connected to the vertical output line <NUM> via a selection transistor <NUM>.

Each pixel <NUM> includes a reset transistor <NUM> that is connected to the FD unit <NUM>. The reset transistor <NUM> resets the FD unit <NUM> in a manner in which the vertical scanning circuit <NUM> sets a signal res at a high level. In the case where the electric charges of the photodiodes 501a and 501b are reset by photodiode resetting, the vertical scanning circuit sets the signals txa and txb at a high level in the period in which the signal res is at a high level.

The signals res, txa, txb, and sel that are supplied from the vertical scanning circuit <NUM> are inputted into the pixels <NUM> in a single row in common. The outputs vout of the pixels <NUM> are inputted into the corresponding column circuits <NUM> via the vertical output lines <NUM> corresponding to the columns in which the pixels <NUM> are arranged.

The structure of each column circuit <NUM> that is arranged for every column will now be described with reference to <FIG>.

<FIG> illustrates the structure of each column circuit <NUM>. The column circuit <NUM> includes an electric current source <NUM>, a switch <NUM>, a differential amplifier <NUM>, a switch <NUM>, a comparator <NUM>, and a switch <NUM>.

The switch <NUM> switches between on and off of the electric current source <NUM>. The switch <NUM> switches between on and off of the differential amplifier <NUM>. The switch <NUM> switches between on and off of the comparator <NUM>.

When the switch <NUM> is on, the vertical scanning circuit <NUM> sets a signal sel that is supplied to the pixel <NUM> in the row to be read at a high level. Consequently, an electric current flows between the amplifier transistor <NUM> and the electric current source <NUM> via the selection transistor <NUM> of the pixel <NUM> in the row to be read. Consequently, the amplifier transistor <NUM> performs a source follower operation. Consequently, a signal (pixel signal) based on the voltage of the FD unit <NUM> is outputted to the vertical output line <NUM>.

When the switch <NUM> is on, the differential amplifier <NUM> operates. In this case, the differential amplifier <NUM> outputs, to the comparator <NUM>, an amplified signal that is obtained by amplifying the pixel signal that is outputted from the pixel <NUM> to the vertical output line <NUM>.

When the switch <NUM> is on, the comparator <NUM> operates. In this case, the comparator <NUM> compares the voltage of the amplifier signal that is outputted from the differential amplifier <NUM> and the voltage of a ramp signal that is supplied from the reference signal generating circuit <NUM>. The comparator <NUM> outputs, as a signal Cout, the result of comparison to each storage unit <NUM> illustrated in <FIG>. The signal level of the signal Cout changes when a relationship in the magnitude of the signal voltage is inverted. A count signal is inputted into the storage unit <NUM> from the counter circuit <NUM>. The count signal changes a count value in response to a timing with which the voltage of the ramp signal that is supplied from the reference signal generating circuit <NUM> starts changing. The storage unit <NUM> latches the count value, based on a timing with which the signal level of the signal Cout changes. This enables the storage unit <NUM> to obtain digital data corresponding to the signal level of the amplifier signal. In this way, the column circuit <NUM> and the storage unit <NUM> carry out AD conversion of the pixel signal.

The digital data that is stored in the storage unit <NUM> is sequentially transferred to the signal processing circuit <NUM> by using the horizontal scanning circuit <NUM> for every column. A series of operations in relation to readout of the pixel signals from the pixels are performed while a pixel row of the pixel array <NUM> is selected by the vertical scanning circuit <NUM>.

<FIG> are timing diagrams illustrating the operation of the image capturing apparatus illustrated in <FIG>. <FIG> is a timing diagram in the case where an image capturing operation is performed. <FIG> is a timing diagram in the case where a focus detection operation and the image capturing operation are performed. The pixel signals are read out from pixels <NUM> included in a part of the rows among the pixels <NUM> in the rows that are arranged in the pixel array <NUM> by performing the image capturing operation and the focus detection operation illustrated in <FIG>. The pixel signals are read out from pixels <NUM> included in another part of the rows by performing the image capturing operation illustrated in <FIG>. The part of the rows and the other part of the rows are assigned depending on a region in which the focus detection is made. In another example, the pixels <NUM> in all of the rows perform the operation in <FIG> for a certain frame. The operation in <FIG> may be performed for another frame. The frame described in this case may correspond to, for example, a period from when the vertical scanning circuit <NUM> starts vertical scanning until when the next vertical scanning is started. The frame can typically correspond to a period in which a control circuit, not illustrated, sets the signal level of a vertical synchronization signal for instructing the start of vertical scanning of the vertical scanning circuit <NUM> at a high level and subsequently sets the signal level of a vertical synchronization signal at a high level. In another example, the frame can also correspond to a period from the starts of reading a signal corresponding to one image that is generated by a signal processing unit from the pixel array <NUM> to the starts of reading a signal corresponding to the next one image from the pixel array <NUM>. For example, it is assumed that a video is shot at <NUM> fps (fps is an abbreviation for frames per second). In this example, the frame can correspond to a period in which a signal corresponding to an image of one frame of <NUM> frames starts to be read out from the pixel array <NUM>, and a signal corresponding to an image of a next one frame subsequently starts to be read out from the pixel array <NUM>.

The operation in <FIG> will now be described.

At time ta1, the vertical scanning circuit <NUM> sets the signal sel that is supplied to the row of the pixel <NUM> that performs the image capturing operation at a high level. Consequently, the selection transistor <NUM> of the pixel <NUM> is turned on.

Subsequently, at time ta2, the vertical scanning circuit <NUM> sets the signal res at a low level and turns the reset transistor <NUM> off. Consequently, the reset of the FD unit <NUM> is released.

The amplifier transistor <NUM> outputs a noise signal (N signal) based on the voltage of the FD unit <NUM> the reset of which is released to the vertical output line <NUM> via the selection transistor <NUM>. The N signal that is outputted to the vertical output line <NUM> is converted into digital data (N data) by AD conversion carried out by the column circuit <NUM> and the storage unit <NUM>.

At time ta3, the vertical scanning circuit <NUM> sets the signals txa and txb at a high level and subsequently sets the signals txa and txb at a low level. As a result of this operation, the electric charge that is accumulated based on the incident light by the photodiode 501a and the electric charge that is accumulated based on the incident light by the photodiode 501b are transferred to the FD unit <NUM>. Accordingly, the voltage of the FD unit <NUM> corresponds to the electric charge that is obtained by adding the electric charge of the photodiode 501a and the electric charge of the photodiode 501b.

The amplifier transistor <NUM> outputs a signal based on the voltage of the FD unit <NUM> that corresponds to the electric charge that is obtained by adding the electric charge of the photodiode 501a and the electric charge of the photodiode 501b, to the vertical output line <NUM> via the selection transistor <NUM>. The signal that is outputted by the amplifier transistor <NUM> will be described. If the voltage of the FD unit <NUM> is based on the electric charge that is accumulated based on the incident light by the photodiode 501a, the signal that is outputted by the amplifier transistor <NUM> is referred to as an A + N signal (the sum of an A signal and the N signal). If the voltage of the FD unit <NUM> is based on the electric charge that is accumulated based on the incident light by the photodiode 501b, the signal that is outputted by the amplifier transistor <NUM> is referred to as a B + N signal (the sum of a B signal and the N signal). In these cases, the signal that is outputted by the amplifier transistor <NUM> and that is based on the voltage of the FD unit <NUM> that corresponds to the electric charge that is obtained by adding the electric charge of the photodiode 501a and the electric charge of the photodiode 501b corresponds to a signal that is obtained by adding the A + N signal and the B + N signal. Accordingly, the signal that is outputted by the amplifier transistor <NUM> is referred to as an A + B + N signal. The A + B + N signal that is outputted to the vertical output line <NUM> is converted into digital data (A + B + N data) by AD conversion carried out by the column circuit <NUM> and the storage unit <NUM>.

At time ta5, the vertical scanning circuit <NUM> sets the signal res at a high level. Consequently, the voltage of the FD unit <NUM> is reset.

At time ta6, the vertical scanning circuit <NUM> sets the signal sel at a low level. This is the end of the selection of the pixels <NUM> in one row. Subsequently, the vertical scanning circuit <NUM> sets the signal sel of the pixels <NUM> in the rows for next to be read at a high level.

The N data and the A + B + N data that are held by the storage unit <NUM> are sequentially read out from the storage unit <NUM> into the signal processing circuit <NUM> by using the horizontal scanning circuit <NUM> for every column. The signal processing circuit <NUM> outputs A + B data that corresponds to a signal of a difference between the A + B + N data and the N data.

An operation at time tb1 is the same as an operation at time ta1 in <FIG>.

An operation at time tb2 is the same as an operation at time ta2 in <FIG>.

At time tb3, the vertical scanning circuit <NUM> sets the signal txa at a high level and subsequently sets the signal txa at a low level. Consequently, the electric charge that is accumulated based on the incident light by the photodiode 501a is transferred to the FD unit <NUM>. The amplifier transistor <NUM> outputs the A + N signal to the vertical output line <NUM> via the selection transistor <NUM>. The A + N signal that is outputted to the vertical output line <NUM> is converted into digital data (A + N data) by AD conversion carried out by the column circuit <NUM> and the storage unit <NUM>.

The FD unit <NUM> holds the electric charge that is transferred from the photodiode 501a until before time tb6.

At time tb6, the vertical scanning circuit <NUM> sets the signal txa and the signal txb at a high level and subsequently sets the signal txa and the signal txb at a low level. Consequently, the FD unit <NUM> holds the electric charge that is accumulated based on the incident light by the photodiode 501b and the electric charge that is accumulated in a period from time tb4 to time tb7 by the photodiode 501a in addition to the electric charge of the photodiode 501a that is held before time tb6. Accordingly, the voltage of the FD unit <NUM> corresponds to the electric charge that is obtained by adding the electric charge of the photodiode 501a and the electric charge of the photodiode 501b.

The amplifier transistor <NUM> outputs the A + B + N signal to the vertical output line <NUM>. The A + B + N signal that is outputted to the vertical output line <NUM> is converted into digital data (A + B + N data) by AD conversion carried out by the column circuit <NUM> and the storage unit <NUM>.

At time tb8, the vertical scanning circuit <NUM> sets the signal res at a high level. Consequently, the voltage of the FD unit <NUM> is reset.

At time tb9, the vertical scanning circuit <NUM> sets the signal sel at a low level. This is the end of the selection of the pixels <NUM> in one row. Subsequently, the vertical scanning circuit <NUM> sets the signal sel of the pixel <NUM> in the row for next to be read at a high level.

The N data, the A + N data, and the A + B + N data that are held by the storage unit <NUM> are sequentially read out from the storage unit <NUM> into the signal processing circuit <NUM> by using the horizontal scanning circuit <NUM> for every column. The signal processing circuit <NUM> outputs a signal (A data) of a difference between the A + N data and the N data and a signal of a difference between the A + B + n data and the N data (A + B data).

The signal processing unit, not illustrated, which receives the data that is outputted by the image capturing apparatus obtains B data that corresponds to a difference between the A data and the A + B data and performs the focus detection by using the A data and the B data. The signal processing unit generates an image by using the A + B data.

<FIG> is referred. The image capturing mode control unit <NUM> is a circuit block that includes the voltage control unit <NUM>, the driving mode selecting unit <NUM>, the frequency control unit <NUM>, and the signal generating unit <NUM> described according to the first embodiment. The image capturing mode control unit <NUM> receives the remaining lifetime information from the lifetime information receiving unit <NUM>. The voltage values that are supplied to all or a part of the circuits in the image sensor <NUM> or the period of the control signal are changed depending on the remaining lifetime that represents the remaining lifetime information. The circuits in the image sensor <NUM> are the pixel array <NUM>, the vertical scanning circuit <NUM>, the column circuits <NUM>, the reference signal generating circuit <NUM>, the storage units <NUM>, the counter circuit <NUM>, the horizontal scanning circuit <NUM>, and the signal processing circuit <NUM>. Alternatively, the operation of a part of the pixel array <NUM>, the vertical scanning circuit <NUM>, the column circuits <NUM>, the reference signal generating circuit <NUM>, the storage units <NUM>, the counter circuit <NUM>, the horizontal scanning circuit <NUM>, and the signal processing circuit <NUM> in the image sensor <NUM> is stopped.

As defined as expression (<NUM>) to expression (<NUM>) according to the first embodiment, the lifetime of the semiconductor apparatus is limited by the current density J of the wiring line, the temperature T of the wiring line, the voltage Vg that is applied to the gate of the transistor, and the maximum substrate current Isub that flows through the semiconductor substrate. The image capturing mode control unit <NUM> changes the state of the operation such that all or a part of the current density J, the temperature T of the wiring line, the voltage Vg that is applied to the gate of the transistor, and the maximum substrate current Isub that flows through the semiconductor substrate decrease in order to prolong the remaining lifetime of the image sensor <NUM>.

Consequently, the throughput of the signal processing of the image capturing apparatus is reduced, for example, the speed of the operation is decreased or the functions are partly stopped. However, while the image capturing operation continues, the lifetime of the image capturing apparatus can be increased.

In the life-prolonging driving mode of the image capturing mode control unit <NUM>, the frame rate, for example, decreases. Specifically, the period of the vertical scanning of the vertical scanning circuit <NUM> decreases relative to the normal driving mode. The decrease in the period of the vertical scanning results in decreases in the operation periods of the reference signal generating circuit <NUM>, the counter circuit <NUM>, the horizontal scanning circuit <NUM>, and the signal processing circuit <NUM>. It can also be said that a decrease in the frame rate means a decrease in the number of times the vertical scanning circuit <NUM> performs the vertical scanning per unit time.

In an example of the life-prolonging driving mode, signals may be read out from the pixels <NUM> in a region of a part of the pixel array <NUM>. That is, the number of the pixels that are read out in the life-prolonging driving mode is smaller than that in the normal driving mode. For example, the pixels <NUM> suitable for a resolution of 4K2K are read in the normal driving mode, and the pixels <NUM> suitable for a resolution of VGA are read in the life-prolonging driving mode. Because of the decrease in the number of the pixels <NUM> that are read, in the column circuits <NUM> and the storage units <NUM>, the horizontal scanning circuit <NUM> scans the storage units <NUM> in a part of the columns. Consequently, while the operation of the image capturing apparatus continues, the remaining lifetime of the image capturing apparatus can be prolonged. This example is one of the examples in which the throughput of the signal processing of the image capturing apparatus is reduced.

In another example, the number of the pixels that perform the focus detection operation in <FIG> in the life-prolonging driving mode may be less than that in the normal driving mode. Consequently, the number of times that the AD conversion of the A + N signal is carried out by the column circuits <NUM> and the storage units <NUM> can be decreased. Consequently, while the image capturing operation continues, the remaining lifetime of the image capturing apparatus can be prolonged. In particular, in the case where an extension in the remaining lifetime is considered to be important, no pixels perform the focus detection operation in <FIG>, and all of the pixels <NUM> carries out drive of the image capturing operation in <FIG>. This example is one of the examples in which the throughput of the signal processing of the image capturing apparatus is reduced.

The lifetime estimating unit <NUM> obtains the remaining lifetime information on the image sensor <NUM> by estimating the remaining lifetime of the image sensor <NUM>, based on the latest state of the operation of the image sensor <NUM> or the history information in relation to the state of the operation as described according to the first embodiment.

The state of the operation means a state in which the image sensor <NUM> operates such as an operation voltage or an operating frequency at which the image sensor <NUM> operates or a state caused as a result of the operation such as a temperature.

The lifetime estimating unit <NUM> obtains the remaining lifetime information after the image capturing operation is performed multiple times. The image capturing operation described herein means an operation for imaging an object in various environments. That is, the image capturing operation does not mean the image capturing operation of the image sensor <NUM> in a known environment such as a test in a manufacturing factory of an image capturing apparatus but means an image capturing operation in an environment in which the remaining lifetime of the image sensor <NUM> changes due to the environment in which the image sensor <NUM> is disposed. Accordingly, the image capturing apparatus according to the present embodiment can estimate the remaining lifetime depending on a previous environment or a current environment in which the image sensor <NUM> is actually disposed and the operation. Since the remaining lifetime is estimated based on the previous environment or the current environment in which the image sensor <NUM> is actually disposed, the remaining lifetime can be estimated more accurately than the lifetime is estimated in a simulation.

According to the embodiment of the present invention described above, as for the image sensor <NUM>, drive for prolonging the lifetime of the image sensor <NUM> is selected based on the remaining lifetime information. This solves a problem in that the image sensor <NUM> spends the lifetime earlier than a service life.

The image capturing apparatus according to the present embodiment can be used, for example, as a camera that is disposed in an area in which it is difficult to replace the image capturing apparatus. Examples of the camera include cameras (disaster monitoring (a volcanic disaster and a meteorological disaster) that are disposed in remote areas and monitoring cameras that are disposed in a space ship, an artificial satellite (such as a weather observation satellite, an exploration satellite, or a space station), and a space exploration robot), and monitoring cameras that are disposed in conflict areas. As for these cameras, the image capturing apparatus is not easy to replace even after the lifetime, and it is desired that the operation of the image capturing apparatus continues. Accordingly, in the case where the estimated remaining lifetime is shorter than the first lifetime threshold, the operation of the image sensor <NUM> is changed from that in the normal driving mode to that in the life-prolonging driving mode. Consequently, while the image capturing operation continues, the remaining lifetime of the operation of the image capturing apparatus can be prolonged.

The image capturing apparatus according to the present embodiment can be used as a vehicle-mounted camera that is taken as an example of the image capturing apparatus that is mounted on a moving object. As for the vehicle-mounted camera, the image sensor <NUM> is not easy to replace during driving even when the remaining lifetime is shorter than the first lifetime threshold. In this case, it is useful that the normal driving mode is changed into the life-prolonging driving mode, the image capturing operation of the image sensor <NUM> continues, and the lifetime of the image sensor <NUM> is increased.

The lifetime estimating unit <NUM> obtains the remaining lifetime information in parallel with the image capturing operation. For example, in the case of a camera that is disposed in a remote area or a camera that continuously shoots a video such as a monitoring camera, the lifetime estimating unit <NUM> obtains the remaining lifetime information in a period in which the image capturing operation is performed. Consequently, while the video is continuously shot, the remaining lifetime of the image sensor <NUM> can be estimated. In other words, the lifetime estimating unit <NUM> may obtain the remaining lifetime information in a period in which vertical scanning is performed. The lifetime estimating unit <NUM> may obtain the remaining lifetime information in a period in which the photoelectric converters accumulate electric charges based on the incident light. The lifetime estimating unit <NUM> may obtain the remaining lifetime information in a period (for example, a period in which AD conversion is carried out) in which the column circuits <NUM> and the storage units <NUM> process the pixel signals. That is, after the image capturing operation is performed multiple times, the lifetime estimating unit <NUM> may obtain the remaining lifetime information in parallel with the period in which the image capturing operation is performed.

Thus, even in the case where the image sensor is not easy to replace, the image capturing apparatus according to the present embodiment enables the lifetime of the image sensor to be prolonged while continuing the operation of the image sensor.

An image capturing apparatus according to the present embodiment, mainly differences from the second embodiment will be described. The image capturing apparatus according to the present embodiment can obtain the remaining lifetime information on the image sensor with precision.

<FIG> illustrates an image capturing apparatus <NUM> according to the present embodiment.

The image capturing apparatus <NUM> according to the present embodiment differs from the image capturing apparatus <NUM> illustrated in <FIG> in including a state obtaining unit <NUM> and a lifetime estimating unit <NUM> that is connected to the state obtaining unit <NUM>.

The following description includes the case where hot carrier injection (HC) is dominant regarding a factor in the impairment in the performance of the image sensor <NUM>. In this case, the remaining lifetime of the image sensor <NUM> is evaluated by using a ratio of a variation ΔId in a drain current over time with respect to a drain current Id of the transistor when the image sensor <NUM> starts operating as a new product, that is, the following expression (<NUM>): <MAT>.

The state obtaining unit <NUM> measures the drain current Id of the transistor of the image sensor <NUM>. The result of measurement is outputted to the lifetime estimating unit <NUM>. The lifetime estimating unit <NUM> calculates a ratio of a variation in the latest drain current Id of a MOS transistor to the known initial drain current Id of the MOS transistor.

It is assumed that the lifetime of the image sensor <NUM> is set to a time at which the IdRATIO in expression (<NUM>) becomes <NUM>, that is, <NUM>%. In this case, the lifetime estimating unit <NUM> sets a threshold for the IdRATIO to <NUM>%. The lifetime estimating unit <NUM> estimates the remaining lifetime of the image sensor <NUM> by comparing a threshold of <NUM>% and the actual IdRATIO.

In an example of a method of measuring the drain current Id of the transistor, a resistance element is disposed in the state obtaining unit <NUM>, an end of the resistance element is connected to the power supply voltage, and the other end of the resistance element is connected to the drain of the transistor of the image sensor <NUM>. A voltage value across both terminals of the resistance element is measured. Consequently, the drain current can be measured by using the expression of V = IR (V is the voltage across both terminals of the resistance element, R is the resistance value of the resistance element, and I is the value of an electric current that flows through the resistance element). The period until the IdRATIO becomes <NUM>%, that is, the remaining lifetime can be estimated by obtaining a variation in the IdRATIO over time.

In another method of obtaining the remaining lifetime information, a variation in electric power consumption of the image sensor <NUM> may be measured. The drain current Id of the transistor can be indirectly measured by measuring a variation between the electric power consumption at measurement and the electric power consumption of the image sensor <NUM> when the image sensor is a new product. The remaining lifetime can be estimated by obtaining a variation in the electric power consumption over time.

An image capturing apparatus according to the present embodiment, mainly differences from the third embodiment will be described. The structure of the image capturing apparatus can be the same as that in <FIG>.

The image capturing apparatus according to the third embodiment estimates the remaining lifetime by using the variation in the drain current Id. According to the present embodiment, another estimating method will be described.

The performance of the semiconductor device and the image sensor impairs due to the HC, the EM, the TDDB, or the negative NBTI as described according to the first embodiment. The lifetimes of the semiconductor device and the image sensor are determined by the impairment in the performance.

Lifetime estimation in relation to the impairment in the performance due to the HC is described according to the third embodiment.

A lifetime LT (EM) when the factor in the impairment in the performance is the EM is obtained by using expression (<NUM>) described according to the first embodiment.

A lifetime LT (TDDB) when the factor in the impairment in the performance is the TDDB is obtained by using expression (<NUM>) according to the first embodiment.

A lifetime LT (NBTI) when the factor in the impairment in the performance is the NBTI can be calculated by using expression (<NUM>) described according to the first embodiment.

A lifetime LT (HC) when the factor in the impairment in the performance is the HC is obtained by using expression (<NUM>) described according to the first embodiment and is also obtained by using expression (<NUM>) described according to the third embodiment.

The state obtaining unit <NUM> performs measurement of all or a part of expression (<NUM>) to expression (<NUM>) and calculates the lifetime of the image sensor <NUM>.

To calculate the lifetime of the image sensor <NUM> by using expression (<NUM>) to expression (<NUM>), the state obtaining unit <NUM> measures the current density J of the electric current that flows through the transistor or the resistance element that is included in the image sensor <NUM>, the temperature T of the wiring line, and the voltage value Vg that is applied to the gate of the transistor.

A method of obtaining the current density J is the same as that according to the third embodiment.

In an example of a method of obtaining the temperature T of the wiring line, the value of an electric current that flows through a diode element, not illustrated, of the state obtaining unit <NUM> is measured. The value of the electric current that flows through the diode element typically depends on the temperature. Accordingly, the temperature T can be obtained by measuring the value of the electric current.

The voltage value Vg can be a gate voltage that is applied to the transistor the lifetime of which is to be estimated. In the case where it can be expected that a difference between a measured value and a design value can be substantially ignored, the voltage value Vg may be the designed value.

The state obtaining unit <NUM> thus obtains the values that are used in expression (<NUM>) to expression (<NUM>). The state obtaining unit <NUM> outputs the obtained values to the lifetime estimating unit <NUM>. That is, the state obtaining unit <NUM> obtains the electric current consumption and the temperature of the image sensor <NUM>. The state obtaining unit <NUM> may further obtain information about the frame rate and the cumulative time of the operation of the image sensor <NUM>.

The lifetime estimating unit <NUM> obtains the lifetime of the image sensor <NUM> by using all or a part of expression (<NUM>) to expression (<NUM>). In the case where the lifetime is estimated by using multiple expressions among expression (<NUM>) to expression (<NUM>), the lifetime estimating unit <NUM> sets the lifetime of the image sensor <NUM> to the minimum lifetime among those obtained by the used expressions.

The lifetime estimating unit <NUM> according to the present embodiment thus estimates a lifetime based on the actual state of the operation of the image sensor <NUM>. Consequently, the lifetime can be estimated with precision.

The lifetime of the image sensor <NUM> greatly varies depending on an usage environment. The lifetime varies depending on the temperature at which the image sensor <NUM> is disposed, the number of times of shooting, and a supplied voltage. For example, a time during which the vehicle-mounted camera is left at a high temperature of <NUM> or more in a low latitude area is longer than that in a high latitude area. A time during which the vehicle-mounted camera is left at a low temperature of <NUM> or less in the high latitude area is longer than that in the low latitude area. Accordingly, the actual lifetime of the image sensor <NUM> can be shorter than a designed lifetime. Consequently, lifetime estimation based on the actual operation of the image sensor <NUM> enables the operation of the image sensor <NUM> to be unlikely to stop suddenly.

According to the present embodiment, the state obtaining unit <NUM> and the lifetime estimating unit <NUM> are disposed outside the image sensor <NUM>. However, the state obtaining unit <NUM> and the lifetime estimating unit <NUM> may be disposed inside the image sensor <NUM>. In particular, in the case where the state obtaining unit <NUM> is disposed on the same semiconductor substrate as the image sensor <NUM>, the state obtaining unit <NUM> is disposed in the same operation environment as the image sensor <NUM>, and the lifetime can be estimated with more precision.

An image capturing apparatus according to the present embodiment, mainly differences from the fourth embodiment will be described.

<FIG> illustrates the structure of the image capturing apparatus according to the present embodiment. The image capturing apparatus according to the present embodiment further includes a memory <NUM> relative to the image capturing apparatus according to the fourth embodiment.

The memory <NUM> holds the history information in relation to the state of the operation.

A lifetime estimating unit <NUM> according to the present embodiment estimates the remaining lifetime by using the history information that is held by the memory <NUM> and information that is outputted from the state obtaining unit <NUM> and generates the remaining lifetime information. Examples of the history information include a history of the image capturing mode of the image capturing apparatus and the cumulative time of the operation of the image capturing apparatus. The image capturing mode includes a period (frame rate) in which the image sensor <NUM> outputs an image signal, an image size, and whether the focus detection function is performed.

In an example of the operation, the remaining lifetime that is estimated by the lifetime estimating unit <NUM> by using all or a part of expression (<NUM>) to expression (<NUM>) as described according to the fourth embodiment is corrected by using the history information that is held by the memory <NUM>.

In another example of the operation, the memory <NUM> holds the IdRATIO that is obtained by using expression (<NUM>) whenever the remaining lifetime is estimated. The lifetime estimating unit <NUM> obtains the variation in the IdRATIO over time by using the IdRATIO at each time that is held by the memory <NUM>. From the variation in the IdRATIO over time, time at which the IdRATIO becomes <NUM>% or more is estimated (that is, the remaining lifetime is estimated). Consequently, the remaining lifetime can be estimated with more precision.

Values that are stored in the memory <NUM> are not limited to the drain current Id of the transistor but may be values in all or a part of expression (<NUM>) to expression (<NUM>).

The memory <NUM> may be located outside the image capturing apparatus <NUM>. The memory <NUM> is preferably a non-volatile memory. In the case where the memory <NUM> is disposed outside the image capturing apparatus <NUM>, processing of manufacturing the image capturing apparatus <NUM> and the memory <NUM> can be divided. Consequently, restrictions on a process of manufacturing the image capturing apparatus <NUM> and a process of manufacturing the memory <NUM> can be decreased.

The image capturing apparatus according to the present embodiment thus includes the memory <NUM> that holds the history information in relation to the state of the operation. The remaining lifetime can be estimated with precision by using the history information that is held by the memory <NUM>.

Each image capturing apparatus described according to the above embodiments can be a multi-layer type sensor in which chips are stacked. For example, a first chip may include the pixel array <NUM>, and a second chip may include the column circuits <NUM>, the reference signal generating circuit <NUM>, the storage units <NUM>, the counter circuit <NUM>, the horizontal scanning circuit <NUM>, the signal processing circuit <NUM>, and the image capturing mode control unit <NUM>. In this case, the second chip may further include the lifetime estimating unit <NUM>. A third chip that includes a signal processing unit that generates an image may also be stacked. In this case, the third chip may include the lifetime estimating unit <NUM>.

<FIG> is a block diagram illustrating the structure of an image capturing system <NUM> according to the present embodiment. The image capturing system <NUM> according to the present embodiment includes an image capturing apparatus <NUM> that has the same structure as one of the image capturing apparatuses according to the above embodiments. Specific examples of the image capturing system <NUM> include a digital still camera, a digital camcorder, and a monitoring camera. <FIG> illustrates an example of the structure of a digital still camera that uses one of the image capturing apparatuses according to the above embodiments as the image capturing apparatus <NUM>.

The image capturing system <NUM> illustrated in <FIG> by way of example includes the image capturing apparatus <NUM>, a lens <NUM> that images an optical image of the object on the image capturing apparatus <NUM>, an aperture stop <NUM> that enables the amount of light that passes through the lens <NUM> to be variable, and a barrier <NUM> that protects the lens <NUM>. The lens <NUM> and the aperture stop <NUM> are optical systems that focus light on the image capturing apparatus <NUM>.

The image capturing system <NUM> also includes a signal processing unit <NUM> that processes an output signal that is outputted from the image capturing apparatus <NUM>. The signal processing unit <NUM> corrects and compresses an input signal in various ways as needed and performs output-signal processing. The signal processing unit <NUM> may have a function of carrying out AD conversion of the output signal that is outputted from the image capturing apparatus <NUM>. In this case, an AD conversion circuit is not necessarily included inside the image capturing apparatus <NUM>.

The image capturing system <NUM> also includes a buffer memory <NUM> in which image data is temporarily stored and an external interface (external I/F) <NUM> for communication with, for example, an external computer. The image capturing system <NUM> also includes a recording medium <NUM> such as a semiconductor memory that is used to record or read image capturing data and a recording medium control interface (recording medium control I/F) <NUM> that is used for recording to or reading from the recording medium <NUM>. The recording medium <NUM> may be contained in the image capturing system <NUM> and may be removable and installable.

The image capturing system <NUM> also includes an overall control-calculation unit <NUM> that makes various kinds of calculations and controls the entire digital still camera and a timing generating unit <NUM> that outputs a various kinds of timing signals to the image capturing apparatus <NUM> and the signal processing unit <NUM>. The timing signals may be inputted from the outside. It is only necessary for the image capturing system <NUM> to include at least the image capturing apparatus <NUM>, the signal processing unit <NUM> that processes the output signal that is outputted from the image capturing apparatus <NUM>. The overall control-calculation unit <NUM> and the timing generating unit <NUM> may perform a part or the whole of the control function of the image capturing apparatus <NUM>.

The image capturing apparatus <NUM> outputs an image signal to the signal processing unit <NUM>. The signal processing unit <NUM> performs predetermined signal processing on the image signal that is outputted from the image capturing apparatus <NUM> and outputs image data. The signal processing unit <NUM> generates an image by using the image signal.

The use of image capturing apparatuses according to the above embodiments to form the image capturing system enables the image capturing system to obtain an image having improved quality.

An image capturing system according to the present embodiment and a moving object will be described with reference to <FIG>, <FIG> and <FIG>.

<FIG> and <FIG> schematically illustrate examples of the structure of the image capturing system according to the present embodiment and the moving object. <FIG> is a flowchart illustrating the operation of the image capturing system according to the present embodiment.

In an example described according to the present embodiment, the image capturing system relates to a vehicle-mounted camera. <FIG> and <FIG> illustrate examples of a vehicle system and the image capturing system that is mounted thereon. An image capturing system <NUM> includes image capturing apparatuses <NUM>, image preprocessing units <NUM>, an integrated circuit <NUM>, and optical systems <NUM>. The optical systems <NUM> forms an optical image of the object on the image capturing apparatuses <NUM>. The image capturing apparatuses <NUM> convert the optical image of the object that is formed by the optical systems <NUM> into electrical signals. Each image capturing apparatus <NUM> is one of the image capturing apparatuses according to the above embodiments. The image preprocessing units <NUM> perform predetermined signal processing on signals that are outputted from the image capturing apparatuses <NUM>. The function of the image preprocessing units <NUM> may be incorporated in the image capturing apparatuses <NUM>. The image capturing system <NUM> includes at least two sets of the optical systems <NUM>, the image capturing apparatuses <NUM>, and the image preprocessing units <NUM>. The outputs from the sets of the image preprocessing units <NUM> are inputted into the integrated circuit <NUM>.

The integrated circuit <NUM> is used for the image capturing system and includes an image processing unit <NUM> that includes a memory <NUM>, an optical distance measuring unit <NUM>, a parallax calculation unit <NUM>, an object recognition unit <NUM>, and an anomaly detection unit <NUM>. The image processing unit <NUM> performs image processing such as a development process or defect correction on the output signals of the image preprocessing units <NUM>. The memory <NUM> stores first record of a captured image and the position of a defect of an imaging pixel. The optical distance measuring unit <NUM> is used for focus on the object and distance measurement. The parallax calculation unit <NUM> calculates parallax (phase difference of a parallax image) from multiple image data that is obtained by the image capturing apparatuses <NUM>. The object recognition unit <NUM> recognizes the subject such as a car, a road, a sign, or a person. The anomaly detection unit <NUM> reports anomaly to a main control unit <NUM> when detecting the anomaly of the image capturing apparatuses <NUM>.

The integrated circuit <NUM> may be built with an exclusively designed hardware, may be built with a software module, or may be built with a combination thereof. Alternatively, the integrated circuit <NUM> may be built with, for example, a FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) or may be built with a combination thereof.

The main control unit <NUM> manages and controls operation of, for example, the image capturing system <NUM>, a vehicle sensor <NUM>, and a control unit <NUM>. The main control unit <NUM> may not be included, and the image capturing system <NUM>, the vehicle sensor <NUM>, and the control unit <NUM> can include respective communication interfaces and can transmit and receive a control signal via a communication network (for example, a CAN standard).

The integrated circuit <NUM> has a function of receiving a control signal from the main control unit <NUM> or transmitting a control signal or a configuration value to each image capturing apparatus <NUM> by using its own control unit. For example, the integrated circuit <NUM> transmits configurations for pulse-driving a voltage switch <NUM> in the image capturing apparatus <NUM>, configurations for switching the voltage switch <NUM> for every frame, or other configurations.

The image capturing system <NUM> is connected to the vehicle sensor <NUM> and can detect a state of a vehicle that is running such as a vehicle speed, a yaw rate, and a steering angle, and states of an environment outside the vehicle, another vehicle, or an obstacle. The vehicle sensor <NUM> also functions as a distance information obtaining unit that obtains information about a distance to the object based on the parallax image. The image capturing system <NUM> is connected to a driving assistance control unit <NUM> that assists a driving in various ways such as automatic steering, automatic cruising, and a function of preventing a collision. In particular, regarding a collision determination function, whether a vehicle collides with another vehicle or an obstacle is presumed, and whether the vehicle collides is determined, based on the result of detection of the image capturing system <NUM> or the vehicle sensor <NUM>. Consequently, in the case where the collision is presumed, avoidance control is started, and a safety device is activated in the collision.

The image capturing system <NUM> is also connected to an alarm device <NUM> that gives an alarm to the driver, based on the result of determination of a collision determination unit. For example, in the case where the collision determination unit has determined that there is a high possibility of a collision, the main control unit <NUM> controls the vehicle to avoid colliding or to reduce damage, for example, in a manner in which the brake is applied, an accelerator is released, or an engine output is reduced. The alarm device <NUM> gives an alarm to a user, for example, in a manner in which the alarm is given by using, for example, a voice, alarm information is displayed on a screen of a display of, for example, a car navigation system or a meter panel, or a sheet belt or a steering wheel is vibrated.

According to the present embodiment, the image capturing system <NUM> takes a picture of the vicinity of the vehicle, for example, the front or rear thereof.

<FIG> illustrates an example of the arrangement of the image capturing system <NUM> in the case where the image capturing system <NUM> takes the picture of the front of the vehicle.

The two image capturing apparatuses <NUM> are disposed at the front of a vehicle <NUM>. Specifically, the two image capturing apparatuses <NUM> are disposed symmetrically with each other with respect to a symmetric axis that coincides with a center line that extends in a direction in which the vehicle <NUM> runs forwards or backwards or a center line of an external shape (for example, the width of the vehicle). This is preferable when information about the distance between the vehicle <NUM> and the object is obtained or the possibility of a collision is determined. The image capturing apparatuses <NUM> are preferably disposed at positions at which the view of the driver is not obstructed when the driver sights circumstances outside the vehicle <NUM> from a driver's seat. The alarm device <NUM> is preferably disposed at a position at which the driver easily sees the alarm device <NUM>.

Failure detecting operation of the image capturing apparatuses <NUM> of the image capturing system <NUM> will now be described with reference to <FIG>. The failure detecting operation of the image capturing apparatuses <NUM> is performed through steps S810 to S880 illustrated in <FIG>.

At step S810, the configurations of the image capturing apparatuses <NUM> during a startup process are set. That is, the configurations for the operation of the image capturing apparatuses <NUM> are transmitted from a location (for example, the main control unit <NUM>) outside the image capturing system <NUM> or inside the image capturing system <NUM>, and the image capturing operation and the failure detecting operation of the image capturing apparatuses <NUM> are started.

Subsequently, at step S820, pixel signals are obtained from effective pixels. At step S830, output values are obtained from failure detecting pixels that are disposed for detecting failure. The failure detecting pixels include respective photoelectric converters as in the effective pixels. A predetermined voltage is stored in the photoelectric converters. The failure detecting pixels output signals corresponding to the voltage stored in the photoelectric converters. The order of the step S820 and the step S830 may be inverted.

Subsequently, at step S840, whether expected output values of the failure detecting pixels match the actual output values from the failure detecting pixels is determined.

In the case where the result of determination at step S840 is that the expected output values match the actual output values, the processing proceeds to step S850, in which determination that the image capturing operation is normal is made, and the processing proceeds to step S860. At step S860, the pixel signals in a scan row are transmitted to the memory <NUM> and are temporarily stored therein. Subsequently, the processing returns to step S820, and the failure detecting operation continues.

In the case where the result of determination at step S840 is that the expected output values do not match the actual output values, the processing proceeds to step S870. At step S870, determination that the image capturing operation is abnormal is made, and an alarm is given to the main control unit <NUM> or the alarm device <NUM>. The alarm device <NUM> displays detection of anomaly on the display. Subsequently, at step S880, the image capturing apparatuses <NUM> are stopped, and the operation of the image capturing system <NUM> ends.

An example described according to the present embodiment illustrates that the flowchart is looped for every row. The flowchart, however, may be looped for every set of rows. The failure detecting operation may be performed for every frame.

At step S870, the alarm may be given to a location outside the vehicle via a wireless network.

The control described according to the present embodiment is exerted to avoid a collision with another vehicle. The present embodiment can be used to control automatic driving for following another vehicle or to control automatic driving for preventing the vehicle from moving out of a lane. The image capturing system <NUM> is not limited to the vehicle such as a car and can be used control automatic driving for following another vehicle or to control automatic driving for preventing the vehicle from moving out of a lane. The image capturing system <NUM> is not limited to the vehicle such as a car and can be used for a moving object (moving device) such as a ship, an aircraft, or an industrial robot. In addition to the moving object, the image capturing system <NUM> may be used for instrument used for recognizing objects in a wide range such as an intelligent transport system (ITS).

The present disclosure is not limited to the above embodiments. Various modifications can be made.

For example, embodiments of the present disclosure include an embodiment of a combination of a part of one of the above embodiments and the other embodiment, and an embodiment that is obtained by replacing a part of one of the above embodiments with a part of the other embodiment.

The above embodiments are specifically described by way of example to carry out the present invention. The technical scope of the present invention is not interpreted in a limited range because of the above embodiments. That is, the present invention can be carried out as various embodiments without departing from the technical concept or principal features thereof.

Claim 1:
An image capturing apparatus (<NUM>, <NUM>) comprising:
an image sensor (<NUM>) that comprises pixels (<NUM>) that are arranged in rows and in columns and that include respective photoelectric converters (501a, 501b), and that is configured to perform an image capturing operation for reading out signals of the pixels;
a lifetime estimating unit (<NUM>, <NUM>, <NUM>) that is configured to estimate a remaining lifetime of the image sensor, and
a driving control unit (<NUM>) that is configured to control drive of the image sensor,
characterized in that
in a case, where the remaining lifetime is longer than a predetermined length, the driving control unit (<NUM>) is configured to drive the image sensor in a first condition, and
in a case, where the remaining lifetime is shorter than the predetermined length, the driving control unit (<NUM>) is configured to drive the image sensor in a second condition,
wherein the number of the pixels that are read out in the second condition is smaller than the number of the pixels that are read out in the first condition, or the number of the pixels that perform a focus detection operation in the second condition is smaller than the number of the pixels that perform a focus detection operation in the first condition.