Patent Description:
Generally, methods for reading charges generated in photodiodes (e.g. in a camera) are known.

After a predetermined exposure time, an (electric) shutter may be used, such that no more charges are generated in the photodiode after the exposure time has expired. The exposure time may be set automatically or based on a user preference.

In such devices and methods, the generated charges are drained from the photodiode and cannot be restored, such that such methods may be referred to as destructive readout methods. Non-destructive readout methods are also known. For example, a parasitic capacitance between the photodiode and a deep trench isolation (DTI) may be utilized to sense the photodiode charge in order to determine a readout time point, which will be discussed under reference of <FIG>.

Patent application publication <CIT> describes a pixel cell including a photodiode coupled to photogenerate image charge in response to incident light. <CIT> describes a matrix-addressed imaging panel that includes one or more AEC electrode receptive field regions that provide a signal representative of exposure specific, respective AEC electrode receptive field regions.

Although there exist techniques for reading out photodiodes, it is generally desirable to provide image element readout circuitry, an image element, and an image element readout method.

According to a first aspect, the invention provides image element readout circuitry in accordance with independent claim <NUM>. According to a second aspect, the invention provides an image element in accordance with claim <NUM>. According to a third aspect, the invention provides an image element readout method in accordance with independent claim <NUM>.

Further aspects are set forth in the dependent claims, the following description and the drawings.

Unless explicitly indicated as "embodiment(s) according to the claimed invention", any embodiment, example, aspect or implementation in the description may include some but not all features as literally defined in the claims and are present for illustration purposes only.

Before a detailed description of the embodiments starting with <FIG> is given, general explanations are made.

As mentioned in the outset, non-destructive pixel readout methods are generally known.

However, it has been recognized that it is desirable to provide downwards scalability of pixels, such that a high resolution can be achieved at a low-cost, for example in the case of CMOS (complementary metal oxide semiconductor) image sensors with a high dynamic range (HDR).

Known HDR pixels (e.g. DCG (dual conversion gain), split PD (photodiode), LOFIC (lateral overflow integration capacitor), or the like) may be challenging to scale below approximately two micrometers since a photodiode may become too small to have a sufficient low-light performance.

Hence, it has been recognized that it may be suitable to decrease a size of readout circuitry.

It has further been recognized that ultra-high-resolution HDR sensors which combine (or bin) neighboring pixels may have a high manufacturing complexity and thus, a high cost.

Moreover, known CMOS image sensor pixel may use correlated double sampling (CDS) for reading or sensing accumulated signal charges in a photodiode, thus creating a proportional output voltage.

However, CDS readout is typically destructive, i.e. there is no more charge left in the photodiode after the sensing is completed. Furthermore, a new integration process for photo charge may be carried out before each readout.

Hence, it has been recognized that it is desirable to provide an automatic control for integration time on each pixel (or group of pixels) according to a light level in the pixel(s), e.g. based on non-destructive pixel-readout, and thereby also increasing a dynamic range of an image sensor.

However, in known pixels, non-destructive pixel-readout may be achieved by providing a deep trench isolation (DTI), which increases a manufacturing cost.

Non-destructive readout as it is generally known is discussed under reference of <FIG> depicting a pixel <NUM>.

In the pixel <NUM>, a photodiode <NUM> is embedded into an N-well of a semiconductor material <NUM>.

The pixel <NUM> further includes a pinning layer <NUM> underneath a silicon dioxide dielectric layer <NUM>, a transfer gate <NUM>, and a floating diffusion <NUM>, as it is generally known.

Moreover, a DTI <NUM> is provided including a metal fill which is coupled to the photodiode <NUM> via a sensing transistor <NUM>, such that a parasitic capacitance between the photodiode and the DTI can be used to determine a voltage change in the photodiode by a sensing circuit <NUM>, which is connected to the DTI <NUM>. The pixel <NUM> is provided on a p-sup layer <NUM>, as it is generally known.

However, it has been recognized that in terms of (downwards) scalability and manufacturing costs, it may be desirable to avoid using a DTI.

Therefore, some embodiments pertain to image element readout circuitry configured to: sense an amount of electric carriers in a photosensitive element, wherein the photosensitive element is configured to generate the electric carriers in response to light being incident on the photosensitive element; and read the electric carriers from the photosensitive element when the amount of electric carriers has reached a predetermined value.

Circuitry may pertain to any entity or multitude of entities which is adaptable to sense and/or read electric carriers according to the present disclosure and/or which is configurable to generate control signals for sensing and/or reading electric carriers, such as a CPU (central processing unit), GPU (graphics processing unit), FPGA (field-programmable gate array), or the like.

The image element readout circuitry may be provided in or for an image element, which may be based on known (semiconductor) technologies, such as CMOS (complementary metal oxide semiconductor), or the like.

Hence, the image element may include a photosensitive element which, when light is incident on the photosensitive element, is configured to generate electric carriers, as it is generally known. The photosensitive element may be based on a photodiode, for example.

Accordingly, the electric carriers may be any electric carriers generated in a photodetection process, such as electrons, electron-hole-pairs, holes, or the like.

The electric carriers may be sensed, after they are generated, as being present in the photosensitive element, for example as a charge, a voltage, a current, or the like (which will be discussed further below). As it is generally known in the field of imaging, charges are typically read from the image element or from the photosensitive element after a predetermined amount of exposure time.

However, according to the present disclosure, the charges are read from the photosensitive element when it is sensed that the amount of electric carriers in the photosensitive element has reached a predetermined value.

For example, if the exposure time is too long or too short in known imaging devices, a resulting image may be over-exposed or under-exposed since it is not determined at which point of time it is sensible to read the electric carriers.

Thus, according to the present disclosure, the charges may only be read when the photosensitive element has been sufficiently exposed with light, but not too much, such that an over- or underexposure may be avoided.

The predetermined value may be any value and may depend on how the amount of electric carriers is sensed. For example, it may be a relative value (e.g. between zero and one or zero and a hundred percent, such as twenty percent, fifty percent, seventy percent, or the like) or an absolute value.

For example, the amount of electric carriers may be sensed based on capacitive sensing in which case the predetermined value may have the dimension of a charge, for example.

Accordingly, the amount of electric carriers may be sensed as an absolute amount of, for example, electrons, or as a charge or may be sensed based on any other indirect or direct way.

Hence, according to the present disclosure, a non-destructive sensing of the electric carriers in the photosensitive element during an image capture process is achieved. Thus, a pixel-individual integration time can be provided, thereby further increasing a dynamic range of an image sensor (in case of multiple pixels or image elements) and a signal to noise ratio (e.g. by averaging multiple read operations). Non-destructive readout according to the present disclosure may also be used to multi-sample an image element or pixel to measure a rate of change of a signal, thus further increasing a dynamic range. Furthermore, per pixel A/D (analog/digital) conversion may be provided, e.g., by measuring a time (by a respective clock) for a pixel to reach a certain signal threshold level. Moreover, according to the present disclosure, image element readout circuitry may be provided which may be applied to a back-side illuminated image sensor as well as to a front-side illuminated image sensor.

In some embodiments, the amount of electric carriers in the photosensitive element is sensed based on capacitive sensing, as discussed above.

A parasitic capacitance with respect to the photosensitive element may be used for sensing the amount of electric carriers. For example, a parasitic capacitance may be present since charged elements (e.g. the photodiode) and a further element have a known distance (known due to manufacturing, for example), such that they may be modelled as a capacitor, as it is generally known. The parasitic capacitance may thus be indicative of the charges or electric carriers being present in the photosensitive element.

Accordingly, in some embodiments, the image element readout circuitry is further configured to: capacitively sense a voltage change in the photosensitive element which is indicative of the amount of electric carriers in the photosensitive element. For example, capacitive sensing may be envisaged, such that, based on a parasitic capacitance between the photodiode and a further element (e.g. a reference voltage node, ground, a pinning layer, a transfer gate, a floating diffusion, a laminating layer, a semiconductor layer, a doped region, or the like), the amount of electric carriers in the photosensitive element may be determined.

The voltage change may be a voltage drop or a voltage rise which may be proportional to the amount of electric carriers (accumulated from photons) in the photosensitive element or wherein any other mathematic relation may be determinable.

In some embodiments, the amount of electric carriers (or photodiode charge) is further sensed by capacitively sensing a voltage with respect to a reference voltage (or ground). For example, the parasitic capacitance between the photosensitive element and reference voltage (or ground) may be used to sense electric carriers in the photosensitive element.

Hence, a capacitive sensing circuit or a sense node may be provided between ground and the capacitor. Based on this, for example, the voltage change may be determined.

In some embodiments, the amount of electric carriers is further sensed based on a capacitive coupling between a transfer gate and the photosensitive element.

A transfer gate may be provided in the image element, as it is generally known and thus, a capacitive sensing circuit may be coupled to the transfer gate. For example, based on such a configuration, a parasitic capacitance between the photosensitive element and the transfer gate may be used to sense the amount of electric carriers in the photosensitive element, as discussed herein.

In some embodiments, the amount of electric carriers is further sensed based on a capacitive coupling between a pinning layer and the photosensitive element.

As it is generally known, a pinning layer may be provided in an image element and thus, it may be capacitively coupled with the photosensitive element , such that it may enable sensing of the photodiode charge, e.g. based on a sensing of a voltage change, or the like.

Hence, in some embodiments a voltage change may be determined based on a capacitive coupling, as discussed herein.

In some embodiments, the amount of electric carriers is sensed based on threshold voltage modulation sensing (also referred to as Vth modulation sensing).

As it is generally known, Vth modulation sensing may use an adaptive sensing voltage for sensing electric carriers. The applied voltage of a sensing node or sense circuit may change with respect to the electric carriers being present in the photosensitive element, such that the amount of electric carriers in the photosensitive element is sensed based on the voltage (change). Such voltage change will result in Vth modulation which can be translated into a corresponding photodiode charge.

As discussed herein, when the amount of electric carriers has reached a predetermined value, the electric carriers are read from the photosensitive element (for generating an imaging signal).

In some embodiments, the electric carriers are read from the photosensitive element based on correlated double sampling.

Correlated double sampling (CDS) is generally known, such that an extensive discussion thereof is omitted. However, it should be noted that the present disclosure is not limited to any type of CDS, such that analog or digital CDS may be envisaged. By using CDS, a signal to noise ratio may further be optimized.

In the case that multiple image elements are used (e.g. in a pixel array), after the CDS readout, each pixel value may be linearized (or normalized), e.g. by dividing the pixel value by the integration time. For example, different pixels may need different integration times until they reach the predetermined value, but if they are not linearized, they may be differently exposed, such that a color distribution in the image may be deteriorated.

Moreover, in the case of multiple pixels, each pixel (image element) may be provided with a row and a column select switch, such that a pixel-individual readout may be achieved. Alternatively, group select switches may be used if a predetermined correlation between neighboring pixels is assumed.

The concepts of the present disclosure may further be combined with event driven techniques (e.g. bus arbitration), without limiting the present disclosure in that regard.

Some embodiments pertain to an image element including: a photosensitive element; and image element readout circuitry configured to: sense an amount of electric carriers in the photosensitive element, wherein the photosensitive element is configured to generate the electric carriers in response to light being incident on the photosensitive element; and read the electric carriers from the photosensitive element when the amount of electric carriers reached a predetermined value, as discussed herein.

In some embodiments, the amount of electric carriers in the photosensitive element is sensed based on capacitive sensing, as discussed herein. In some embodiments, the image element readout circuitry is further configured to: capacitively sense a voltage change in the photosensitive element which is indicative of the amount of electric carriers in the photosensitive element, as discussed herein. In some embodiments, the amount of electric carriers is further sensed by capacitively sensing a voltage with respect to a reference voltage, as discussed herein. In some embodiments, the amount of electric carriers is further sensed based on a capacitive coupling between a transfer gate and the photosensitive element, as discussed herein. In some embodiments, the amount of electric carriers is further sensed based on a capacitive coupling between a pinning layer and the photosensitive element, as discussed herein. In some embodiments, the amount of electric carriers is sensed based on threshold voltage modulation sensing, as discussed herein. In some embodiments, the electric carriers are read from the photosensitive element based on correlated double sampling.

Some embodiments pertain to an image element readout method including: sensing an amount of electric carriers in a photosensitive element, wherein the photosensitive element is configured to generate the electric carriers in response to light being incident on the photosensitive element; and reading the electric carriers from the photosensitive element when the amount of electric carriers has reached a predetermined value, as discussed herein.

The image element readout method may be carried out with image element readout circuitry according to the present disclosure.

In some embodiments, the image element readout method further includes: capacitively sensing a voltage change in the photosensitive element which is indicative of the amount of electric carriers in the photosensitive element, as discussed herein. In some embodiments, the amount of electric carriers is sensed based on threshold voltage modulation sensing, as discussed herein. In some embodiments, the electric carriers are read from the photosensitive element based on correlated double sampling, as discussed herein.

The methods as described herein are also implemented in some embodiments as a computer program causing a computer and/or a processor to perform the method, when being carried out on the computer and/or processor. In some embodiments, also a non-transitory computer-readable recording medium is provided that stores therein a computer program product, which, when executed by a processor, such as the processor described above, causes the methods described herein to be performed.

Returning to <FIG>, there is depicted an image element <NUM> according to the present disclosure in a circuit diagram.

The image element <NUM> includes a photodiode <NUM>. Symbolically, for depicting a parasitic capacitance which is present with respect to the photodiode, a capacitor <NUM> is depicted which is coupled to a sensing circuit <NUM> (also referred to as a non-destructive photodiode sensing circuit).

However, as discussed herein, a parasitic capacitance may be present due to the charged elements, such that the capacitor <NUM> is not provided, in some embodiments, and it is therefore depicted in dashed lines. This also applies to the upcoming <FIG>, <FIG>, <FIG> symbolically depicting capacitors <NUM>, <NUM>, <NUM>, and <NUM> for illustrating the presence of a capacitance without providing a capacitor.

Returning to <FIG>: The sensing circuit <NUM> is configured to sense an amount of electric carriers in the photodiode <NUM> by sensing or determining a parasitic capacitance voltage drop between the photodiode <NUM> and a further element which is not specified herein, but which will be discussed in the following figures.

The image element <NUM> further includes selection circuitry <NUM>, such that the photodiode can be read when it is sensed that the amount of electric carriers has reached a predetermined value.

The selection circuitry includes a transfer gate switch <NUM> which is coupled to the photodiode <NUM>. The transfer gate switch <NUM> is further coupled to a reset gate switch <NUM> and to a source follower SF <NUM>.

The transistor SF <NUM> is further coupled to a column select switch <NUM> with one pin and with another pin to a row select switch <NUM>.

In this embodiment, the image element readout circuitry includes the sensing circuit <NUM> and the selection circuitry <NUM>.

A capture sequence may be carried out as follows (which will be described in pseudo-code):.

<FIG> depicts an image element including image element <NUM> readout circuitry according to the present disclosure.

The image element <NUM> includes a photodiode <NUM> which is embedded in an N-well of a semiconductor material <NUM>. The image element <NUM> further includes a pinning layer <NUM>, a transfer gate <NUM>, and a floating diffusion <NUM>. The floating diffusion <NUM> is further coupled with a reset switch <NUM> which is coupled to a supply voltage VDD <NUM>.

Between the photodiode <NUM> and a reference voltage VREF (which may be ground, in some embodiments), a symbolic capacitor <NUM> is depicted for illustrating the presence of a parasitic capacitance. Moreover, a sensing switch <NUM> is provided between ground GND (optionally VREF) and a sense node <NUM> for determining the parasitic capacitance voltage drop caused by accumulation of negatively charged photo-electrons, as discussed herein. The image element <NUM> is provided on a p-sup layer <NUM>, as it is generally known.

The image element <NUM> can be operated with an image element readout method <NUM>, as will be discussed in the following under reference of <FIG> and <FIG>.

<FIG> depicts a flow-chart of the image element readout method <NUM> which is similar to the pseudo-code given above. Generally, the pseudo-code given above or the image element readout method <NUM> may be applied to all embodiments of the present disclosure and is not limited to the embodiments of <FIG> and <FIG>.

At <NUM>, the photodiode <NUM> is reset, the sense node <NUM> is pre-set to Vref and an iteration value i is set to zero.

At <NUM>, the reset switch <NUM> is turned off (i.e. disconnected), such that the sense node <NUM> is made floating (i.e. has a high impedance).

At <NUM>, light (signal) is integrated, such that the iteration i is incremented, at <NUM>.

At <NUM>, a voltage (V_sense) at the sense node <NUM> is measured. The measured voltage is compared to a threshold voltage V_th, at <NUM>. If V_sense is smaller or equal to V_th, the light is integrated again, at <NUM>, and so on.

At <NUM>, if V_sense is larger than V_th, the photodiode is read with CDS, as discussed herein.

Moreover, the electric carriers are linearized based on the i-value (i.e. the measured signal is divided by the number of iterations), such that, in a case of multiple pixels or image elements, the light signal is roughly brought to the same level (in case, other image elements do not need as much or more iterations).

However, it is not necessary, in some embodiments, that Vth is reached at a maximum integration time (which may depend on a sensor frame rate). In such a case, the image element may be read by "regular" CDS.

<FIG> depicts a diagram <NUM> for illustrating the sensing according to the present disclosure.

On an ordinate of the diagram <NUM>, a voltage of the photodiode V_PD is depicted versus a time on an abscissa.

Moreover, iterations i are depicted at which a voltage V_pin (or V_sense) on the sense node is determined. At a beginning of the acquisition, V_pin is set to GND, as discussed herein. It should be noted that, if V_sense is measured, it may be shifted down an absolute value, i.e. may start at GND instead of V_pin.

After an Nth integration, it is determined that the measured voltage V_PD has reached a value below the threshold value V_th, such that the photodiode is read. When the photodiode has been read, the photodiode is reset, as discussed herein. The measurement time, after the Nth has been carried out is named T_int (integration time), such that the photodiode is read before a saturation is reached.

<FIG> depicts an embodiment according to the claimed invention of an image element <NUM> according to the present disclosure in which a parasitic capacitance between a photodiode and a transfer gate is determined.

The image element <NUM> is different from the image element <NUM> in that a parasitic capacitance <NUM> is present between the photodiode and the transfer gate <NUM> instead of ground. Hence, a corresponding sensing circuit <NUM> is coupled to the transfer gate <NUM>, such that a respective parasitic capacitance can be used for sensing the amount of electric carriers in the photodiode, as discussed herein.

The remaining elements correspond to the image element <NUM>, such that a repetitive description thereof is omitted.

An acquisition process or capture process for the image element <NUM> may be carried out according to the following pseudo-code:.

<FIG> depicts a further embodiment of an image element <NUM> according to the present disclosure which is different than the image elements <NUM> and <NUM> in that a parasitic capacitance <NUM> is between the photodiode <NUM> and the pinning layer <NUM> is used. Accordingly, a sense circuit <NUM> is coupled to the pinning layer <NUM>.

The remaining elements are similar as in the other image elements discussed herein, such that a repetitive description thereof is omitted.

<FIG> depicts an embodiment of a sensing circuit <NUM> which utilizes CDS (as an embodiment of image element readout circuitry according to the present disclosure) in a block diagram.

The sensing circuit <NUM> includes a comparator <NUM> which compares a pre-charge voltage V_precharge with a threshold voltage V_th, such that a parasitic capacitance with respect to a photodiode <NUM> is used. When a difference between V_precharge and V_th reaches a predetermined value, an integration is stopped and the photodiode <NUM> is read out.

<FIG> depicts a further embodiment of image element readout circuitry <NUM> according to the present disclosure for reading out a photosensitive element <NUM> based on Vth modulation sensing, as discussed herein.

A sensing circuit SENS is provided next to the photosensitive element <NUM>, which is supplied with a supply voltage VDD, which is coupled to a first floating diffusion node FD. A second floating diffusion node FD is coupled with a signal line voltage VSL which is coupled ground, via a diode. Moreover, the second floating diffusion node FD is coupled with a third floating diffusion node FD, which is coupled with a reset transistor, wherein the reset transistor is supplied with VDD. Moreover, the third floating diffusion is coupled with an amplification transistor AMP (which is supplied with VDD), wherein the amplification transistor AMP is further coupled with a selection transistor SEL, which is coupled with the diode. A transfer gate TG is further provided between SENS and the third floating diffusion.

<FIG> depicts plane view of the image element readout circuitry <NUM>, such that it is shown that the sensing circuit SENS is provided on top of the photosensitive element <NUM> and the photodiode <NUM> is coupled to the floating diffusion FD via the transfer gate TG. The remaining elements correspond to the elements as discussed under reference of <FIG> and a repetitive description thereof if omitted.

<FIG> depicts a timing diagram <NUM> according to which control signals are applied to the image element readout circuitry <NUM>. A sense signal is applied to the sensing circuit SENS at a first point of time t1 for a first time duration T1 as a non-destructive signal.

According to the embodiment described under reference of <FIG>, an electrical reliability of non-destructive sensing is provided, wherein a coupling between the photodiode and the sensing circuit is optimized.

At a second point of time t2 (which is when T1 is over), which lies after t1, a reset signal is applied to RST for a duration T2 and a selection signal is applied to SEL for a duration T3.

Roughly in the middle of T3, a reset level is reached. In a last section of T3, a point of time t3 is reached in which a pulse is applied to the transfer gate for roughly the duration T2.

If a reset is needed, at a time point t4, which lies after T3 is over, a pulse for the duration T2 is applied to SENS, after which, for the duration T3 again, a signal is applied to SEL for reaching a signal level.

At the end of such a readout sequence, a negative bias exists between TG and SENS.

<FIG> depicts a further embodiment of an image element readout method <NUM> according to the present disclosure.

At <NUM>, a voltage change in the photosensitive element is capacitively sensed, which is indicative of the amount of electric carriers in the photosensitive element, as discussed herein.

At <NUM>, the electric carriers are read from the photosensitive element based on CDS, as discussed herein.

At <NUM>, an amount of electric carriers in a photosensitive element is sensed based on Vth modulation sensing, as discussed herein.

<FIG> is a cross-sectional illustration of a semiconductor device <NUM> according to the present disclosure.

The semiconductor device <NUM> includes a photodiode PD which is provided between an on-chip color filter OCCF (on which an on-chip lens OCL is provided) and readout circuitry <NUM> according to the present disclosure.

The readout circuitry <NUM> includes a vertical signal line VSL <NUM> which is connected to a floating diffusion <NUM>. Next to the vertical signal line <NUM>, a sense circuit node <NUM> is provided which is adapted to capacitively sense electric carriers in the photodiode PD, as discussed herein. Moreover, a reference voltage VDD node <NUM> is provided next to the sense circuit node <NUM> which is connected to a further floating diffusion below a substrate boundary <NUM> and to a further floating diffusion above the substrate boundary <NUM>.

Next to the reference voltage VDD node <NUM>, a reset transistor RST <NUM> is provided, as discussed above, and a transfer gate <NUM> is provided next to the reset transistor. Roughly below the transfer gate <NUM>, a floating diffusion <NUM>. Moreover, a connection <NUM> is provided above the floating diffusion <NUM>, to which the reference voltage VDD is connected. The floating diffusion is further connected with an amplifier <NUM>. The transfer gate <NUM> extends into the first silicon substrate <NUM>, which is indicated by a same hatching.

Next to the transfer gate, a selection transistor SEL <NUM> is provided and next to the selection transistor SEL <NUM>, a further vertical signal line VSL <NUM> is provided which is connected with a floating diffusion above the substrate boundary <NUM>.

The photodiode PD is provided in a first silicon substrate <NUM> and the readout circuitry is provided on a second silicon substrate <NUM> and partly extending through the second silicon substrate <NUM> with respective connection conductors. Floating diffusions are provided in the respective silicon substrates, i.e. floating diffusions above the substrate boundary <NUM> are provided in the second silicon <NUM> substrate and floating diffusions below the substrate boundary <NUM> are provided in the first silicon substrate <NUM>, or the like.

Moreover, left and right of the photodiode, in the first semiconductor substrate <NUM>, RDTIs (rear deep trench isolations) <NUM> are provided, wherein such RDTIs may also be applied in other embodiments.

<FIG> depicts a further embodiment of a (part of) a semiconductor device <NUM> according to the present disclosure in two different perspectives. On top, a view is shown in which a shallow trench isolation <NUM> is provided next to an SiO<NUM> insulating portion <NUM>.

On the bottom, it is shown that a vertical transfer <NUM> gate is provided between the shallow trench isolations, such that a voltage can be sensed based on capacitive sensing between the vertical transfer gate and the photodiode (not depicted).

Generally, STIs (shallow trench isolations) and VTGs (vertical transfer gates) may be applied separately or together in the embodiments of the present disclosure.

The technology according to an embodiment of the present disclosure is applicable to various products. For example, the technology according to an embodiment of the present disclosure may be implemented as a device included in a mobile body that is any of kinds of automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility vehicles, airplanes, drones, ships, robots, construction machinery, agricultural machinery (tractors), and the like.

<FIG> is a block diagram depicting an example of schematic configuration of a vehicle control system <NUM> as an example of a mobile body control system to which the technology according to an embodiment of the present disclosure can be applied. The vehicle control system <NUM> includes a plurality of electronic control units connected to each other via a communication network <NUM>. In the example depicted in <FIG>, the vehicle control system <NUM> includes a driving system control unit <NUM>, a body system control unit <NUM>, a battery control unit <NUM>, an outside-vehicle information detecting unit <NUM>, an in-vehicle information detecting unit <NUM>, and an integrated control unit <NUM>. The communication network <NUM> connecting the plurality of control units to each other may, for example, be a vehicle-mounted communication network compliant with an arbitrary standard such as controller area network (CAN), local interconnect network (LIN), local area network (LAN), FlexRay (registered trademark), or the like.

Each of the control units includes: a microcomputer that performs arithmetic processing according to various kinds of programs; a storage section that stores the programs executed by the microcomputer, parameters used for various kinds of operations, or the like; and a driving circuit that drives various kinds of control target devices. Each of the control units further includes: a network interface (I/F) for performing communication with other control units via the communication network <NUM>; and a communication I/F for performing communication with a device, a sensor, or the like within and without the vehicle by wire communication or radio communication. A functional configuration of the integrated control unit <NUM> illustrated in <FIG> includes a microcomputer <NUM>, a general-purpose communication I/F <NUM>, a dedicated communication I/F <NUM>, a positioning section <NUM>, a beacon receiving section <NUM>, an in-vehicle device I/F <NUM>, a sound/image output section <NUM>, a vehicle-mounted network I/F <NUM>, and a storage section <NUM>. The other control units similarly include a microcomputer, a communication I/F, a storage section, and the like.

The driving system control unit <NUM> may have a function as a control device of an antilock brake system (ABS), electronic stability control (ESC), or the like.

The driving system control unit <NUM> is connected with a vehicle state detecting section <NUM>. The vehicle state detecting section <NUM>, for example, includes at least one of a gyro sensor that detects the angular velocity of axial rotational movement of a vehicle body, an acceleration sensor that detects the acceleration of the vehicle, and sensors for detecting an amount of operation of an accelerator pedal, an amount of operation of a brake pedal, the steering angle of a steering wheel, an engine speed or the rotational speed of wheels, and the like. The driving system control unit <NUM> performs arithmetic processing using a signal input from the vehicle state detecting section <NUM>, and controls the internal combustion engine, the driving motor, an electric power steering device, the brake device, and the like.

The body system control unit <NUM> controls the operation of various kinds of devices provided to the vehicle body in accordance with various kinds of programs.

The battery control unit <NUM> controls a secondary battery <NUM>, which is a power supply source for the driving motor, in accordance with various kinds of programs. For example, the battery control unit <NUM> is supplied with information about a battery temperature, a battery output voltage, an amount of charge remaining in the battery, or the like from a battery device including the secondary battery <NUM>. The battery control unit <NUM> performs arithmetic processing using these signals, and performs control for regulating the temperature of the secondary battery <NUM> or controls a cooling device provided to the battery device or the like.

The outside-vehicle information detecting unit <NUM> detects information about the outside of the vehicle including the vehicle control system <NUM>. For example, the outside-vehicle information detecting unit <NUM> is connected with at least one of an imaging section <NUM> and an outside-vehicle information detecting section <NUM>. The imaging section <NUM> includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside-vehicle information detecting section <NUM>, for example, includes at least one of an environmental sensor for detecting current atmospheric conditions or weather conditions and a peripheral information detecting sensor for detecting another vehicle, an obstacle, a pedestrian, or the like on the periphery of the vehicle including the vehicle control system <NUM>.

The environmental sensor, for example, may be at least one of a rain drop sensor detecting rain, a fog sensor detecting a fog, a sunshine sensor detecting a degree of sunshine, and a snow sensor detecting a snowfall. The peripheral information detecting sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR device (Light detection and Ranging device, or Laser imaging detection and ranging device). Each of the imaging section <NUM> and the outside-vehicle information detecting section <NUM> may be provided as an independent sensor or device, or may be provided as a device in which a plurality of sensors or devices are integrated.

<FIG> depicts an example of installation positions of the imaging section <NUM> and the outside-vehicle information detecting section <NUM>. Imaging sections <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> are, for example, disposed at at least one of positions on a front nose, sideview mirrors, a rear bumper, and a back door of the vehicle <NUM> and a position on an upper portion of a windshield within the interior of the vehicle. The imaging section <NUM> provided to the front nose and the imaging section <NUM> provided to the upper portion of the windshield within the interior of the vehicle obtain mainly an image of the front of the vehicle <NUM>. The imaging sections <NUM> and <NUM> provided to the sideview mirrors obtain mainly an image of the sides of the vehicle <NUM>. The imaging section <NUM> provided to the rear bumper or the back door obtains mainly an image of the rear of the vehicle <NUM>. The imaging section <NUM> provided to the upper portion of the windshield within the interior of the vehicle is used mainly to detect a preceding vehicle, a pedestrian, an obstacle, a signal, a traffic sign, a lane, or the like.

Incidentally, <FIG> depicts an example of photographing ranges of the respective imaging sections <NUM>, <NUM>, <NUM>, and <NUM>. An imaging range a represents the imaging range of the imaging section <NUM> provided to the front nose. Imaging ranges b and c respectively represent the imaging ranges of the imaging sections <NUM> and <NUM> provided to the sideview mirrors. An imaging range d represents the imaging range of the imaging section <NUM> provided to the rear bumper or the back door. A bird's-eye image of the vehicle <NUM> as viewed from above can be obtained by superimposing image data imaged by the imaging sections <NUM>, <NUM>, <NUM>, and <NUM>, for example.

Outside-vehicle information detecting sections <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> provided to the front, rear, sides, and corners of the vehicle <NUM> and the upper portion of the windshield within the interior of the vehicle may be, for example, an ultrasonic sensor or a radar device. The outside-vehicle information detecting sections <NUM>, <NUM>, and <NUM> provided to the front nose of the vehicle <NUM>, the rear bumper, the back door of the vehicle <NUM>, and the upper portion of the windshield within the interior of the vehicle may be a LIDAR device, for example. These outside-vehicle information detecting sections <NUM> to <NUM> are used mainly to detect a preceding vehicle, a pedestrian, an obstacle, or the like.

Returning to <FIG>, the description will be continued. The outside-vehicle information detecting unit <NUM> makes the imaging section <NUM> image an image of the outside of the vehicle, and receives imaged image data. In addition, the outside-vehicle information detecting unit <NUM> receives detection information from the outside-vehicle information detecting section <NUM> connected to the outside-vehicle information detecting unit <NUM>. In a case where the outside-vehicle information detecting section <NUM> is an ultrasonic sensor, a radar device, or a LIDAR device, the outside-vehicle information detecting unit <NUM> transmits an ultrasonic wave, an electromagnetic wave, or the like, and receives information of a received reflected wave. On the basis of the received information, the outside-vehicle information detecting unit <NUM> may perform processing of detecting an object such as a human, a vehicle, an obstacle, a sign, a character on a road surface, or the like, or processing of detecting a distance thereto. The outside-vehicle information detecting unit <NUM> may perform environment recognition processing of recognizing a rainfall, a fog, road surface conditions, or the like on the basis of the received information. The outside-vehicle information detecting unit <NUM> may calculate a distance to an object outside the vehicle on the basis of the received information.

In addition, on the basis of the received image data, the outside-vehicle information detecting unit <NUM> may perform image recognition processing of recognizing a human, a vehicle, an obstacle, a sign, a character on a road surface, or the like, or processing of detecting a distance thereto. The outside-vehicle information detecting unit <NUM> may subject the received image data to processing such as distortion correction, alignment, or the like, and combine the image data imaged by a plurality of different imaging sections <NUM> to generate a bird's-eye image or a panoramic image. The outside-vehicle information detecting unit <NUM> may perform viewpoint conversion processing using the image data imaged by the imaging section <NUM> including the different imaging parts.

The driver state detecting section <NUM> may include a camera that images the driver, a biosensor that detects biological information of the driver, a microphone that collects sound within the interior of the vehicle, or the like. The biosensor is, for example, disposed in a seat surface, the steering wheel, or the like, and detects biological information of an occupant sitting in a seat or the driver holding the steering wheel. The in-vehicle information detecting unit <NUM> may subject an audio signal obtained by the collection of the sound to processing such as noise canceling processing or the like.

The integrated control unit <NUM> controls general operation within the vehicle control system <NUM> in accordance with various kinds of programs. The integrated control unit <NUM> is connected with an input section <NUM>. The input section <NUM> is implemented by a device capable of input operation by an occupant, such, for example, as a touch panel, a button, a microphone, a switch, a lever, or the like. The integrated control unit <NUM> may be supplied with data obtained by voice recognition of voice input through the microphone. The input section <NUM> may, for example, be a remote control device using infrared rays or other radio waves, or an external connecting device such as a mobile telephone, a personal digital assistant (PDA), or the like that supports operation of the vehicle control system <NUM>. The input section <NUM> may be, for example, a camera. In that case, an occupant can input information by gesture. Alternatively, data may be input which is obtained by detecting the movement of a wearable device that an occupant wears. Further, the input section <NUM> may, for example, include an input control circuit or the like that generates an input signal on the basis of information input by an occupant or the like using the above-described input section <NUM>, and which outputs the generated input signal to the integrated control unit <NUM>. An occupant or the like inputs various kinds of data or gives an instruction for processing operation to the vehicle control system <NUM> by operating the input section <NUM>.

The storage section <NUM> may include a read only memory (ROM) that stores various kinds of programs executed by the microcomputer and a random access memory (RAM) that stores various kinds of parameters, operation results, sensor values, or the like. In addition, the storage section <NUM> may be implemented by a magnetic storage device such as a hard disc drive (HDD) or the like, a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

The general-purpose communication I/F <NUM> is a communication I/F used widely, which communication I/F mediates communication with various apparatuses present in an external environment <NUM>. The general-purpose communication I/F <NUM> may implement a cellular communication protocol such as global system for mobile communications (GSM (registered trademark)), worldwide interoperability for microwave access (WiMAX (registered trademark)), long term evolution (LTE (registered trademark)), LTE-advanced (LTE-A), or the like, or another wireless communication protocol such as wireless LAN (referred to also as wireless fidelity (Wi-Fi (registered trademark)), Bluetooth (registered trademark), or the like. The general-purpose communication I/F <NUM> may, for example, connect to an apparatus (for example, an application server or a control server) present on an external network (for example, the Internet, a cloud network, or a company-specific network) via a base station or an access point. In addition, the general-purpose communication I/F <NUM> may connect to a terminal present in the vicinity of the vehicle (which terminal is, for example, a terminal of the driver, a pedestrian, or a store, or a machine type communication (MTC) terminal) using a peer to peer (P2P) technology, for example.

The dedicated communication I/F <NUM> is a communication I/F that supports a communication protocol developed for use in vehicles. The dedicated communication I/F <NUM> may implement a standard protocol such, for example, as wireless access in vehicle environment (WAVE), which is a combination of institute of electrical and electronic engineers (IEEE) <NUM>. 11p as a lower layer and IEEE <NUM> as a higher layer, dedicated short range communications (DSRC), or a cellular communication protocol. The dedicated communication I/F <NUM> typically carries out V2X communication as a concept including one or more of communication between a vehicle and a vehicle (Vehicle to Vehicle), communication between a road and a vehicle (Vehicle to Infrastructure), communication between a vehicle and a home (Vehicle to Home), and communication between a pedestrian and a vehicle (Vehicle to Pedestrian).

The positioning section <NUM>, for example, performs positioning by receiving a global navigation satellite system (GNSS) signal from a GNSS satellite (for example, a GPS signal from a global positioning system (GPS) satellite), and generates positional information including the latitude, longitude, and altitude of the vehicle. Incidentally, the positioning section <NUM> may identify a current position by exchanging signals with a wireless access point, or may obtain the positional information from a terminal such as a mobile telephone, a personal handyphone system (PHS), or a smart phone that has a positioning function.

The beacon receiving section <NUM>, for example, receives a radio wave or an electromagnetic wave transmitted from a radio station installed on a road or the like, and thereby obtains information about the current position, congestion, a closed road, a necessary time, or the like. Incidentally, the function of the beacon receiving section <NUM> may be included in the dedicated communication I/F <NUM> described above.

The in-vehicle device I/F <NUM> is a communication interface that mediates connection between the microcomputer <NUM> and various in-vehicle devices <NUM> present within the vehicle. The in-vehicle device I/F <NUM> may establish wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), near field communication (NFC), or wireless universal serial bus (WUSB). In addition, the in-vehicle device I/F <NUM> may establish wired connection by universal serial bus (USB), high-definition multimedia interface (HDMI (registered trademark)), mobile high-definition link (MHL), or the like via a connection terminal (and a cable if necessary) not depicted in the figures. The in-vehicle devices <NUM> may, for example, include at least one of a mobile device and a wearable device possessed by an occupant and an information device carried into or attached to the vehicle. The in-vehicle devices <NUM> may also include a navigation device that searches for a path to an arbitrary destination. The in-vehicle device I/F <NUM> exchanges control signals or data signals with these in-vehicle devices <NUM>.

The vehicle-mounted network I/F <NUM> is an interface that mediates communication between the microcomputer <NUM> and the communication network <NUM>. The vehicle-mounted network I/F <NUM> transmits and receives signals or the like in conformity with a predetermined protocol supported by the communication network <NUM>.

The microcomputer <NUM> of the integrated control unit <NUM> controls the vehicle control system <NUM> in accordance with various kinds of programs on the basis of information obtained via at least one of the general-purpose communication I/F <NUM>, the dedicated communication I/F <NUM>, the positioning section <NUM>, the beacon receiving section <NUM>, the in-vehicle device I/F <NUM>, and the vehicle-mounted network I/F <NUM>. For example, the microcomputer <NUM> may calculate a control target value for the driving force generating device, the steering mechanism, or the braking device on the basis of the obtained information about the inside and outside of the vehicle, and output a control command to the driving system control unit <NUM>. For example, the microcomputer <NUM> may perform cooperative control intended to implement functions of an advanced driver assistance system (ADAS) which functions include collision avoidance or shock mitigation for the vehicle, following driving based on a following distance, vehicle speed maintaining driving, a warning of collision of the vehicle, a warning of deviation of the vehicle from a lane, or the like. In addition, the microcomputer <NUM> may perform cooperative control intended for automatic driving, which makes the vehicle to travel autonomously without depending on the operation of the driver, or the like, by controlling the driving force generating device, the steering mechanism, the braking device, or the like on the basis of the obtained information about the surroundings of the vehicle.

The microcomputer <NUM> may generate three-dimensional distance information between the vehicle and an object such as a surrounding structure, a person, or the like, and generate local map information including information about the surroundings of the current position of the vehicle, on the basis of information obtained via at least one of the general-purpose communication I/F <NUM>, the dedicated communication I/F <NUM>, the positioning section <NUM>, the beacon receiving section <NUM>, the in-vehicle device I/F <NUM>, and the vehicle-mounted network I/F <NUM>. In addition, the microcomputer <NUM> may predict danger such as collision of the vehicle, approaching of a pedestrian or the like, an entry to a closed road, or the like on the basis of the obtained information, and generate a warning signal. The warning signal may, for example, be a signal for producing a warning sound or lighting a warning lamp.

The sound/image output section <NUM> transmits an output signal of at least one of a sound and an image to an output device capable of visually or auditorily notifying information to an occupant of the vehicle or the outside of the vehicle. In the example of <FIG>, an audio speaker <NUM>, a display section <NUM>, and an instrument panel <NUM> are illustrated as the output device. The display section <NUM> may, for example, include at least one of an on-board display and a head-up display. The display section <NUM> may have an augmented reality (AR) display function. The output device may be other than these devices, and may be another device such as headphones, a wearable device such as an eyeglass type display worn by an occupant or the like, a projector, a lamp, or the like. In a case where the output device is a display device, the display device visually displays results obtained by various kinds of processing performed by the microcomputer <NUM> or information received from another control unit in various forms such as text, an image, a table, a graph, or the like. In addition, in a case where the output device is an audio output device, the audio output device converts an audio signal constituted of reproduced audio data or sound data or the like into an analog signal, and auditorily outputs the analog signal.

Incidentally, at least two control units connected to each other via the communication network <NUM> in the example depicted in <FIG> may be integrated into one control unit. Alternatively, each individual control unit may include a plurality of control units. Further, the vehicle control system <NUM> may include another control unit not depicted in the figures. In addition, part or the whole of the functions performed by one of the control units in the above description may be assigned to another control unit. That is, predetermined arithmetic processing may be performed by any of the control units as long as information is transmitted and received via the communication network <NUM>. Similarly, a sensor or a device connected to one of the control units may be connected to another control unit, and a plurality of control units may mutually transmit and receive detection information via the communication network <NUM>.

Incidentally, a computer program for realizing the functions of image element readout circuitry according to the present disclosure can be implemented in one of the control units or the like. In addition, a computer readable recording medium storing such a computer program can also be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. In addition, the above-described computer program may be distributed via a network, for example, without the recording medium being used.

In the vehicle control system <NUM> described above, image element readout circuitry according to the present disclosure can be applied to the integrated control unit <NUM> in the application example depicted in <FIG>.

In addition, at least part of the image element readout circuitry according to the present disclosure may be implemented in a module (for example, an integrated circuit module formed with a single die) for the integrated control unit <NUM> depicted in <FIG>. Alternatively, image element readout circuitry according to the present disclosure may be implemented by a plurality of control units of the vehicle control system <NUM> depicted in <FIG>.

It should be recognized that the embodiments describe methods with an exemplary ordering of method steps. The specific ordering of method steps is however given for illustrative purposes only and should not be construed as binding. For example the ordering of <NUM> and <NUM> in the embodiment of <FIG> may be exchanged. Other changes of the ordering of method steps may be apparent to the skilled person.

In some embodiments, also a non-transitory computer-readable recording medium is provided that stores therein a computer program product, which, when executed by a processor, such as the processor described above, causes the methods described to be performed.

All units and entities described in this specification and claimed in the appended claims can, if not stated otherwise, be implemented as integrated circuit logic, for example on a chip, and functionality provided by such units and entities can, if not stated otherwise, be implemented by software.

Claim 1:
Image element readout circuitry configured to:
sense an amount of electric carriers in a photosensitive element (<NUM>), wherein the photosensitive element (<NUM>) is configured to generate the electric carriers in response to light being incident on the photosensitive element (<NUM>); and
read the electric carriers from the photosensitive element (<NUM>) when the amount of electric carriers has reached a predetermined value wherein the amount of electric carriers in the photosensitive element (<NUM>) is sensed based on capacitive sensing,
wherein the capacitive sensing uses a capacitive coupling between a transfer gate (<NUM>) and the photosensitive element.