Patent ID: 12192644

The use of cross-hatching or shading in the accompanying figures is generally provided to clarify the boundaries between adjacent elements and to facilitate legibility of the figures. Accordingly, neither the presence nor the absence of cross-hatching or shading conveys or indicates any preference or requirement for particular materials, material properties, element proportions, element dimensions, commonalities of similarly illustrated elements, or any other characteristic, attribute, or property for any element illustrated in the accompanying figures.

Additionally, it should be understood that the proportions and dimensions (either relative or absolute) of the various features and elements (and collections and groupings thereof) and the boundaries, separations, and positional relationships presented therebetween, are provided in the accompanying figures merely to facilitate an understanding of the various embodiments described herein and, accordingly, may not necessarily be presented or illustrated to scale, and are not intended to indicate any preference or requirement for an illustrated embodiment to the exclusion of embodiments described with reference thereto.

DETAILED DESCRIPTION

The provided disclosure relates to pulse-width modulation (PWM) image sensors and, particularly, to PWM image sensor arrays on a stacked wafer that orient a charge-to-time converter (CTC) and a time-to-digital converter (TDC) in a Z-direction, thereby reducing a size of an image sensor pixel in an XY-plane. Further aspects of the provided disclosure provide flexible control of a transfer function of a TDC of a PWM image sensor. Other aspects of the provided disclosure are additionally provided herein.

Many electronic devices, such as smart phones, tablets, laptop computers, and so on, comprise one or more cameras for capturing image and/or video information. For example, a smart phone may have one or more cameras configured to capture high-resolution images and videos. The captured images and videos may be stored in a local storage of the smart phone and/or may be transmitted, via a network, to other devices for the purposes of sharing social media, such as pictures or videos, with other users and/or for any other purpose where images or videos are used.

Image sensors may be used in various types of cameras, as referenced above, and may comprise an array of multiple pixels which convert analog information (e.g., electromagnetic radiation, such as light waves) into digital signals for use in, and/or display on, an electronic device. Through this process of analog-to-digital conversion, a digital image may be created that corresponds to a scene and/or real-world objects that are the subjects of an image-capture operation. The digital image may be created by agglomerating digital signals from each pixel associated with the image sensor. As a result, increasing a number of pixels for a particular image sensor may cause a corresponding increase in a resolution of a resulting image.

However, electronic devices typically have limited space for internal components, as other components typical in electronic devices, such as microphones, displays, sensors, and so on, may require a portion of the limited space available in an electronic device. There is, therefore, limited space for pixels in an image sensor and a resolution of an image sensor may likewise be limited by the amount of space available. Furthermore, design considerations (e.g., a device thickness or aesthetic appearance) may further limit the amount of space available in a particular electronic device, which may further limit space available for an array of image sensor pixels, thereby resulting in a decreased resolution of images captured by an associated image sensor and/or camera.

Additionally, certain image sensor architecture, such as certain CMOS sensor architectures, may produce images with certain deficiencies in quality. For example, traditional sensors may struggle producing high quality images in low or high light conditions, in situations where an object is moving with respect to the image sensor, in producing images without pixel saturation, and in producing images with a high dynamic range.

The provided disclosure relates to a PWM image sensor that may have a reduced pixel size, thereby increasing a number of potential pixels in a particular area, may permit flexible control of a digital-domain transfer function, may provide single-shot high dynamic range (HDR) imaging, may reduce movement-blur, and may provide additional functionality as described below. The PWM image sensor may utilize PWM control during time-based conversion processes, by, for example, controlling a switch supplying voltage and current to a load. The average value of the voltage and current may be modified by changing a rate of the switch.

In accordance with the provided disclosure, a PWM image sensor may comprise a charge-to-time converter (CTC) and a time-to-digital converter (TDC). The CTC may be positioned on a top wafer and the TDC may be positioned on a logic wafer, where the logic wafer is positioned below the top wafer in a vertical orientation (e.g., a Z-direction). By utilizing space in the Z-direction, the PWM image sensor may have a higher pixel density in an XY-plane. Stacking the CTC and the TDC at the wafer-level may further reduce the overall die and/or module size when compared to traditional image sensors.

The CTC may be communicatively coupled with the TDC, such as through one or more electrical traces, and may transmit write signals to the TDC. As discussed herein, the CTC may generate a write (WRT) signal when a sense node, otherwise referenced as a floating diffusion (FD), of the CTC accumulates a threshold number of electrons from a photocurrent generated by a photodiode of the CTC. Once the threshold number of electrons is reached, the WRT signal may be generated and sent to the TDC in order to latch, or otherwise mark, image data. In some cases, the threshold value for the threshold number of electrons is modifiable, either manually or automatically, in order to minimize a signal-to-noise ratio (SNR), extend a dynamic range (for all pixels, or a portion thereof, of the PWM sensor), control an exposure setting, and so on.

The TDC may further permit control of a transfer function in a digital domain during the time-to-digital conversion process. Through control of the transfer function, a dynamic range (DR) of the PWM image sensor may be extended during certain conditions (e.g., low-light conditions or high-light conditions). Control of various additional functions, such as over a SNR, exposure settings (either manual or automatic), selection of triggering time, and so on, may be achieved through the use of the PWM image sensor.

These and other embodiments are discussed below with reference toFIGS.1A-16. However, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes only and should not be construed as limiting.

FIG.1Aillustrates an example pulse-width modulation (PWM) image sensor100capturing light112reflected from an object110. InFIG.1, the object110is depicted as an automobile, though it is appreciated that any object, or combination of objects, may be the subject of an image detection operation of a PWM image sensor100. For example, a face of a user of an electronic device within which the PWM image sensor100is disposed may be a subject as detected by the PWM image sensor100.

During an image detection operation, light112is reflected from an object110, and/or scene, and is received at a PWM image sensor100and, in particular, at one or more photodiodes of the PWM image sensor100. As described herein, the light112may be natural light (e.g., produced the Sun), external artificial light (e.g., from external bulbs), or artificial light sources associated with the PWM image sensor100(e.g., a light source of a flash-emitting element). An example light source114is depicted inFIG.1and may be omitted in some embodiments. In some cases, the light source114is a light-emitting diode and may be used as a flashlight and/or camera flash. The light source114may be operatively coupled with the PWM image sensor100. The light112may, in some implementations, be light that is not visible to the human eye including, for example, infrared or ultraviolet light.

The PWM image sensor100may comprise multiple layers, such as a top wafer102and a bottom logic wafer104. The top wafer102and the bottom logic wafer104may be formed from a semiconductor material, such as silicon (Si) or gallium arsenide (GaAs), and may be bonded together in a number of manners, such as, but not limited to, direct bonding, plasma activated bonding, eutectic bonding, hybrid bonding, any combination thereof, and so on. In some cases, the top wafer102and the bottom logic wafer104are on opposite sides of a single wafer.

A charge-to-time converter (CTC) array106may be positioned on the top wafer102. As depicted inFIG.1A, the CTC array106may comprise a number of CTC pixels arranged in a regular and repeating manner, though any manner of arrangement of the CTC pixels may be used in accordance with the provided disclosure. The CTC array106may be formed on the top wafer102through the use of any potential technique or combination of techniques such as, for example, epitaxial growth, material deposition, etching processes, p- or n-type doping, soldering, and so on. Each CTC pixel of the CTC array106may comprise a photodiode, a reset gate, and a comparator. The photodiode may be any photodiode and may convert electromagnetic energy (e.g., light waves) into a current (e.g., a photocurrent). A strength of the photocurrent may depend on an intensity of incoming electromagnetic energy, such that brighter light results in a stronger photocurrent. The reset gate may comprise a switch configured to reset a CTC pixel to begin a detection period (e.g., by coupling/decoupling the CTC pixel from a voltage source). The comparator may be used to determine when a floating diffusion (FD) voltage reaches a predetermined threshold voltage, as discussed with reference toFIG.3. Each CTC pixel may additionally include one or more capacitors, which may be referenced as a sense node/FD capacitor (CFD). In some embodiments, the CTC is a MOS-PN (PN-type metal-oxide-semiconductor) hybrid device that directly converts light to time under low voltage.

A time-to-digital converter (TDC) array108may be provided on the bottom logic wafer104, which may be positioned beneath, or otherwise stacked with, the top wafer102. The top wafer102and the bottom logic wafer104may be separate wafers or may, in some cases, be opposite sides of the same wafer. The TDC array108may be coupled with the CTC array106through one or more communication pathways116such as, but not limited to, vertical transfer gates, through-silicon vias (TSVs), bond pads, and so on.

The TDC array108may comprise a number of individual TDC pixels. Each TDC pixel may comprise a static random-access memory (SRAM) that uses latching circuitry to store a bit of data. The number of latches in each SRAM may be any number depending on a desired resolution of each TDC pixel and each latch may correspond to a bit of data. In a non-limiting example, a SRAM for each individual TDC pixel comprises five latches, corresponding to five bits of data. An external counter may be additionally communicatively coupled with the TDC array108(e.g., to each TDC pixel) and a bi-directional data bus may transmit data (e.g., image data) to and from the external counter. The external counter may define a sampling rate at which a photocurrent (e.g., as generated from a photodiode) is sampled. Though an SRAM is discussed, any type of memory, such as a dynamic random-access memory (DRAM), may be used as a TDC pixel of the TDC array108.

Each TDC pixel may be coupled to a respective CTC pixel, such that the number of TDC pixels and CTC pixels are equivalent. Furthermore, each TDC pixel and CTC pixel pair may form a pixel of the PWM image sensor100and may correspond to a pixel on an image generated by the PWM image sensor100. In this way, a number of TDC/CTC pixel pairs may correspond to a maximum image resolution capable of being produced by the PWM image sensor100. In a non-limiting example, if a potential maximum resolution of an image produced by the PWM image sensor100is 1792×828, there may be 1,483,776 TDC/CTC pixel pairs provided for the PWM image sensor100. Though a particular example is provided, the number of TDC/CTC pixel pairs is not limited to any particular value and any number may be provided. As the TDC pixels and CTC pixels are arranged in a Z-direction, by stacking a bottom logic wafer104and a top wafer102, a size of the TDC/CTC pixel pair may be reduced in an XY-plane.

FIG.1Bdepicts an example representation of a PWM image sensor100comprising a CTC array106stacked with a TDC array108. As depicted inFIG.1B, the CTC array106may be provided on a top wafer102. The CTC array106may generate a signal (e.g., a WRITE signal) and may transmit the signal to the TDC array108. The TDC array108may be provided on a bottom logic wafer104. As depicted inFIG.1A, the CTC array106and the TDC array108may be provided in a stacked arrangement.

FIG.1Cdepicts an example CTC circuit106in accordance with aspects of the provided disclosure. It is noted that the CTC circuit106is merely one example of a CTC circuit and any number of arrangements may be provided in accordance with the associated disclosure.

The CTC circuit106may include a comparator107, a photodiode109, a floating diffusion capacitor111, and a reset gate113. An operation of the CTC circuit106may begin in response to a RESET signal being applied to the reset gate113. As depicted inFIG.1C, a pixel voltage VPXmay be provided and may be applied to the CTC circuit106when the reset gate113is closed and may be prevented from reaching components of the CTC circuit106when opened.

A photodiode109may additionally be provided and may use light (e.g., photons) to generate an associated photocurrent (e.g., a current). A strength of the photocurrent may depend on an intensity of the light as detected by the photodiode109. The generated photocurrent may discharge the floating diffusion capacitor (CFD)111to bias the input of the comparator107. The comparator107may compare the incoming number of electrons with a threshold number of electrons. The threshold number of electrons may be established by inputting a threshold voltage Vth to the comparator107. The threshold voltage Vth may be controllable and may be changeable (e.g., by a controller) to establish different triggering thresholds. Once a threshold number of electrons is reached at the comparator107, a WRITE signal may be generated (e.g., to a TDC circuit).

FIG.2depicts example electronic components of a pixel200of a PWM image sensor. Particularly, inFIG.2a potential operation of a pixel200is discussed with additional reference to the timing diagram depicted inFIG.3. While an operation with reference to one pixel is discussed, it is appreciated than a large number of individual pixels may be used in a PWM image sensor, such as in an array, as discussed with respect toFIG.1A. Operations of different pixels may be identical or may vary depending on a photocurrent produced by a photodiode and/or on other potential settings, either analog or digital, of the PWM image sensor.

As discussed with respect toFIG.1A, a pixel200of a PWM image sensor may comprise a CTC/TDC pixel pair. In particular, the charge-to-time converter (CTC)206may be positioned on a first wafer (e.g., a top wafer102) and the time-to-digital converter (TDC)208may be positioned on a second wafer (e.g., a bottom logic wafer104) stacked with the first wafer. In some cases, the CTC206and the TDC208are positioned in alternate arrangements, such as on opposite sides of the same wafer. Additionally or alternatively, it is noted that the particular arrangement of the CTC206and the TDC208is not particularly limiting. ThoughFIG.1Adepicts a CTC stacked on top of a TDC, other arrangement may be used in accordance with the provided disclosure.

An operation of the pixel200will now be discussed with reference to bothFIGS.2and3. As noted above,FIG.3depicts a timing diagram of operations of a pixel of a PWM image sensor (such as the pixel200) and, for ease of description,FIGS.2and3will be discussed in tandem.

As depicted inFIG.2, a reset (RST) signal214may be applied to a CTC206. The reset signal214may initiate the beginning of a detection period and may clear, or reset, electrons stored at a sense node and/or floating diffusion (FD). With reference toFIG.3, a reset timing graph252depicts an operation of a RST signal214. As depicted, the reset signal214may be binary (e.g.,1and0and/or “on” and “off”). For example, the reset signal may be provided to the CTC206by the opening and closing of a switch, where one end of the switch is connected to the CTC206(e.g., a sense node/FD of the CTC206) and another end of the switch is connected to a fixed voltage (e.g., a pixel voltage). Before a detection period is initiated, the switch may be closed, thereby supplying the pixel voltage to the CTC206to dissipate any electric charge which may be present at the sense node/FD (e.g., by prior image-detection operations). At a beginning of a detection period, at a time230, the switch may be opened and the pixel voltage may be stopped. This may permit electrons to begin accumulation at a sense node/FD of the CTC206. After an end of the detection period, the switch may be closed, depicted inFIG.3as between a triggering time232and an end of a frame time234, thereby supplying the pixel voltage to dissipate the previous electric charge created by the accumulation of electrons. Subsequent detection periods may be initiated when the previous electric charge is completely dissipated and/or when the switch moves back to an open position, thereby stopping the pixel voltage from being applied to the sense node/FD of the CTC206.

At a time230, when the detection period begins as a result of the RST signal214, a number of electrons217amay begin to be accumulated at a sense node/FD of the CTC206, as depicted in an electron timing graph256. The electron timing graph256depicts the accumulation of electrons during the detection period. A voltage timing graph254depicts a voltage216aof the sense node/FD, which corresponds with the number of electrons217a. The number of electrons217amay increase, at a rate dependent on a light intensity/photocurrent generated by a photodiode of the CTC206, until a threshold number of electrons217bis reached. Likewise, the voltage216amay decrease until reaching a threshold voltage216b. The time when the voltage216areaches the threshold voltage216band when the number of electrons217areaches a threshold number of electrons217bmay be referred to a triggering time232.

The threshold voltage216bmay be established by supplying a preselected voltage to the CTC206as depicted inFIG.2. In particular, the threshold voltage216bmay be applied to a first input of a comparator of the CTC206. As a second input of the comparator of the CTC206may receive the voltage216a, corresponding to a number of received electrons217aaccumulated at a sense node/FD of the CTC206during a detection period, the comparator may be capable of detecting when the voltage216amatches the threshold voltage216b, thereby resulting in a triggering time232.

At the time232, corresponding to an end of the detection period, a write (WRT) signal218may be generated at the CTC206and may be transmitted to the TDC208. The WRT signal218may correspond to the accumulated number of electrons217aover a particular time period, corresponding to a brightness of reflected light212from a scene and/or object which is the subject of an image (e.g., object210). As depicted in the write signal graph258, the WRT signal218initiates after the triggering time232. The WRT signal218may correspond to initiating the process of latching a count in the TDC208. A select timing graph260depicts a read signal222being applied to the TDC208, as depicted inFIGS.2and3. Each pixel (corresponding to a CTC/TDC pair) may be read row-by-row.

A time234, signifying an end of a frame time, may correspond to a time when all detection and signal analysis processes are complete. For example, the image data of one or more pixels of a PWM image sensor may be fully processed at this stage and a subsequent process may initiate.

A data bus220may additionally be provided between the TDC208and an external counter (not depicted). The external counter may feed count data to the TDC208or may otherwise transmit and/or receive data from the TDC208. Further description concerning the count data is described with reference toFIGS.4-5.

An example operation in accordance withFIGS.2-3will now be provided. To initiate an image detection process, a reset signal214may be applied to a charge-to-time converter (CTC)206. Light212reflected off of an object210may be received by the CTC206and a photodiode of the CTC206may convert the light212to a current (e.g., a photocurrent). The current may supply electrons to a sense node/floating diffusion where the electrons are accumulated. Once the accumulated electrons reach a threshold number of electrons (corresponding to a threshold voltage216supplied to the CTC206), a write signal218corresponding to image information may be sent from the CTC206to a time-to-digital converter (TDC)208. The write signal218may latch a count in the TDC208and the count may be read by a read signal222. Any number of pixels of a PWM sensor may be read in any order (e.g., row-by-row) and image information of the object210may be generated. A bi-directional data bus220may additionally be provided between the TDC208and an external counter. The bi-directional data bus220may be bi-directional to conserve a pixel area, so as to further reduce a size of the pixel200, though a unidirectional data bus may alternatively be used.

FIGS.4-5depict graphs related to a non-linear counter, as discussed with reference to the external counter discussed with respect toFIGS.2-3.FIG.4depicts a graph400showing a relationship between a photocurrent (e.g., a current produced by a photodiode in response to receiving light) and a triggering time (e.g., a time corresponding to a length of a detection period for a particular pixel of a PWM image sensor).

As can be seen inFIG.4, as a photocurrent increases, an associated triggering time for a pixel of the PWM image sensor decreases. This occurs as a high photocurrent results in a high rate of electron accumulation at a sense node/FD of a CTC. As the electrons quickly accumulate at the sense node/FD, a triggering time may occur quickly. Conversely, a low photocurrent may result in a low rate of electron accumulation, thereby lengthening a necessary triggering time. Put simply, as the photocurrent increases the triggering time decreases, and, as the photocurrent decreases the triggering time increases. As depicted inFIG.4, this relationship is not linear, but is instead curved between the X- and Y-axes. The curved line may be defined by the equation

Ttrig=q*NTHIp⁢h,
where Ttrigis a triggering time (e.g., a length of a detection period for a particular pixel), q is an elementary charge (e.g., the charge of a proton), Nthis a threshold number of electrons required to end a detection period, and Iphis a photocurrent produced by a photodiode of a CTC. As the terms are used herein, a triggering time may refer to a time when a particular pixel is triggered and a detection period may refer to a period when the entire PWM image sensor is active.

As indicated by the graph400, a relationship between a photocurrent and a triggering time is non-linear. As such, if a linear sampling counter is used, highlights corresponding to a photocurrent at an end portion or a beginning portion of the curve may be compressed. That is, a linear counter having ticks separated by a consistent time would either compress the photocurrent sample above about 0.1 Amperes (A), as depicted inFIG.4. This would result in a poor image quality for low- and/or high-light situations (resulting in highlight compression for pixels corresponding to relative bright/dark pixels).

To avoid potential highlight compression, an aspect of the provided disclosure provides a non-linear counter505to allow for a uniform sampling of a photocurrent. In this way, a relationship between a TDC output and a photocurrent may be linear, allowing for high quality highlight capture without compression within either low- or high-photocurrent ranges. Graph500, as depicted inFIG.5, depicts this linear relationship between the TDC output and the photocurrent, Iph, in accordance with the non-linear counter505. The non-linear counter505depicts the non-linear time between successive counter ticks, where high photocurrent is sampled at a faster rate than low photocurrent in accordance with a steeper photocurrent curve in a high photocurrent range, as depicted inFIG.4.

In some cases, each tick of the non-linear counter505is separated by a time defined by the equation

t=2b*tm⁢i⁢n2b-n,
where b is a bit depth associated with a TDC, n is a counter step number, 2b−n is the TDC counter code, and tminis the counter delay needed to detect a maximum possible photocurrent, Imax, producible by a photodiode of the CTC. The Imaxvalue may be based on physical properties of the photodiode used in the CTC and/or may be established by software associated with a PWM image sensor. In some cases, tminis defined by the equation

tm⁢i⁢n=q*NTHIm⁢a⁢x,
which is similar to the equation defining the curve inFIG.4with the exception that the current is a maximum possible photocurrent producible by a photodiode in the PWM image sensor rather than a detected photocurrent. It is additionally noted that the tminvalue is changeable based on changing a threshold number of electrons (e.g., by establishing different threshold voltages applied to the CTC).

FIGS.6A-6Bdepict graphs displaying example dynamic range (DR) extension.FIG.6Adepicts a graph600acorresponding to DR extension with a constant threshold number of electrons (Nth) andFIG.6Bdepicts a graph600bcorresponding to DR extension with a variable Nth. As the term is used herein, a “triggering time” refers to a moment when a threshold number of electrons has been accumulated at a sense node/FD for a particular pixel of a PWM image sensor. The triggering time is initiated when a detection period begins and occurs when a particular pixel is triggered.

As depicted in graph600a, corresponding to a constant Nth, an integrated charge (IC) threshold602amay be set to a constant value. During a detection period, a time (t) required to reach the IC threshold602adepends on a value of a photocurrent produced by a photodiode in response to light. For example, as indicated by box ‘1’ in graph600a, a minimum photocurrent Imingenerated by a photodiode results in a triggering time Ttrig, corresponding to a time when the number of electrons accumulated at a sense node/FD reaches the IC threshold602afor a particular pixel. Similarly, a photocurrent of 4Iminproduced by the photodiode, as indicated by box ‘4’ in graph600a, results in a second triggering time below the Ttrigvalue depicted inFIG.6A, as the time required to reach the IC threshold602awould be faster due to a faster accumulation of electrons at the sense node/FD. Likewise, respective triggering times would differ based on a value of a photocurrent produced by the photodiode, such as, but not limited to, photocurrents ranging from Imm-7Imm. In this way, a particular triggering time is based on the value of an associated current and a threshold number of electrons (e.g., an IC threshold602a).

In some cases, the IC threshold602ais the same across a number of pixels in a PWM image sensor. That is, a PWM image sensor may receive different amounts of light at different pixels, thereby causing respective photodiodes associated with respective pixels to generate different amounts of photocurrent. As such, a triggering time may differ for different pixels, based on a value of an associated photocurrent.

FIG.6Bdepicts a graph600bwith a variable IC threshold602b. As depicted inFIG.6B, the variable IC threshold602bmay result in a lowered threshold number of electrons necessary to end a triggering period for a particular pixel of a PWM image sensor. In this way, smaller photocurrents (e.g., Imin/4) beneath a typical minimum photocurrent may be detectable without resulting in lengthy triggering times. For example, inFIG.6B, a photocurrent of Imin/4 is depicted. If the IC threshold were at the level depicted inFIG.6A, the IC threshold602a, the triggering time required to reach a threshold number of electrons at a sense node/FD, would grow drastically. To reduce the triggering time, the IC threshold may be lowered.

However, if the IC threshold were lowered across each pixel of a PWM image sensor, valuable image data may be lost without much benefit (as a triggering time may already be small). For example, in high light states (e.g., resulting in a photocurrent of 7Imin), a high percentage (e.g., 90+%) of electrons may be undetected if the IC threshold were set identically as it was set for low-light states (e.g., Imin/4). Additionally, the remaining electrons in high-light states may be capable of being received within a short time period, unlike low-light states.

The variable IC threshold602bmay be set manually (e.g., by a DR extension knob) or may be set automatically in response to detected light intensity levels. For example, if an electronic device is in a dark environment (as detectable by any number of sensors), the variable IC threshold602bmay be lowered.

The variable IC threshold602bmay differ across different pixels of a PWM image sensor. For example, pixels under high-light conditions (resulting in a high photocurrent) may have a relatively high IC threshold and pixels under low-light conditions (resulting in a low photocurrent) may have a relatively low IC threshold. In additional or alternative embodiments, an IC threshold may gradually decrease (e.g., at a constant rate) when the IC threshold has not yet been reached. In this way, an overall detection period may have an easily predictable end-point regardless of a value of an associated photocurrent and regardless of triggering times for individual PWM pixels.

FIG.7depicts a graph700with various example TDC transfer functions. A TDC transfer function may refer to an encoding curve for an output curve based on an illuminance (measured in lux; 1×). In the graph700, depicted inFIG.7, transfer functions are depicted for a 5-bit output code (e.g., where a TDC can store 5 bits of information).

Curve702depicts a logarithmic TDC transfer function ƒ or a variable threshold number of electrons (Nth). Curve704depicts a logarithmic TDC transfer function ƒ or a constant Nth. Curve706depicts a linearizing TDC transfer function ƒ or a constant Nth. The particular transfer function used may be selected based on a specific application for which a PWM image sensor is used (e.g., depending on a light-condition of a camera, a video or picture mode, and so on). For example, curves704and706(representing TDC transfer functions) may be used for high light conditions where a minimum illuminance is above around 300 1×, though any value may be used in accordance with the provided disclosure.

The TDC transfer function may be controllable in the time domain by counter steps defined by the equation t=ƒ(n)*tmin, where tminis a counter delay as described above. The function ƒ(n) may reference any linear, logarithmic, and/or piece-wise profile. As a non-limiting example of such a profile represented by ƒ(n), a logarithmic function may be defined by the equation

f⁡(n)=Im⁢a⁢xIm⁢i⁢n*10p,
where Imaxis a maximum photocurrent produced by a photodiode and Iminis a minimum photocurrent produced by the photodiode. The value p may be defined by the equation

p=n*log10(Im⁢a⁢x/Im⁢i⁢n)2b-1,
where n is a range from 0 to 2b−1 and where b is a bit depth of the TDC (e.g., five in the example depicted inFIG.7). In accordance with the provided disclosure, any equation may represent ƒ(n) including, but not limited to, the logarithmic equation represented above, in order to define a transfer function. In the above example, the logarithmic response may reduce or eliminate fixed-pattern noise (FPN) due to threshold variation of a current source (e.g., a current source transistor).

Each of the depicted TDC transfer functions, and any other suitable transfer function, may be used in a PWM image sensor. As discussed above, the transfer function flexibility for a PWM image sensor may allow the PWM image sensor to be specifically tailored toward particular application (e.g., an application on an electronic device) needs.

FIG.8depicts a graph800depicting a signal-to-noise ratio (SNR) for a first threshold number of electrons (Nth1; defined by curve804) and a second threshold number of electrons (Nth2; defined by curve806). The dashed line802may represent a photon shot noise, Npbot, in situations where the SNR is largely equivalent to the Npbot.

The SNR may be represented by the equation

SNR=NphotNphot+Ndark+σq2,
where Npbotis the photon shot noise, caused by statistical quantum fluctuations, Ndarkis dark noise, representing thermal noise from electron movement, and σq2is read noise, representing voltage fluctuations of a PWM image sensor (e.g., during a read process). In situations where the threshold number of electrons (Nth) is much higher than potential read noise and dark noise, the SNR is largely defined by the photon shot noise. This can result in the SNR being approximately equivalent to the square root of Nth, represented by line802as the photon noise limit.

As discussed above, the curve804represents a first threshold number of electrons (Nth1) and the curve806represents a second threshold number of electrons (Nth2). With reference to Nth1, a SNR may increase to the photon noise limit802as a photocurrent Iphis increased. However, once the photon noise limit802is reached, the SNR may remain consistent as other aspects of SNR (e.g., read noise) can be ignored as the values are overshadowed by the photon shot noise. This results in a system where SNR does not increase once a threshold level has been reached. At a higher threshold level, Nth2, the SNR may be larger than the corresponding SNR for Nth1, but may nevertheless remain consistent once the photon noise limit802has been reached, regardless of an increased photocurrent Iph.

In this way, pixel saturation of any particular pixel of the PWM image sensor may be avoided and the highest detectable current (e.g., photocurrent) may be defined by the counter delay of the TDC. Further, in situations where the read noise is much less than the photon shot noise, a dynamic range of a PWM pixel may be determined by a ratio between a maximum triggering time Ttrigto a minimum triggering time tminas defined, in the case of constant Nth, by the equation

DR=20⁢log10(TDETtm⁢i⁢n)=20⁢log10(2b),
where b is a bit depth of the TDC.

FIGS.9A-9Bdepict graphs related to SNR shaping through the use of Nthcontrol. As discussed with respect toFIG.8, above, a SNR value may be approximated as the square root of Nth, assuming that a photon shot noise dominates the overall noise of a SNR. To extend dynamic range, particular in low light conditions, a lower value of Nthmay be selected during a detection period, if a threshold number of electrons is not accumulated in a sense node/FD within a particular period of time. This may allow a faster detection period, as otherwise particular pixels may require a large amount of time to accumulate the threshold number of electrons and/or may blur an image if the image sensor is in motion relative to a scene and/or object that is the subject of an image.

With reference toFIG.9B, depicting a graph900b, a first threshold number of electrons Nth1and a second threshold number of electrons Nth2are depicted. Within a particular detection period, a photocurrent defined by line904may be expected to reach Nth1at a triggering time Ttrigwith respect to a particular pixel. However, in low-light situations, a lower photocurrent may be produced by a photodiode and a triggering time may be reached at a slower rate, as defined by line906. To avoid lengthy triggering times, the Nthvalue may be lowered. As discussed herein, Nthmay be lowered by lowering a threshold voltage applied to a CTC. A Nthvalue may additionally or alternatively be lowered in other ways, such as decreasing a capacitance of a sense node/FD capacitor or having a double conversion gain structure.

FIG.9Adepicts a reduction of a Nthvalue by reducing a voltage applied to a CTC. As depicted in graph900a, a voltage902is reduced from a Vth1value to a Vth2value after a certain time period has elapsed. As a result, the triggering time may conclude (indicated by Ttrig) when a Nth2value is reached, instead of an initial Nth1value (seeFIG.9B). By ramping down Nth1n this way, SNR may additionally be controlled, as SNR may be approximated as the square root of Nth. This may allow SNR shaping when using a PWM sensor.

FIGS.10A-10Bdepict graphs corresponding to different detection periods for a number of pixels of a PWM image sensor. As discussed above, different pixels may receive different amounts of light resulting in respective photocurrents having different values. Therefore, different pixels may reach a threshold number of electrons (Nth) at different times. In traditional image sensors, this may result in a blurry image for fast-moving objects (e.g., an object may be in a different position at the beginning of a detection period when compared to an end of the detection period).

As depicted in graph1000aofFIG.10A, motion blur may be minimized for a PWM image sensor as a majority of all pixels may reach a Nthvalue within a short period of time. In the example depicted in graph1000a, for example, 94% of all pixels are triggered within a triggering time corresponding to TDET/17. The remaining 6% of pixels (e.g., pixels where an associated photocurrent has a value of Iminas depicted by line1002) may be triggered within a triggering time corresponding to TDET. Through this control, motion blur of a resulting image may be minimized, or eliminated, particularly for pixels generating a large photocurrent. This is due to at least 94% of the pixels comprising an image reaching a triggering time within a very small window (e.g., 1/17thof an overall detection time). A sharper image, therefore, may result.

FIG.10Bdepicts a graph1000bdepicting a relationship between a TDC code (e.g., a TDC code in least significant bits (1 sb) units) and time (t). Line1004depicts an example of constant Nthwhere a threshold number of electrons remains consistent across one or more pixels and/or across a detection period. Line1006depicts an example of ramped Nthwhere a threshold number of electrons changes across one or more pixels and/or across a detection period (see, e.g.,FIGS.9A-9B). Due to the faster conversion curve of line1004(representing a constant Nthcase), a higher number of pixels may be triggered in a shorter period of time. In contrast, the slower conversion curve of line1006(representing a ramped Nthcase), may result in slower triggering times. As such, a conversion curve corresponding to the line1004may represent a sharper image, when compared to the conversion curve corresponding to line1006.

FIG.11depicts an example structure1100for TDC timing control of a PWM image sensor. For a particular PWM image sensor, a resolution of the time-to-digital conversion (e.g., by a TDC) may be given by

Δ⁢I=Im⁢a⁢x2b-1,
where Imaxis a maximum photocurrent of a photodiode and b is a bit depth of a TDC. Using the resolution, a smallest time step may additionally be given by:

Δ⁢t=q*NthIm⁢a⁢x-Δ⁢I-q*NthIm⁢a⁢x=tm⁢i⁢n2b-2,
where Nth1s a threshold number of electrons collected as a sense node/FD, q is an elementary charge (e.g., the elementary charge of a proton), and tminis a counter delay. Using the smallest time step Δt, a clock frequency Falk may be additionally given by

Fc⁢l⁢k=1Δ⁢t.

As depicted in the example structure1100, a clock frequency Fclk, as defined in the previous paragraph, may be scaled down from a maximum photocurrent, Imax, by a clock division coefficient K, referenced as divider1102. The clock frequency may be proportional to the maximum photocurrent. In cases where a maximum photocurrent is expected (e.g., for a highest amount of measurable light), K may be equivalent to the value ‘1.’

The clock frequency divided by the clock division coefficient may be applied to a clock gating1104. The clock gating1104may be used to save power, so that an associated clock circuit is not operated while not currently in use. By applying the clock frequency and clock division coefficient to the clock gating, the clock may be activated. A counter delay may additionally be provided to the clock gating1104. A counter1106(e.g., a non-linear counter as discussed with respect toFIGS.4-5) may sample a photocurrent produced by a photodiode and a comparator1108(e.g., a digital comparator) may be used to determine when a number of electrons accumulated at a sense node/FD reaches a threshold number of electrons (e.g., by a threshold voltage input into the comparator1108).

As depicted inFIG.11, the TRIGGER signal may be a non-linear synchronization signal for the gray counter1110. The counter1106, the comparator1108, and the lookup table1112may be used to generate the TRIGGER signal. That is, the lookup table1112may hold clock division values for every TDC count within a particular range (e.g., 1 to 2b−1), the comparator1108may compare the counter1106value with the present value from the lookup table1112, and the TRIGGER signal may be generated when the two values correspond. The comparator1108may additionally send a reset signal to the counter1106. The divider1102may additionally be used to affect a speed of the gray counter1110and may control an exposure or detection time of an associated PWM image sensor. In some implementations, such as in a ramped Nthmode, the lookup table1112may additionally hold Nthvalues for every TDC count.

FIG.11merely discusses one manner of TDC timing control, and any number of TDC timing control methods may be used in accordance with a PWM image sensor of the provided disclosure. A number of such examples are provided inFIGS.12-15B.

FIG.12depicts an example method1200of exposure control where an initial Nthvalue is predetermined by a SNR expectation andFIG.13depicts an example method1300of exposure control where a maximum value of Nth(e.g., Nmax) is used.

As depicted at operation1202, initial values for a detection period, TDET, a threshold number of electrons, Nth, and a clock frequency, Folk, may be used as a starting point for an exposure control. These initial values may correspond to an expected SNR and/or by a range of photocurrent expected in an object/scene which is the subject of an image captured by a PWM image sensor.

At operation1204, a value for the clock division coefficient K may be calculated. As discussed above with respect toFIG.11, the clock division coefficient may be used as a scaling factor to scale the clock frequency down from a maximum level (e.g., a level corresponding to a maximum photocurrent). As depicted at operation1204, the value for K may be calculated from the equation

K=ROUND⁢(Fc⁢l⁢k⁢q⁢Nt⁢h(2b-2)⁢Imax),
where the function ROUND rounds the calculated value to the nearest whole number. The ROUND function may additionally or alternatively round up or round down.

At operation1206, a new clock frequency is determined by dividing the original clock frequency (e.g., as indicated at operation1202) by the value for K as determined at operation1204. At operation1208, a determination is made using the new clock frequency. The determination is whether a detection period, divided by 2 to a power corresponding to a bit depth of a TDC, is greater than or equal to the expression

2b⁢(2b-2)⁢KFc⁢l⁢k⁡(n).
In other words, a determination is made as to whether a conversion time is less than or equal to a detection period.

If the operation at1208is answered in the negative (“NO”), then the detection period may be changed to the conversion time as indicated at operation1210. If the operation at1208is answered in the positive (“YES”), then the detection period may remain the same. In the case of operation1210, the low light detection time may be extended to reduce a frame rate of a PWM pixel. At operation1212, values for K, tmin, and/or TDETmay be determined and/or stored (e.g., in a lookup table). As described herein, the method1200depicted inFIG.12generally corresponds to a one shot scene estimate in a logarithmic mode (e.g., a logarithmic transfer function).

FIG.13depicts an alternate method1300of exposure control. As indicated at operation1302, a maximum ‘N’ value (e.g., a threshold number of electrons accumulated at a sense node/FD) may be used as a starting point for exposure control. In contrast, the method1200depicted inFIG.12uses a predetermined Nthvalue using a SNR expectation (e.g., based on a one shot estimate).

At operation1302, initial values for a detection period (TDET), a clock frequency (Fclk), a maximum number of accumulated electrons (Nmax), and a minimum number of accumulated electrons (Nmin) may be obtained. At operation1304, a clock frequency division coefficient K may be calculated. The equation depicted at operation1304may be the same as that depicted in operation1204, with reference toFIG.12, with the exception that a maximum threshold number of electrons, Nmax, value is used instead of a threshold number of electrons Nthas based on a one shot estimate.

At operation1306, the calculated clock frequency division coefficient K may be used to update a clock frequency value, as indicated by Fclk(n). In this way, a clock frequency coefficient corresponding to a maximum threshold level of electrons may be calculated. In some cases, the value for ‘K’ at this initial step is equal to ‘1.’

At operation1308, a determination is made as to whether a conversion time is less than or equal to a detection period. If the conversion time is not less than or equal to the detection period (e.g., “NO”), then the method may proceed to operation1310where it is determined whether a Nmaxvalue is greater than a Nminvalue. If Nmaxis greater than Nmin(“YES” at operation1310), the Nmaxvalue may be reduced by ‘1’ and may be used as a new Nmaxvalue at operation1302and onwards. In this way, exposure control may be performed incrementally.

If, at operation1310, the Nmaxvalue is less than the Nminvalue, then the method1300may continue (via “NO”) to operation1312, where a detection period is calculated using the same formula as provided at operation1210inFIG.12. Values for K, tmin, and TDETmay then be obtained and/or stored at operation1314.

If, at operation1308, a detection period is greater than or equal to the conversion time, the method1300may proceed immediately to operation1314, where values for K, tmin, and TDETmay be obtained and/or stored. In this way, the method1300may be used to reduce a threshold number of electrons down to a minimum level (Nmin) before decreasing a frame rate (e.g., at operation1312). Method1300, therefore, may generally use a step-by-step approach to control a particular exposure value.

FIG.14depicts an additional method1400of exposure control in a ramped Nthmode (e.g., a mode as depicted inFIGS.9A-9B). In the ramped Nthmode, the Nmaxand Nminvalues are constant and, therefore, there is no need to change associated values in a lookup table (LUT), as depicted inFIG.13.

At operation1402, initial values for a clock frequency (Fclk), a detection period (TDET), a maximum threshold number of electrons (Nmax), and a minimum threshold number of electrons (Nmin) may be obtained. At operation1404, a clock frequency division coefficient K may be calculated using a charge q, the clock frequency, a maximum threshold number of electrons, and a maximum photocurrent, as discussed above. At operation1406, an updated clock frequency may be obtained by dividing the original clock frequency by the clock frequency division coefficient K.

At operation1408, a determination is made as to whether a detection period (TDET) is greater than or equal to a conversion time, as defined by the equation

q⁢Nmin(2b-1)Imax≤TD⁢E⁢T.
If the detection period is greater than or equal to the conversion time, the method1400may proceed to operation1412where values for K, tmin, and TDETmay be obtained. If operation1408is answered in the negative (e.g., “NO”), a frame rate may be changed by changing a detection period in accordance with the equation

TD⁢E⁢T=q⁢Nmin(2b-1)Imax.
After the detection period is changed, the method1400may proceed to operation1412, as discussed above.

FIGS.15A-15Bdepict examples of an auto-exposure process.FIG.15Adepicts a method1500of an auto-exposure process using a system-on-a-chip lookup tables (SOC LUT). At operation1502, initial values for K, tmin, and TDETare input from address ‘N’ of a LUT associated with a PWM image sensor at i=0. At operation1504, a window signal value may be obtained from the PWM image sensor and a mean brightness S may be calculated. To calculate S, the equation

S=Ip⁢h(2b-1)Imax
may be used, where Imax, is directly proportional to a TDC clock frequency and is defined as

Imax=q⁢Nmax⁢Fc⁢l⁢k(2b-2)⁢K,
where q is an elementary charge, Nmaxis a maximum threshold number of electrons received at a sense node/FD, Fclkis a clock frequency of the TDC, b is a bit depth of the TDC, Iphis the mean photocurrent, and K is a division coefficient. The equation for mean brightness S may be simplified as

S=k⁢1*L*KFc⁢l⁢k,
where k1 is a constant value.

At operation1506, a determination whether the mean brightness is less than or equal to ‘1’ is made. If the mean brightness is larger than ‘1,’ a lookup table (LUT) may be queried to obtain data for C log2S and C log2K at operation1508, which, in the embodiment depicted inFIGS.15A-15B, correspond to LUT1 and LUT3 (see LUT1550as depicted inFIG.15B). Though LUT1 and LUT3 are queried at operation1508, any LUT column may store the necessary information according to particular LUT settings.

Once values from LUT1 and LUT3 are received at operation1508, a counter step number n may be calculated, at operation1510, by the equation n=C log2S+C log2K−K2, where n is the counter step number, C is defined by the equation

C=N-1log2⁢Lmax-log2⁢Lmin,
and K2 is defined by the equation K2=C log2Lmin−C log2Fclk+C log2k1. As used herein, the values for illuminance (L) may be derived from the equation log2L=log2S−log2K+log2Fclk−log2k1 and minimum and maximum values for illuminance may be given by

Ln=(LmaxLmin)1N-1⁢Ln-1,
where Ln is the window illuminance.

At operation1512, the calculated value for n is used. For example, as depicted at operation1512, if n is less than or equal to 0, then n may be updated to equal ‘0.’ If n is greater than or equal to N, then n may be updated to equal ‘N.’ The LUT may be queried, at operation1514, to obtain table data for K, tmin, and Ttrigat LUT2, LUT4, and LUT5 (see table1550as depicted inFIG.15B), though the particular column for these values is provided merely for explanatory purposes. At operation1518, the values obtained at operation1514may be output to the PWM image sensor to automatically update exposure settings.

Operation1516may occur if a mean brightness value is less than 1 as determined at operation1506. In this situation, n may be set to equal N/2 before querying a LUT at operation1514.

At operation1520, an i value may be incremented by ‘1.’ If, at operation1522, i is less than or equal to 3, the operation may restart at operation1504. Otherwise, if i is greater than 3, the method1500may end at operation1524.

As described above,FIG.15Bdepicts an example LUT1550, including columns for mean brightness, LUT1, an address, and LUT2-LUT5. This LUT1550is provided merely for explanatory purposes and any potential LUT layout may be used in accordance with the provided disclosure.

FIG.16depicts an example LUT1600including a mean brightness S segment LUT1602and an illuminance segment LUT1604. As discussed above, a logarithmic TDC mode may allow a PWM image sensor to capture a whole scene dynamic range (e.g., high dynamic range HDR) in a single shot. In this way, average illuminance of a scene may be measured to assist in identifying an illuminance segment n in the LUT1600. In some cases, the illuminance segment n may be calculated by the equation

n=C⁢log2⁢Lmax-C⁢log2⁢Lmin=C⁢S⁢P2b-1-K⁢2,
where S is the mean measured brightness, b is a bit depth of a TDC, P is a number of stops, and C and K2 are constants.

By identifying the correct illuminance segment n, appropriate scene brightness may be reached for a particular image without certain portions appearing too dark or too bright. As further depicted inFIG.16, data may be transmitted from a PWM sensor1606to the LUT1600. Further, I2C (inter-integrated circuit) may be used to attach the LUT1600(particularly the n segment1604) to the PWM sensor1606.

Additional processes may additionally occur during any time of the example operations. For example, to provide color to an image, a Bayer filter array may be provided and associated processing electronics may determine a color of any particular pixel. Further, software features may be provided to vary an operation of a PWM image sensor. Any component, structure, filter, method, process, and so on may be used in accordance with operations of a PWM image sensor.

Other examples and implementations are within the scope and spirit of the disclosure and appended claims. For example, features implementing functions may also be physically located at various positions, including being distributed such that portions of functions are implemented at different physical locations. Also, as used herein, including in the claims, “or” as used in a list of items prefaced by “at least one of” indicates a disjunctive list such that, for example, a list of “at least one of A, B, or C” means A or B or C or AB or AC or BC or ABC (i.e., A and B and C). Further, the term “exemplary” does not mean that the described example is preferred or better than other examples.

The foregoing description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the described embodiments. However, it will be apparent to one skilled in the art that the specific details are not required in order to practice the described embodiments. Thus, the foregoing descriptions of the specific embodiments described herein are presented for purposes of illustration and description. They are not targeted to be exhaustive or to limit the embodiments to the precise forms disclosed. It will be apparent to one of ordinary skill in the art that many modifications and variations are possible in view of the above teachings.