Patent Description:
A TDC provides a digital representation of a time associated with events. For example, a TDC may be used to measure a time between a first event and a second event.

An implementation of a TDC may use a counter. The counter is started when the first event occurs and it is stopped when the second event occurs. The resulting count of the counter is a representation of the time between the first event and the second event. The time may be calculated by using the resulting count and the frequency of the clock received by the counter. In this type of TDC implementation, a higher clock frequency typically results in higher time resolution.

A TDC may be used, for example, in ranging systems that use time of flight (ToF) techniques to determine distance. For example, in ToF systems, a pulse of light is emitted, e.g., with a vertical-cavity surface-emitting laser (VCSEL) and reflected off an object back to a photonic sensor, such as a single photon avalanche diode (SPAD). The time taken for the light to travel to the object and be reflected back onto the single photonic sensor may be used to determine the distance between the object and the device based on the known speed of light. In such ToF system, a TDC may be used to generate a digital representation of the time between the transmitting of the pulse of light and the receiving of the reflected pulse by the photonic sensor.

One example of such a system is described in <CIT>.

The invention is defined in independent claims <NUM>, <NUM> and <NUM>.

The making and using of the embodiments disclosed are discussed in detail below. It should be appreciated, however, that the present invention provides many applicable inventive concepts that can be embodied in a wide variety of specific contexts. The specific embodiments discussed are merely illustrative of specific ways to make and use the invention, and do not limit the scope of the invention.

The description below illustrates the various specific details to provide an in-depth understanding of several example embodiments according to the description. The embodiments may be obtained without one or more of the specific details, or with other methods, components, materials and the like. In other cases, known structures, materials or operations are not shown or described in detail so as not to obscure the different aspects of the embodiments. References to "an embodiment" in this description indicate that a particular configuration, structure or feature described in relation to the embodiment is included in at least one embodiment. Consequently, phrases such as "in one embodiment" that may appear at different points of the present description do not necessarily refer exactly to the same embodiment. Furthermore, specific formations, structures or features may be combined in any appropriate manner in one or more embodiments.

Embodiments of the present invention will be described in a specific context, a TDC that includes a circuit for histogram generation, e.g., for a ToF image sensor. Embodiments of the present invention may be used in applications other than a ToF image sensor, such as applications in which time measurements between two signal pulses, with corresponding histogram generation, is desirable. Some embodiments may be used in ToF applications including proximity sensors, light detection and ranging (LIDAR), depth profiling, autofocus for cameras, and others. Embodiments may also be used in applications that measure time between events and/or utilize time stamps of events.

In an embodiment of the present invention, a ToF image sensor includes an array of pixels arranged in N rows and M columns. Each pixel includes a SPAD and a corresponding TDC underneath the SPAD. M control circuits respectively associated with corresponding columns of the pixel array generate, in cooperation with the SPADs in the respective column, respective zoomed histograms associated with distance measurements between the ToF image sensor and a target. In some embodiments, the zoomed histograms are compact histograms that include depth information around a detected target and exclude depth information that is farther away from the target. Thus, some embodiments advantageously can achieve higher resolution depth detection without increasing the number of bins of the histogram. In some embodiments, a smaller number of bins advantageously results in lower area consumption of the ToF image sensor without sacrificing the resolution of the ToF image sensor.

<FIG> shows ToF imaging system <NUM>, according to an embodiment of the present invention. ToF imaging system <NUM> includes illumination source <NUM>, ToF image sensor <NUM>, and processor <NUM>. ToF image sensor <NUM> includes SPAD array <NUM> and TDC and histogram circuit <NUM>. In some embodiments, TDC and histogram circuit <NUM> is distributed inside SPAD array <NUM>. In other embodiments, TDC and histogram circuit <NUM> is disposed adjacent to SPAD array <NUM>.

In some embodiments, timing generation circuit <NUM>, illumination source <NUM>, and ToF image sensor <NUM> are implemented inside integrated circuit (IC) <NUM>, while processor <NUM> is separate from IC <NUM>. In other embodiments, processor <NUM> is inside IC <NUM>.

During normal operation, illumination source <NUM> emits radiation pulses <NUM> (e.g., light signals or light pulses) towards object <NUM>, e.g., at times controlled by timing generator circuit <NUM>. Reflected radiation pulses <NUM> are sensed by SPAD array <NUM>. TDC and histogram circuit <NUM> generates digital representations (e.g., in the form of a histogram) of the time between the emissions of radiation pulses <NUM> and receptions of reflected radiation pulses <NUM>. Processor <NUM> then processes the information data received from ToF image sensor <NUM>, e.g., to determine the distance to object <NUM>.

Illumination source <NUM> may be implemented in any way known in the art. For example, illumination source <NUM> may be implemented as a VCSEL. Other implementations are also possible.

Processor <NUM> may be implemented as a general purpose digital signal processor (DSP), processor or controller that includes, for example, combinatorial circuits coupled to a memory. Processor <NUM> may also be implemented as a custom application-specific integrated circuit (ASIC). Other implementations are also possible.

ToF image sensor <NUM> may be a direct ToF (DTOF) image sensor, e.g., for a mobile device. In some embodiments, ToF image sensor <NUM> has a spatial resolution of 480x360, a <NUM> ns or better precision across distance range of <NUM>, <NUM> ns, and a TDC nominal resolution of about <NUM> ps. In other embodiments, ToF image sensor may have a spatial resolution different than 480x360, such as higher resolution or a lower resolution, and may have a precision different than <NUM> ns across distance range of <NUM>, <NUM> ns, such as a precision of <NUM> ps, <NUM> ps, or higher, or a precision of <NUM> ns, <NUM> ns, or lower, across a distance higher than <NUM>, such as <NUM> or higher, or a distance lower than <NUM>, such as <NUM>, <NUM>, or lower.

ToF image sensor <NUM> is configured to generate one or more histograms with data corresponding to the times between emitted and received radiation pulses. ToF image sensor <NUM> includes SPAD array <NUM> and TDC and histogram circuit <NUM>. For example, <FIG> show diagrams illustrating top views of layouts of possible implementations of a portion of ToF imaging system <NUM> (not to scale), according to embodiments of the present invention. The description that follows focuses on the implementation shown in <FIG> but applies in a similar manner to the implementations of <FIG> and <FIG>.

As shown in <FIG>, timing generation circuit <NUM> includes PLL <NUM>, clock control logic circuit <NUM>, and measurement control circuit <NUM>. ToF image sensor <NUM> includes vertical clock tree <NUM>, clock drivers <NUM>, column control <NUM>, data readout circuit <NUM>, and SPAD array <NUM>. SPAD array <NUM> includes N rows <NUM>, where N is a positive integer greater than <NUM> such as <NUM>, <NUM>, <NUM>, <NUM>, etc. Each row <NUM> has M pixels <NUM>, where M is a positive integer greater than <NUM> such as <NUM>, <NUM>, <NUM>, <NUM>, etc. Each pixel <NUM> includes one or more SPADs, where each SPAD or group of SPADs is associated with a respective TDC. Column control <NUM> includes M control circuits <NUM> (one per column of SPAD array <NUM>). In some embodiments, each pixel <NUM> includes a respective TDC physically located inside the pixel (e.g., beneath the SPAD or group of SPADs). In some embodiments, the TDC(s) are embedded in SPAD array <NUM> (e.g., vertically spread under a column of pixels, such as shown, e.g., in <FIG>). In some embodiments, the TDC(s) (and histogram generation circuits) are located in the edge of SPAD array <NUM> (such as shown, e.g., in <FIG>).

During normal operation, PLL <NUM> generates clock CLKPLL and provides it to measurement control circuit <NUM>. Measurement control circuit <NUM> generates system clock CLKASYS (not shown) based on clock CLKPLL for use in ToF imaging system <NUM>. Measurement control circuit <NUM> also transmits a clock signal that is synchronized with the generation of radiation pulses <NUM> to clock control logic circuit <NUM>. Clock control logic circuit <NUM> generates clock CLKsample, which is synchronized with the generation of radiation pulses <NUM>, based on the clock received from measurement control circuit <NUM> and provides clock CLKsample to SPAD array <NUM> using clock trees and clock drivers to balance clock timing.

As will be explained in more detail later, each TDC associated with SPAD array <NUM> generates, in cooperation with its respective control circuit <NUM>, a zoomed histogram based on radiation pulses received by the SPAD(s) associated with the respective TDC. The zoomed histogram is generated using an n-step successive approximation approach, where n is a positive integer greater than <NUM>, such as <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, etc. Data readout circuit <NUM> is used as an interface so that processor <NUM> can read the zoomed histograms from ToF image sensor <NUM>.

In some embodiments, the zoomed histograms are transferred in parallel by the control circuits <NUM> to the readout circuit <NUM> using, e.g., a parallel bus. The zoomed histograms are read from readout circuit <NUM> by processor <NUM> using, e.g., a serial bus, such as serial peripheral bus interface (SPI) or serial high-speed interfaces such as MIPI D-PHY, M-PHY or C-PHY. Other implementations are also possible. In some embodiments, the zoomed histograms are read directly by data readout circuit <NUM> from each of pixels <NUM> and forwarded to processor <NUM>, e.g., using a serial communication interface. In some embodiments, data readout circuit <NUM> optionally includes temporary storage to store one or more of the zoomed histograms.

Column control <NUM> includes M control circuits <NUM> (one for each column). In some embodiments, each of the control circuits <NUM> may be shared among the TDCs of SPAD array <NUM> in a different manner. For example, in some embodiments, column control <NUM> includes M/<NUM> control circuits <NUM>, where each control circuit <NUM> is shared by two columns. Other implementations are also possible.

Each pixel <NUM> includes a SPAD coupled to a TDC. For example, in some embodiments, such as shown in <FIG>, the SPAD is disposed on a top layer of an IC (such as IC <NUM>), while the corresponding TDC is disposed below the corresponding SPAD in a layer between the substrate and the SPAD. Other implementations are possible. For example, in some embodiments, more than one SPAD (e.g., in the top layer) share a TDC (e.g., disposed below the corresponding group of SPADs), e.g., via an OR tree or other combinatorial logic, such as a digital adder, for example. Some embodiments, such as shown in <FIG>, each pixel <NUM> includes one or more SPADs and the corresponding TDCs and histogram generation circuits are vertically spread under a column of pixels. Some embodiments, such as shown in <FIG>, each pixel <NUM> includes one or more SPADs and the corresponding TDCs and histogram generation circuits are located in the edge of the pixel array.

SPAD array <NUM> may occupy an area as small as <NUM> by <NUM> (<NUM><NUM>) or smaller, or as big as <NUM> by <NUM> (<NUM><NUM>) or bigger. The area occupied by SPAD array <NUM> does not have to be square (e.g., <NUM> by <NUM>). Other dimensions for SPAD array <NUM> are also possible.

Portions of ToF imaging system <NUM>, such as SPAD array <NUM>, column control circuit <NUM>, PLL <NUM>, clock control logic circuit <NUM>, vertical clock tree <NUM>, clock drivers <NUM>, data readout circuit <NUM>, and measurement control circuit <NUM> may be implemented in an IC (such as IC <NUM>) having a monolithic semiconductor substrate. The specific layout locations of particular blocks shown in <FIG> are show as an example only. Other arrangements are also possible.

In some embodiments, ToF imaging system <NUM> may be implemented in a multi-chip package having more than one semiconductor substrate. Other embodiments may implement one or more portions of ToF imaging system <NUM> with discrete components.

PLL <NUM> may be implemented in any way known in the art. PLL may generate the reference clock at frequencies such as <NUM>, <NUM>, <NUM> or higher. Lower frequencies, such as <NUM> or lower may also be used. In some embodiments, PLL <NUM> may be a fractional PLL configured to generate a clock with a frequency between <NUM> and <NUM>. Other frequencies may also be used.

In an embodiment of the present invention, a histogram with a small number of bins (e.g., <NUM> bins) is generated locally (e.g., physically located in pixel <NUM>). A control circuit (e.g., control circuit <NUM>) performs a successive approximation method to incrementally zoom into a region of interest (depth of interest) by reading the local histogram and configuring the TDC and/or histogram generation circuit to focus on the region of interest.

<FIG> shows a schematic diagram of pixel <NUM>, according to embodiments of the present invention. In some embodiments, pixel <NUM> may be implemented, e.g., as pixel <NUM>. As shown in <FIG>, pixel <NUM> includes TDC and histogram circuit <NUM>, SPAD <NUM>, and buffer <NUM>. TDC and histogram circuit <NUM> includes TDC <NUM>, and histogram generation circuit <NUM>. TDC and histogram circuit <NUM> may also be referred to as zoom TDC. In some embodiments, pixel <NUM> includes SPAD <NUM>, but does not include TDC <NUM> or histogram generation circuit <NUM> (such as shown, e.g., in <FIG> and <FIG>). However, the explanation that follows applies in a similar manner to pixels that include SPAD <NUM> but do not include TDC <NUM> and/or histogram generation circuit <NUM> (such as shown, e.g., in <FIG> and <FIG>).

During normal operation, reflected radiation pulse <NUM> stimulates SPAD <NUM>. Each time SPAD <NUM> is stimulated, a signal is propagated through buffer <NUM> (e.g., buffer <NUM> pulses). TDC <NUM> receives the pulses from buffer <NUM> and determines ToF based on clock CLKsample (which is synchronized with the emission of radiation pulses <NUM>.

Histogram generation circuit <NUM> initially generates a coarse histogram of coarse bins (e.g., <NUM> bins) based on the output of TDC <NUM>. Control circuit <NUM> receives from histogram generation circuit <NUM> the coarse histogram and determines (e.g., based on a peak search), the bin(s) in which the target is located. Control circuit <NUM> then configures histogram generation circuit <NUM> to zoom into the bins of interest (e.g., around the peak found using, e.g., a peak search algorithm), thereby generating a fine (zoom) histogram with fine bins (e.g., <NUM> bins) around the peak found based on new outputs from TDC <NUM>.

In some embodiments, histogram generation circuit <NUM> comprises memory, such as an SRAM memory located inside each pixel <NUM>, configured to store the count of bins of the coarse and fine histograms. In some embodiments, the memory may be located outside pixels <NUM>, such as in an area adjacent to SPAD array <NUM> (e.g., inside column control <NUM>). In some embodiments, the same memory cells may be used first to store the coarse histogram, and subsequently to store the fine histogram (thereby erasing some or all data associated with the coarse histogram).

TDC <NUM> is configured to output a timestamp (a digital code D<NUM> indicative of time) and may be implemented in any way known in the art. For example, in some embodiments, an address indicative of time is propagated through SPAD array <NUM>, and each TDC <NUM> latches the address at the instant that the corresponding SPAD(s) are asserted, and subsequently outputs such address. In some embodiments, TDC <NUM> may be implemented as a gray-code latch with low-voltage differential signaling (LVDS) clocks. Other implementations are also possible.

<FIG> shows a schematic diagram of pixel <NUM>, according to embodiments of the present invention. In some embodiments, pixel <NUM> may be implemented, e.g., as pixel <NUM>. Pixel <NUM> operates in a similar manner as pixel <NUM>. Pixel <NUM>, however, includes SPAD cluster <NUM> instead of a single SPAD <NUM>, and includes (and/or is associated with) OR tree <NUM> coupled to TDC <NUM> instead of buffer <NUM>. Even though SPAD cluster <NUM> is shown to include <NUM> SPADs, a different number of SPADs, such as <NUM>, <NUM>, <NUM>, or more may also be used.

<FIG> shows a flow chart of embodiment method <NUM> for generating a fine, zoom, histogram, according to an embodiment of the present invention. <FIG> shows coarse histogram <NUM> and fine histogram <NUM>, according to an embodiment of the present invention. <FIG> may be understood in view of <FIG>.

During step <NUM>, histogram generation circuit <NUM> generates a coarse histogram, such as coarse histogram <NUM>, based on the output of TDC <NUM>. In this example, coarse histogram <NUM> has <NUM> coarse bins. Some embodiments may generate histograms with a different number of bins, such as <NUM> bins, <NUM> bins, <NUM>, bins, or higher, or <NUM> bins, <NUM> bins, or lower.

Each coarse bin has a coarse time window. For example, in some embodiments, each coarse bin has a time window of <NUM> ns. Other embodiments may use a different time window for each coarse bin, such as <NUM> ns.

During step <NUM>, the control circuit <NUM> associated with histogram generation circuit <NUM> receives (e.g., reads) from histogram generation circuit <NUM> the coarse histogram. In some embodiments, control circuit <NUM> reads from histogram generation circuit <NUM> the coarse histogram using a parallel interface.

During step <NUM>, control circuit <NUM> performs a peak search to determine the highest peak of the histogram. Control circuit <NUM> may use any suitable peak search algorithm, such as a linear search, for example. In the example shown in <FIG>, the peak is located in coarse bin <NUM>.

During step <NUM>, control circuit <NUM> configures histogram generation circuit <NUM> so that it zooms into bins at or near the detected peak. In the example shown in <FIG>, histogram generation circuit <NUM> is configured to zoom into bins <NUM>, <NUM>, <NUM>, and <NUM>.

During step <NUM>, histogram generation circuit <NUM> generates a fine histogram, such as fine histogram <NUM>, based on the output of TDC <NUM>. In this example, fine histogram <NUM> has <NUM> fine bins, <NUM> fine bins per coarse bin. Some embodiments may generate fine histograms with a different number of bins per coarse bin, such as <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, etc., for example.

Each fine bin has a fine time window when compared to the coarse bin. For example, for coarse bins having a time window of <NUM> ns, each fine bin may have a corresponding time window of <NUM> ns.

In some embodiments, the number of coarse bins and the number of fine bins is the same, thereby advantageously allowing for a full reuse of the underlying memory. In some embodiments, the effective resolution of the coarse bins may be doubled by using time-offset techniques, such as explained below with respect to <FIG>.

As shown, some embodiments ignore outputs of TDC <NUM> that are outside the zoomed histogram time window (in the example of <FIG>, between the time windows associated with bins <NUM>-<NUM>). Some embodiments may advantageously achieve higher precision (e.g., the precision of a histogram having <NUM> bins) using only a fraction of the bins (e.g., <NUM> bins).

As shown, histogram generation is performed locally in ToF image sensor <NUM> (e.g., between pixel <NUM> and control circuit <NUM>) instead of in processor <NUM>. By reducing the number of bins in the histogram, some embodiments advantageously achieve lower area consumption of ToF image sensor <NUM> since less memory is devoted for histogram generation and storage.

As another example, an embodiment that generates a <NUM> coarse bin histogram and a <NUM> fine bin zoom histogram may achieve the depth precision of a histogram having <NUM> bins while only transmitting to data readout circuit <NUM><NUM> bins of the histogram.

In some embodiments, SPAD <NUM> is disposed in a top layer of an IC (such as IC <NUM>) while buffer <NUM>, TDC <NUM>, and histogram generation circuit <NUM> are disposed in layers between a substrate of the IC and SPAD <NUM>. In some embodiments, circuits <NUM>, <NUM>, and <NUM> are fully underneath SPAD <NUM>. In other words, in some embodiments, SPAD <NUM> is fully on top of circuits <NUM>, <NUM>, and <NUM>. In other embodiments, circuits <NUM>, <NUM>, and <NUM> are only partially underneath SPAD <NUM>. In yet other embodiments, circuits <NUM>, <NUM>, and <NUM> are not underneath SPAD <NUM>. For example, in some embodiments, TDC <NUM> and histogram generation circuit <NUM> may be located in an area outside SPAD array <NUM>.

<FIG> shows a flow chart of embodiment method <NUM> for generating a coarse histogram, according to an embodiment of the present invention. Step <NUM> may be implemented by method <NUM>. <FIG> shows coarse histogram <NUM> and fine histogram <NUM>, according to an embodiment of the present invention. <FIG> may be understood in view of <FIG>, and <FIG>.

During step <NUM>, a first histogram <NUM> is generated, e.g., in a similar manner as in step <NUM>. In this example, histogram <NUM> has <NUM> coarse bins.

During step <NUM>, a second histogram is generated by applying an offset of half a cycle (half of the coarse step) to, e.g., CLKsample. Other than the clock offset, the second histogram is generated in a similar manner as the first histogram. The second histogram is not shown in <FIG>.

During step <NUM>, the first and second histograms (each having <NUM> bins) are combined into a <NUM> bins coarse histogram. In some embodiments, the offset technique (which may also be referred to as a dither) advantageously spreads the energy to better resolve the target energy and to overcome, at least partially, the quantization effect that may be exhibited in the coarse bins. <FIG> shows in <NUM> the <NUM> bins of the combined coarse histogram (without showing the counts).

As shown in <FIG>, the combined histogram is read during step <NUM>, and step <NUM>, <NUM>, and <NUM> are performed, e.g., as described with respect to <FIG>. Fine histogram <NUM> shows a histogram that may result from performing the steps <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>.

<FIG> and <FIG> describe a <NUM>-step successive approximation register (SAR) zoom histogram generation. Some embodiments may perform more than <NUM> steps for SAR zoom histogram generation. For example, in some embodiments, steps may be performed in the following sequence: <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. For example, <FIG> shows a flow chart of embodiment method <NUM> for generating a fine, zoom, histogram using an n-step SAR, according to an embodiment of the present invention. Method <NUM> is similar to method <NUM>. Method <NUM>, however, includes a loop using a counter (i) that is used in steps <NUM>, <NUM>, and <NUM>.

As shown, when n is equal to <NUM>, method <NUM> is similar to method <NUM> (<NUM>-step SAR zoom histogram generation). However, method <NUM> also illustrates performing a <NUM>-step SAR zoom histogram generation (when n is equal to <NUM>) or higher.

<FIG> shows histograms <NUM>, <NUM>, and <NUM> generated while performing a <NUM>-step SAR using method <NUM>, according to an embodiment of the present invention. In the example of <FIG>, each bin of histogram <NUM> has a time window of <NUM> ns per bin. Histogram <NUM> covers a depth of <NUM>. Each bin of histogram <NUM> has a time window of <NUM> ns per bin. Histogram <NUM> covers a depth of <NUM>. Each bin of histogram <NUM> has a time window of <NUM> ps per bin. Histogram <NUM> covers a depth of <NUM>.

<FIG> shows a possible implementation of a portion of pixel <NUM>, according to an embodiment of the present invention. As shown, histogram generation circuit <NUM> includes window of interest circuit <NUM>, and histogram circuit <NUM>. Histogram circuit <NUM> includes histogram storage <NUM>, accumulator circuit <NUM>, and controller <NUM>.

During normal operation, TDC <NUM> receives start event <NUM> (e.g., based on clock CLKsample) each time a radiation pulse <NUM> is emitted. TDC <NUM> also receives stop event <NUM> from SPAD <NUM> each time SPAD <NUM> receives reflected radiation pulse <NUM>. TDC <NUM> generates digital code <NUM> based on the time Δt between t<NUM> (the time of start event <NUM>) and t<NUM> (the time of stop event <NUM>).

Window of interest circuit <NUM> receives digital code <NUM> and determines whether digital code <NUM> is within the window of interest. If digital code <NUM> is outside the window of interest, digital code <NUM> is ignored. If digital code <NUM> is inside the window of interest, then digital code <NUM> is used to update histogram storage <NUM>, e.g., by controller <NUM> performing a read, update, write-back operation. For example, in some embodiments, histogram storage <NUM> stores each bin in a register. Controller <NUM> reads the bin of histogram storage <NUM> that corresponds to digital code <NUM>, increments the content read by <NUM> using adder <NUM> of accumulator circuit <NUM>, and writes back the accumulated data into the bin of histogram storage <NUM> that corresponds to digital code <NUM>.

In some embodiments, the window of interest is determined based on the content of register <NUM>. For example, in some embodiments, the register <NUM> points to the start bin of the window of interest and the window of interest has a fixed duration (e.g., <NUM> bins) starting at the location indicated by register <NUM>. In other embodiments, the duration of the window of interest may also be modified (e.g., by using another register not shown). Other implementations are also possible.

In some embodiments, histogram generation circuit <NUM> operates in a coarse mode (when processing the coarse histogram) and in a fine mode (when processing the fine histogram). In the coarse mode, histogram generation circuit <NUM> uses only the MSBs of code <NUM> to generate the coarse histogram (and ignores the LSBs). In the fine mode, histogram generation circuit uses only the LSBs of code <NUM> to generate the fine histogram (and ignores the LSBs).

In some embodiments, clock CLKsample is a single clock signal. In some embodiments, clock CLKsample includes a plurality of clock signals in a multi-phase manner or with a multi-bit code. TDC <NUM> uses the plurality of clock signals to determine code <NUM>.

Controller <NUM> may be implemented, e.g., as an asynchronous state machine. In some embodiments, controller <NUM> may also perform one or more operations of window of interest circuit <NUM>. In some embodiments, controller <NUM> may be implemented as a controller or processor that includes, for example, combinatorial circuits coupled to a memory. Other implementations are also possible.

Histogram storage <NUM> may be implemented with, e.g., volatile or non-volatile memory. For example, histogram storage <NUM> may be implemented with registers implemented with, e.g., D-flip-flops, static random-access memory (SRAM), latch-based memory (latch cells), and/or ripple counters (e.g., made with D-flip-flops). Other implementations are also possible.

<FIG> shows timing diagram <NUM> of integration and readout times, according to an embodiment of the present invention.

As shown in <FIG>, during the coarse integration step <NUM>, coarse operations, which include step <NUM>, are performed in parallel for each row. During the SAR processing step <NUM>, which includes the readout of histogram storage <NUM> by control circuit <NUM>, peak search step <NUM> and the configuration of register <NUM> during step <NUM> are performed sequentially or partially sequentially (e.g., since control circuit <NUM> is shared, e.g., by a column of pixels <NUM>). During the fine integration step <NUM>, fine operations, which includes step <NUM>, are performed in parallel for each row. During the readout of fine histogram step <NUM>, processor <NUM> sequentially reads each fine histogram for each pixel <NUM> (e.g., by sequentially reading each fine histogram of a row, and then sequentially reading each fine histogram of the next row, etc.).

Some embodiments may perform portions of steps <NUM>, <NUM>, <NUM>, and <NUM>, in parallel. For example, in a column having N pixels <NUM>, a first subset of pixels subset<NUM> may include, e.g., N/<NUM> pixels in the column and a second subset of pixels subset<NUM> may include the other N/<NUM> of pixels in the column. Each subset subset<NUM> and subset<NUM> may perform operations in parallel. In some embodiments, there may be more than two subsets. In some embodiments, the number of pixels in each subset may not be equal.

<FIG> shows timing diagram <NUM> of integration and readout times of a first (subset<NUM>) and second (subset<NUM>) subset of pixels, according to an embodiment of the present invention. As shown, steps associated with subset<NUM> and subset<NUM> may be performed in parallel. When the subset<NUM> is performing the coarse integration (step <NUM>), coarse peak search (step <NUM>), and fine integration (step <NUM>), the subset<NUM> is performing the fine histogram readout (step <NUM>). Similarly, when the subset<NUM> is performing the coarse integration (step <NUM>), coarse peak search (step <NUM>), and fine integration (step <NUM>), the subset<NUM> is performing the fine histogram readout (step <NUM>).

<FIG> shows timing diagram <NUM> of integration and readout times of a first and second subset of pixels for a <NUM>-step SAR zoom histogram generation, according to an embodiment of the present invention.

<FIG> shows a schematic diagram of histogram generation circuit <NUM>, according to an embodiment of the present invention. Histogram generation circuit <NUM> may be implemented as histogram generation circuit <NUM>. Histogram generation circuit <NUM> includes asynchronous state machine <NUM>, accumulator <NUM>, histogram storage <NUM>, address decoder <NUM>, register <NUM>, and buffer <NUM>. Histogram storage <NUM> includes a plurality of registers for storing bins of the histogram.

During normal operation, address decoder <NUM> receives digital code <NUM> from TDC <NUM>. Address decoder <NUM> checks whether digital code <NUM> is within the window of interest based on the content of register <NUM>. If digital code <NUM> is outside the window of interest, address decoder <NUM> disables state machine <NUM> (e.g., by asserting disable signal <NUM>) until the next digital code <NUM> is received. The next digital code <NUM> received may be processed without delay and may not be ignored.

If digital code <NUM> is inside the window of interest, address decoder <NUM> does not disable state machine <NUM> (e.g., by deasserting disable signal <NUM> or ensuring that disable signal <NUM> is deasserted). State machine <NUM> receives digital code <NUM> from TDC <NUM> and also receives from address decoder <NUM> digital code <NUM> associated to the address of the register bin of histogram storage <NUM> that corresponds to digital code <NUM>.

State machine <NUM> performs a read/accumulate/write operation by reading the register bin of histogram storage <NUM> that corresponds to digital code <NUM> from histogram storage <NUM>, providing the read value <NUM> and digital code <NUM> to accumulator <NUM>, receiving the accumulated value (e.g., read value + <NUM>) from accumulator <NUM>, and writing back to the register bin of histogram storage <NUM> that corresponds to digital code <NUM> the accumulated value.

During the time in which state machine <NUM> is performing the read/accumulate/write operation, any new digital code <NUM> received may be ignored. For example, if performing the read/accumulate/write operation takes <NUM> ns, then during the <NUM> ns after receiving a digital code <NUM>, any other digital code <NUM> received is ignored. When asynchronous state machine <NUM> is not performing the read/accumulate/write operation, asynchronous state machine, upon receipt of a new code <NUM>, performs a read/accumulate/write operation unless asynchronous state machine <NUM> is disabled by address decoder <NUM>.

In some embodiment, if the pulse repetition rate (PRT) of the emitted radiation pulses <NUM> is short (e.g., shorter than twice the read/accumulate/write operation time), then a single reflected radiation pulse <NUM> is processed per emitted radiation pulse <NUM>. If the PRT is long (e.g., longer than twice the read/accumulate/write operation time), then multiple reflected radiation pulses <NUM> may be received and processed per emitted radiation pulse <NUM>.

In some embodiments, the PRT is selected such that it includes a blanking time (e.g., after the deadzone) to account for radiation pulses that are reflected from far objects.

State machine <NUM> may be implemented as a synchronous state machine or as an asynchronous state machine. State machine <NUM> may be implemented, e.g., using combinatorial logic, e.g., coupled to a memory.

Control circuit <NUM> is configured to read the content of the registers of histogram storage <NUM>, e.g., sequentially (e.g., during step <NUM>), via buffer <NUM>. In some embodiments, control circuit <NUM> reads the data from histogram storage <NUM> using bus <NUM> that is also connected to other pixels <NUM> in the column.

Control circuit <NUM> is also configured to write register <NUM>, e.g., based on the processing of the read registers (e.g., based on the result of a peak search of the read registers).

Control circuit <NUM> may be implemented with combinatorial logic physically located in column control <NUM> and may be shared by, e.g., N pixels (where N is the number of rows in SPAD array <NUM>). In some embodiments, control circuit <NUM> may be implemented inside each pixel <NUM>, and each control circuit <NUM> may be dedicated to each pixel <NUM>. Other implementations are also possible. For example, in some embodiments, each pair of pixels may share a control circuit <NUM>.

In some embodiments, register <NUM> points to the first register bin in which the window of interest begins. For example, during coarse integration step <NUM>, register <NUM> may point to the first register bin <NUM>. During step <NUM> (after SAR processing step <NUM>), control circuit <NUM> writes to register <NUM> a value so that register <NUM> points to the first register of the window of interest. For example, in the example shown in <FIG>, control circuit <NUM> writes register <NUM> so that it points to register bin <NUM>.

In some embodiments, register <NUM> determines the number of bins included in the window of interest. In such embodiments, the content of register <NUM> may also be used to determine whether code <NUM> is inside the window of interest. For example, during coarse integration step <NUM>, the content of register <NUM> is such that the window of interest covers all the coarse bins. During step <NUM>, the content of register <NUM> is indicative of the number of bins for the fine integration (e.g., covering <NUM> coarse bins instead of all coarse bins). The same process can be extended for n-step SAR histogram generations, where n is higher than <NUM>.

<FIG> shows a schematic diagram of control circuit <NUM>, according to an embodiment of the present invention. Control circuit <NUM> may be implemented as control circuit <NUM>. Control circuit <NUM> includes histogram storage <NUM>, selection circuits <NUM> and <NUM>, filter <NUM>, peak detector <NUM>, look-up-table (LUT) <NUM>, and buffers <NUM> and <NUM>. Control circuit <NUM> has a coarse mode of operation and a fine mode of operation. Control circuit may operate in the coarse mode of operation during step <NUM>. Control circuit <NUM> may operate in the fine mode of operation during step <NUM>. For embodiments having more than <NUM>-step SAR histogram generation, control circuit <NUM> may operate in the coarse mode except for the last step after the fine integration step (step <NUM>), in which control circuit <NUM> operates in the fine mode of operation.

During the coarse mode of operation, control circuit <NUM> receives histogram data, e.g., sequentially, from, e.g., each pixel <NUM> in a column of pixels via bus <NUM>. For each pixel <NUM>, control circuit <NUM> receives, e.g., sequentially, data from each register bin of histogram storage <NUM> and stores it in histogram storage <NUM>. In some embodiments, storing the histogram data into histogram storage <NUM> may be skipped and the subsequent steps may be performed directly upon receipt of the histogram data from histogram storage <NUM>. Such embodiments may be implemented, e.g., without histogram storage <NUM>.

Selection circuit <NUM> (e.g., a multiplexer) provides, e.g., sequentially, each received bin data to peak detector <NUM> (e.g., via one or more circuits or directly).

In some embodiments, a low pass filter (LPF) <NUM> to filter the bin data before performing a peak search by peak detector <NUM>. For example, in some embodiments, a <NUM>-tap FIR filter is used to smooth the histogram. Other implementations are also possible. For example, some embodiments may be implemented without filter <NUM>, or with a filter of a different type.

In some embodiments, selection circuit <NUM> (e.g., a multiplexer) is used to select between a filtered version of the bin data (e.g., bin data<NUM> is a filtered version of bin data<NUM>) or a non-filtered version of the bin data (e.g., bin data<NUM> is equal to bin data<NUM>) to be used by peak detector <NUM> to perform the peak search. Some embodiments may be implemented without selection circuit <NUM>, such as by having LPF <NUM> directly connected to peak detector <NUM> or by having the output of selection circuit <NUM> directly connected to peak detector <NUM>.

Peak detector <NUM> performs a peak search of the received bin data to identify the index of the bin that has the highest accumulated value. For example, in some embodiments, a linear search is performed for the L bins, and the highest accumulated value is provided to LUT <NUM>. In other embodiments, a peak is only detected if the highest value is higher than a predetermined threshold. If the bin with the highest value is smaller than the predetermined threshold, then no peak is detected (e.g., no target is detected) and step <NUM> is repeated after step <NUM> (instead of step <NUM>).

If a peak is detected by peak detector <NUM>, LUT <NUM> receives the bin index for the peak bin (for the bin with the highest count). LUT then generates and writes a value into register <NUM> (e.g., via buffer <NUM>) so that register <NUM> points to the beginning of a desired window of interest based on the index of the peak bin. For example, in the example of <FIG>, the peak bin is bin <NUM>, and the register <NUM> is written such that register <NUM> points to bin <NUM> as the beginning of the window of interest.

During the fine mode of operation, control circuit <NUM> receives histogram data, e.g., sequentially, from, e.g., each pixel <NUM> in a column of pixels via bus <NUM>. For each pixel <NUM>, control circuit <NUM> receives, e.g., sequentially, data from each register bin of histogram storage <NUM> and forwards it to readout circuit <NUM> (e.g., sequentially) via, e.g., circuits <NUM>, and <NUM> during step <NUM>.

<FIG> shows a flow chart of embodiment method <NUM> for generating the peak bin index by peak detector <NUM>, according to an embodiment of the present invention.

During step <NUM>, peak detector <NUM> receives bin data2, and variables i, maxCount, and maxIdx are initialized. Variable i is used to iterate through each count of each bin of bin data<NUM>. Variable maxCount is used to store the maximum count (the count of the peak). The variable maxIdx is used to store the index of the bin having the maximum count.

During step <NUM>, variable is initialized to <NUM> (e.g., to point to the first bin in bin data<NUM>), variable maxCount is initialized to <NUM> (e.g., to indicate that the no bin has exhibited a peak), and variable maxIdx is initialized to-<NUM> (e.g., to indicate that the no bin has exhibited a peak). In some embodiments, different values may be used to initialized variables i, maxCount, and maxIdx.

During step <NUM>, the count of the bin located in position i (Bin[i]) is compared with the count stored in maxCount. If the Bin[i] is higher than maxCount, then maxCount is updated to be equal to Bin[i] and maxIdx is updated to be equal to i (during step <NUM>). If not, variables maxCount and maxIdx are not updated.

During step <NUM>, variable i is incremented to iterate through the next bin in bin data<NUM>. During step <NUM>, if all bins of bin data<NUM> have been iterated through (if i is not less than total number of histogram bins L), then the peak index is set to maxIdx during step <NUM>. If there are still bins to be iterated through in bin data2 (if i is less than total number of histogram bins L), then step <NUM> is performed.

<FIG> shows a flow chart of embodiment method <NUM> for generating the peak bin index by peak detector <NUM>, according to an embodiment of the present invention. Method <NUM> is similar to method <NUM>. Method <NUM>, however, only considers counts of a bin if the count is above a predetermined threshold (represented by variable minCount).

In some embodiments, the predetermined threshold minCount may be fixed. In other embodiments, threshold minCount may be based on, e.g., the ambient noise of the histogram, shot noise of the histogram, or other parameter (and may, thus, be dynamically changed). For example, <FIG> shows a flow chart of embodiment method <NUM> for generating the peak bin index by peak detector <NUM> based on ambient noise, according to an embodiment of the present invention.

As shown in <FIG>, minCount is a function of ambient bin (or ambient counter). The ambient bin may have, for example, a count equal to the average ambient noise associated with bin data<NUM>.

In some embodiments, the ambient bin is populated at a time in which no reflected light pulses are expected (such as before emitting light pulses, e.g., before coarse integration, as shown by optional ambient integration block in <FIG>). Thus, the ambient count (of the ambient bin) may be used as an estimate for the number of SPAD activations associated with ambient light, as opposed to reflected light pulses. Such ambient count, therefore, may be used during the peak search.

In some embodiments, minCount is equal to the count of ambient bin (ambientBin). In other embodiments, minCount may be based on shot noise. For example, in some embodiments, minCount may be given by: <MAT>. Other implementations are also possible.

Claim 1:
A method implemented in a histogram generation circuit (<NUM>) and a control circuit (<NUM>) configured to cooperate with the histogram generation circuit, the method comprising:
receiving a first plurality of digital codes from a time-to-digital converter, TDC (<NUM>), associated with a respective pixel of a pixel array, the respective pixel comprising a single photon avalanche diode, SPAD (<NUM>), wherein the TDC has an input coupled to the SPAD to receive signals from the SPAD, based on radiation pulses received by the SPAD;
generating (<NUM>) a coarse histogram (<NUM>; <NUM>) from the first plurality of digital codes, the coarse histogram comprising a plurality of coarse bins that collectively correspond to a coarse histogram depth range from a lower coarse histogram depth to a higher coarse histogram depth;
detecting (<NUM>) a peak coarse bin from the plurality of coarse bins, wherein the peak coarse bin corresponds to a peak coarse bin depth range from a lower coarse peak depth to a higher coarse peak depth;
after receiving the first plurality of digital codes, receiving a second plurality of digital codes from the TDC; and
generating (<NUM>) a fine histogram (<NUM>; <NUM>) from the second plurality of digital codes based on the detected peak coarse bin, the fine histogram comprising a plurality of fine bins that collectively correspond to a fine histogram depth range from a lower fine histogram depth to a higher fine histogram depth, wherein the fine histogram depth range is narrower than the coarse histogram depth range, wherein the lower fine histogram depth is lower or equal to the lower coarse peak depth, and wherein the higher fine histogram depth is higher or equal to the higher coarse peak depth,
characterized in that generating the coarse histogram comprises:
generating (<NUM>) a first coarse histogram from a first portion of the first plurality of digital codes based on a first clock;
generating (<NUM>) a second coarse histogram from a second portion of the first plurality of digital codes based on an offset first clock, the second coarse histogram being generated by applying an offset of half a cycle; and
combining (<NUM>) the first and second coarse histograms to generate the coarse histogram.