Patent Description:
Lidar (light detection and ranging) has an important role for autonomous vehicles due to its high range and angular resolution. Receivers based on Single-Photon Avalanche Photodiodes (SPADs) are promising for automotive lidar applications due to their high sensitivity, high time resolution and high array resolution enabled by planar device structure (e.g. CMOS SPAD).

SPADs have a proven record of time-of-flight in time-correlated single photon counting (TCSPC) configuration for short range and low noise applications, such as fluorescence lifetime microscopy. Typically, SPAD pixels are connected to time-to-digital converters (TDC) for time-of-flight estimation. Due to internal and external noise, as well as the probabilistic nature of single-photon operation mode, one acquisition cycle typically comprises multiple time-of-flight measurements and a true signal is identified statistically. Adapting the use of SPADs to accommodate long range noise in automotive lidar applications requires a substantial redefinition of the architecture. A typical processing flow for lidar applications involves constructing a timestamp histogram and identifying a target reflection by identifying a peak in the histogram. Such an approach involves a trade-off between resolution and memory requirements because a smaller bin width results in a higher resolution but requires more memory capacity. Memory may be very limited in embedded automotive applications, putting a limit on resolution achieved via this route. Existing solutions may involve splitting a complete acquisition cycle into multiple steps, involving course resolution steps followed by finer resolution steps to zoom into a region of interest. In this way a lower number of bins may be used at each step. A disadvantage of this approach is that the number of measurement available is split between steps, effectively lowering the signal-to-noise ratio (SNR). As the result the sensitivity is reduced, since compressed data from any step cannot be carried over to the next step.

<CIT> discloses a SPAD array with a gated histogram construction, in which a gating generator is controlled during a first sequence of acquisition periods so as to sweep a gating interval over the acquisition periods and identify a respective detection window for a sensing element, and during a second sequence of the acquisition periods to fix the gating interval for each sensing element to coincide with the respective detection window.

According to a first aspect there is provided a method of calculating time of flight of a lidar signal, the method comprising:.

The addition of cumulative delta or phase to the range estimation method allows the implementation of wider histogram bins without this meaning a significant loss in the accuracy of the measurements, thereby allowing a substantial reduction of memory requirements without a corresponding sacrifice of sensitivity and range resolution. The method also allows for the point of maximum photon events concentration within one or more histogram bins to be determined, disregarding the bin width, making it possible to estimate the range to a target with a higher precision. Intra-bin cumulative delta or phase value can be shared between group of bins or the compete histogram for further memory reduction. Other data, such as second central moment of timestamps, may provide more useful information, such as material or relative orientation, without implying additional post-processing on the full histogram. A second central moment of an identified peak may be obtained from a sum of squares of the calculated time differences, for example by calculating a root mean square value from the cumulative total of time differences.

The cumulative total of the calculated time difference may be combined for multiple time portions over the acquisition time period, which reduces the memory requirement further. Combining the cumulative total over multiple bins does not result in a loss of accuracy because noise will on average cancel out for bins having no peak over a background noise level.

The cumulative total of the calculated time difference may be applied to the reference point of the time portion of the histogram containing the identified peak by dividing the cumulative total of the calculated time difference by the cumulative count of photons received during the time portion containing the identified peak. In other words, an average may be determined from the cumulative total that is used to estimate the time of flight.

A measure of noise may be subtracted from the cumulative count of photons received during the time portion containing the identified peak.

The reference point of each successive time portion may be at a mid-point of the time portion.

The time of flight, t, for the identified peak may be calculated from: <MAT> where tr is a time from the start of the acquisition time period to the reference point of the time portion, ∑ Δ is the cumulative total of the calculated time difference, Hi is the cumulative count of photons and HNoise is a measure of noise in the histogram.

The measure of noise, HNoise, may be calculated from an average of photon counts across the histogram other than the time portion containing the peak.

According to a second aspect there is provided a lidar transceiver comprising:.

The cumulative total of the calculated time difference may be combined for multiple time portions over the acquisition time period.

The receiver may be a single photon avalanche diode. The receiver may comprise an array of single photon avalanche diodes, the processor being configured to perform steps i) to v) for each single photon avalanche diode in the array in parallel.

Embodiments will be described, by way of example only, with reference to the drawings, in which:.

It should be noted that the Figures are diagrammatic and not drawn to scale. Relative dimensions and proportions of parts of these Figures have been shown exaggerated or reduced in size, for the sake of clarity and convenience in the drawings. The same reference signs are generally used to refer to corresponding or similar feature in modified and different embodiments.

Disclosed herein in general terms is a method and apparatus that separates peak detection and range estimation into two parallel processes. Wide-bin histogram bins can be used for target detection, thereby reducing memory requirements. Additional inter-bin cumulative delta or phase values per bin, group of bins or the complete histogram are estimated in real time for more accurate range estimation, thus achieving resolution and / or accuracy higher than would otherwise be imposed by the bin width of a histogram. The inter-bin cumulative delta or phase may be a real-time sum of timestamps relative to a reference point of each histogram bin, for example the bin middle, or in other words the phase relative to the reference. The technique leverages the fact that the total contribution from external noise (such as sunlight) and internal noise (the so called dark count rate) to inter-bin cumulative delta or phase value will tend to cancel out because the noise within a single laser shot is largely uniform. Changing noise between laser shots, or slowly shifting noise, will not result in clutter-edges (abrupt noise changes) observable in the histogram.

The characteristics of a true signal can be extracted by analysing a cumulative inter-bin delta or phase value, the bin count of corresponding bin, together with a measured noise level. For example, a peak location of a reflected signal can be estimated by dividing the final cumulative delta value by the bin count excluding noise counts, which can be approximated using available noise statistics, e.g. bin counts for other non-target bins. In a similar manner, more data, such as second central moment of timestamp, can be gathered, and may provide more information about the signal, e.g. material or relative orientation.

<FIG> illustrates an example of a lidar transceiver <NUM> similar to that disclosed in <CIT>. The lidar transceiver <NUM> includes sensor circuitry <NUM> and processing circuitry <NUM>. The sensor circuitry <NUM> produces and senses detected signals corresponding to physical objects located in an operational region relative to a location of the sensor circuitry <NUM>. The sensor circuitry <NUM> may include a SPAD array that senses optical signals and, in response, produces detected signals.

The sensor circuitry <NUM> transmits a signal in the form of a light pulse that may be reflected from physical objects within the operational region of the transmitted signals using transmitter circuitry <NUM>. Signals reflected from the target objects are detected via receiver circuitry <NUM> of the sensor circuitry <NUM>. The produced signals and received reflected signals are used to determine a time-of-flight of the signal as transmitted and reflected.

As may be appreciated, SPADs (including Geiger-mode avalanche photodiodes) are detectors capable of capturing individual photons with very high time-of-arrival resolution, typically of the order of a few tens of picoseconds. SPADs can be fabricated in dedicated semiconductor processes or in standard CMOS technologies, using known arrays of SPAD sensors, such as used in three-dimensional (3D) imaging cameras. An example of a suitable SPAD array is disclosed by <NPL>.

In the example in <FIG>, the sensor circuitry <NUM> includes TX front-end circuitry <NUM> and RX front-end circuitry <NUM>. As further described herein, the TX front-end circuitry <NUM> and RX front-end circuitry <NUM> are used to send reference signals and detect reflected signals, for example via a TX lens <NUM> and a RX lens <NUM> respectively. The reflected signals correspond to physical objects and are used to estimate a time-of-flight for the reference signal and reflected signals that is indicative of a distance of the physical object from the sensor circuitry <NUM>.

The apparatus <NUM> includes processing circuitry <NUM> in communication with the sensor circuitry <NUM>. The processing circuitry <NUM> is used to generate a histogram using the detected signals and uses the histogram to determine or estimate distances of physical objects from the sensor circuitry <NUM>. The processing circuitry <NUM> can include an array of processing circuits coupled to the SPAD array circuitry.

The processing circuitry <NUM> and sensor circuitry <NUM> may be part of an automotive lidar system. In the example of <FIG>, The processing circuitry <NUM> includes a controller circuit <NUM>, histogram circuit <NUM>, digital signal processor <NUM>, and storage circuit (e.g., random access memory, RAM) <NUM>. The controller circuit <NUM> sends reference or timing signals to the TX and RX front-end circuitries <NUM>, <NUM> to synchronize time origins for time-of-flight estimation. The controller circuit <NUM> can also control histogram parameters and/or settings, e.g., bin width settings. The TX and RX front-end circuitries <NUM>, <NUM> can include a variety of known front-end signal-receiving circuitry and related logic for processing of I/O signals related to the controller circuit <NUM>. The output of RX front-end circuitry <NUM> is a time-of-flight record of a single detection in digital format (hereinafter time or data record). Upon detection of the signal, the histogram is updated with the new data record. The histogram circuit <NUM> can be used to generate and store the histogram.

The sensor circuitry <NUM> and the processing circuitry <NUM> can operate concurrently in real-time. For example, the sensor circuitry <NUM> produces the detected signals while the processing circuitry <NUM> is constructing information for the histogram. For a vehicle lidar system, the sensor circuitry <NUM> may operate in an autonomous, or semi-autonomous, driving mode.

<FIG> illustrates an example of operation of the type of apparatus shown in <FIG>. As previously described, the physical objects, e.g. object <NUM>, can be detected using a time-of-flight technique by transmitting signals via a TX front-end circuit <NUM> and receiving reflected signals in response thereto via a RX back-end circuit <NUM>. The range or distance r is calculated based on the time-of-flight t<NUM>-t<NUM>, where t<NUM> is the time at which the light signal is emitted from the emitter <NUM> and t<NUM> is the time at which the light signal is detected at the detector <NUM>. Time stamping of a received signal can be made via an edge or peak using thresholding and time-to-digital converter (TDC) <NUM> or by use of an analog-to-digital converter (ADC) and processing (e.g., with the TDC and ADC being high speed circuits). The transmitter in <FIG> is an optical-signal generation circuitry <NUM> and the receiver is an optical-signal processing circuitry <NUM>, together forming the sensor circuitry <NUM> illustrated in <FIG>. The optical-signal generation circuitry <NUM> generates optical radiation, which when reflected off physical objects such as object <NUM>. Reflections are detected by the optical-signal processing circuitry <NUM>. Additional circuitry is configured to process signals output from the optical-signal generation circuitry <NUM> and the optical-signal processing circuitry <NUM>.

The optical-signal generation circuitry <NUM> includes a transmitter driver <NUM>, a timing engine <NUM>, and a transmitter <NUM> that form a transmitter path (also referred to as a transmit path). Emission of optical radiation by the transmitter <NUM> is controlled by the transmitter driver <NUM> and timing engine <NUM> (which may form part of the controller circuit <NUM> in the processing circuitry <NUM> of <FIG>). The transmitter <NUM> constitutes an illuminator such as a laser. Example illuminators include a light emitting diode, an edge-emitting laser, a vertical cavity surface emitting laser, and a known array thereof. Optical radiation emitted by the transmitter <NUM> travels until it is reflected by physical objects (e.g., trees, buildings, other automotive vehicles). The reflected optical radiation is sensed/received by the detector <NUM> of the optical-signal processing circuitry <NUM>, at which point time-of-flight of the emitted optical radiation is determined.

The optical-signal processing circuitry <NUM> includes an array of single-photon avalanche diodes (SPADs) <NUM> and a corresponding array of threshold (level-detection) circuits <NUM> (as well as a quenching circuit <NUM> for each SPAD and threshold circuit <NUM>). Each SPAD <NUM> is configured to detect the reflected optical radiation. Once detected by a SPAD <NUM>, the optical radiation is thresholded (e.g., compared to a quiescent point by threshold circuit <NUM>) before it is passed to additional circuitry for further processing. The sensor circuitry of the lidar system can further include a lens and a bandpass filter, which can be located in the lidar system with respect to the array of SPADs <NUM>.

In some examples the transmitter path may include a micro-electro-mechanical system (MEMS) scanner and a MEMS driver for two-dimensional (2D) steering of a laser beam or one-dimensional (1D) steering of an array of laser beams or line-laser. In other examples, the transmitter or transmitter path may include an optical phase array and drivers for scanning a laser beam.

In yet other examples, the transmitter or the transmitter path may include a VCSEL (vertical cavity surface emitting laser) array.

The transmitter or transmitter path may include a lens system for spreading the beam into a complete field-of-view (e.g. flash) or for laser beam collimation. It can be thus be appreciated that a number of different configurations can be implemented for the transmitter or transmitter path.

The additional circuitry, which is configured to process the at least one signal output form the optical-signal generation circuitry <NUM> and the optical-signal processing circuitry <NUM>, comprises a time-to-digital converter (TDC) <NUM>, a histogram circuitry (e.g., block/IC) <NUM>, and signal processing circuitry <NUM>. The TDC <NUM> is configured to receive the detected reflected optical radiation from the optical-signal processing circuitry <NUM> (which is indicative of time of incidence of a single photon) as detected by the constituent SPAD array <NUM>. Time of incidence is used to increment a count in memory of the respective incidence photon of the optical-signal processing circuitry <NUM> via the histogram circuitry <NUM> and for signal processing via the signal processing circuitry <NUM>.

Upon detection of a reflected signal, the timing circuit (e.g., the TDC <NUM> illustrated in the embodiment of <FIG> ) outputs the time difference between the event and the reference/signal, referred to as measured time-of-flight record, that can be written in a time-of-flight record register or latch. The records from time-of-flight registers or latches can be read-out and passed to the histogram block for subsequent histogram generation and storage. Although the timing circuit (e.g., the TDC <NUM>) is illustrated by <FIG> as being separate from the optical-signal processing circuitry <NUM>, the timing circuit <NUM> may form part of the optical-signal processing circuitry <NUM>.

The digital signal processor (e.g., signal processing circuitry <NUM>) can be responsible for execution of algorithms for signal detection (e.g., CFAR or "Constant False Alarm Rate" detection algorithms) and for writing detected signals to RAM. A final point cloud generated from distinct reflections can be retrieved from RAM. Processing by the digital signal processor may extend beyond signal detection including but not limited to: subsequent point-cloud filtering, segmentation, object classification, and state estimation.

A storage circuit, such as the storage circuit <NUM> depicted in <FIG>, can be configured to store respective counts of the photons that arrive in a plurality of different time bins. While obtaining the measurements, the memory that is associated with the optical-signal processing circuitry <NUM> stores respective counts of the photons that arrive at the processing circuitry in multiple different time bins, which span the detection window that the transmitter driver <NUM> and timing engine <NUM> set for the optical-signal processing circuitry <NUM>. A controller can process a histogram of the respective counts over the different bins for the sensing circuitry so as to derive and output a respective time-of-arrival value for the optical-signal processing circuitry <NUM>.

As may be appreciated, embodiments are not to be limited to use of a TDC and can include apparatuses having an analog-to-digital converter (ADC). For example, the detected reflected optical radiation is provided to the ADC via a correlated double sampling (CDS) circuit that receives the detected signal from the optical-signal processing circuitry <NUM> and outputs a sampled signal whose voltage is proportional to the detected time-of-arrival of the photon to the ADC. For general and specific information related to estimating time-of-flight and specific information on use of TDCs or ADCs, reference is made to <CIT> <CIT>.

<FIG> illustrate examples of histograms with different bin widths, each histogram having bins of equal width over the operational region corresponding to an acquisition time period. A TDC resolution of 333ps may be assumed in these examples (e.g., LSB in each case corresponding to 333ps). As is illustrated by <FIG>, in a histogram with bin width of 1xLSB (e.g., bin width 1xLSB/<NUM> ns and having <NUM> bins), a strong signal from a physical object at <NUM> results in a distinct peak <NUM>, while a weaker signal from another physical object at <NUM> is indistinguishable, as indicated by <NUM>. As illustrated by <FIG>, in a histogram with bin width of 8xLSB (e.g., bin width 8xLSB/<NUM> ns and having <NUM> bins) both peaks at <NUM> and at <NUM> are distinguishable, as illustrated respectively by <NUM> and <NUM>. A possible disadvantage (depending on the application and specific embodiment) of 8xLSB bin width is the loss of the range resolution, being <NUM> for <NUM> bins as against <NUM> for <NUM> bins. A <NUM> range resolution, depending on the use-case, may be sufficient for the other physical objects at <NUM> (e.g., as illustrated by <NUM>) but not for the physical object at <NUM> (e.g., as illustrated by <NUM>). <FIG>, in which bin width is set to 64xLSB (e.g., bin width 64xLSB/<NUM> ns and having <NUM> bins), demonstrates a different side effect at the other extreme: if bin width is set too wide, a weak signal is washed out with noise, as illustrated by <NUM> (e.g., peak at <NUM>) as compared to <NUM> (e.g., peak at <NUM>).

The trade-off between resolution for strong signals and detection reliability for weak signals is signal power and the resulting respective cumulative photon distribution. Due to the blocking nature of SPAD array circuitry, in which earlier photons block later ones due to device quenching and subsequent dead-time, different signal shapes than expected may exist in the resulting histogram. A strong signal saturates the SPAD array circuitry almost instantaneously, while a weak signal requires a longer integration time for the SPAD array circuitry to become saturated.

The effect of range resolution due to the increase of the bin widths is shown in <FIG>, in which the RMS distance error is plotted as a function of target distance for different size bins, with <FIG> showing results for bins of 1xLSB, <FIG> bins of 8xLSB and <FIG> bins of 64xLSB. In each case, the error (dotted line) is shown along with a moving mean value (solid line). The smallest errors are shown for the smallest bin size, as would be expected, together with the highest stability over the range detected. Wider bin widths of 8xLSB or 64xLSB suffer from higher errors as well as sampling errors, which can be seen in the saw-tooth shape of each plot. The accuracy also worsens as the signal becomes weaker with higher target distances, such as it is the case when a target recedes from a lidar based on a SPAD detector.

One solution to the above problems is to implement adaptive histograms, in which the bin width is adapted as a function of target distance, as disclosed in <CIT>. This can result in a reduced memory requirement by using wider bins for smaller target distances where the percentage accuracy is acceptable, while using narrower bins for higher target distances. A problem with this, however, is that the memory requirements are still substantial and the issue of using smaller bins for higher target distances where peaks may become indistinguishable, as shown in <FIG>, remains.

<FIG> illustrates an example method of calculating time of flight of a lidar signal, as carried out by a lidar transceiver having a transmitter, receiver and processor. In a first step <NUM>, a histogram is defined in terms of a number of bins, each having a defined width, the histogram covering an acquisition time period with a plurality of first bins for recording photon counts at each of a plurality of time portions over the acquisition time period and a corresponding plurality of second bins for recording a cumulative time difference between an arrival time of a photon and a reference point of the time portion. In a second step <NUM>, an acquisition time period is started and a light pulse is transmitted by the transmitter. The light pulse travels to any objects in a line of sight of the transmitter and photons are reflected back to the receiver.

If a photon is received (step <NUM>), two processes occur for each time portion. If a photon is not received, the process proceeds to step <NUM>. In a first process, at step <NUM> the histogram is updated by adding a photon count to a bin corresponding to the time portion in which the photon is received. In a second process, which may occur in parallel with the first process, a time of arrival of the photon is recorded and a time difference calculated between a reference point of the time portion and the time of arrival. This time difference is added to a cumulative total stored in a second bin corresponding to the time portion (step <NUM>). At step <NUM>, a count of light pulses is incremented and, if the count has not yet reached a predetermined number of light pulses (step <NUM>), the process starting from step <NUM> repeats. Once the predetermined number of pulses has been reached, at step <NUM> the resulting histogram is analysed to determine a peak from the photon counts in the first bins. For the peak identified in step <NUM>, at step <NUM> a time of flight is estimated based on the reference point of the time portion covered by the first bin and the cumulative time difference in the second bin.

This method of calculating time of flight solves the above-mentioned problems by taking into account the distribution of photon events within a single bin or a number of bins by considering a timestamp relative to a reference point set. The range estimation achieved considers the effect of the sum of timestamps relative to the reference point to determine the point, within the bin, of the highest concentration of photon events. This in turn corresponds to the position of the target, given the fact that the total contribution from external (sunlight) and internal noise (so called dark count rate) to inter-bin cumulative delta or phase value will tend to cancel out, as the noise within a single laser shot is largely uniform.

With improved range detection via inter-bin cumulative delta or phase value estimation, the accuracy of histogram bin-width of 8xLSB not only reaches but exceeds that of traditional histogram-based approach with bin-width of 1xLSB and 8xLSB. This is shown in <FIG>, which illustrate RMS error as a function of target distance for 8xLSB bins and 64xLSB bins respectively. For 8xLSB bins the RMS error ranges from below <NUM> up to around <NUM>, rising to around <NUM> for a target distance of <NUM>, while the sampling errors are much reduced compared with the corresponding example shown in <FIG>. For 64xLSB bins, the RMS error is consistently higher, generally between around <NUM>-<NUM> for target distances above around <NUM>, but this is lower than most of the range shown in <FIG> and again the sampling errors are much reduced.

The sampling errors are overcome because the range-detection is not based on merely histogram peak detection. At the same time, sensitivity corresponds roughly to the sensitivity of a traditional histogram-based approach of the same bin-width, i.e. higher that of 1xLSB (as a higher portion of a weak signal is integrated into a single bin, as demonstrated in <FIG>).

Depending on the position of the reference point for the time portion represented by each bin, the timestamp relative to the reference point will vary. Selection of broader bin widths allows for the obtention of greater magnitudes for the timestamps relative to the reference point within a single bin.

The memory requirements for the method are substantially lower for the same accuracy and range resolution. Comparing a traditional histogram-based approach of 1xLSB bin-width with that disclosed herein based on intra-bin cumulative delta or phase value estimation of 8xLSB bin-width, range-accuracy is similar or better. Assuming a maximum bin count of <NUM> or 8bits for 1xLSB, for a traditional approach the memory requirement would be 4096x8b = 32kb. For the method described herein, assuming an intra-bin cumulative delta value for every bin and estimation relative to the bin middle, a maximum bin count would correspond to (256x8) or 12bits for 8xLSB and a maximum delta value, being the maximum bin count multiplied by the maximum possible distance to the middle, i.e. <NUM> or 3bits, the memory requirement would be 512x(12b+15b) = 13kb. The memory requirement is thereby reduced by factor of <NUM> for a similar accuracy. The memory requirements may be further reduced by sharing intra-bin cumulative delta or phase-values between multiple pixels or a complete histogram. Also, the maximum count is capped by the number of measurement cycles and could be lower than 12bits.

Equation <NUM> below shows an example of how the inter-bin cumulative delta or phase value may be calculated. The result is then used in signal processing, together with the histogram analysis, to determine the range estimation of the target position. In this case Δ refers to the different timestamp relative to the reference point, while Hi corresponds to the photon detection count stored at the histogram bin under analysis and HNoise represents the noise level from the surroundings of the histogram bin under analysis or the complete histogram. The approximation for the noise can be achieved, for example, by computing the mean or the median of the neighbouring histogram bins on both sides of the histogram bin of interest.

In a general aspect therefore, the time of flight may be estimated from the cumulative total of the calculated time difference, ∑ Δ for the time portion of the histogram containing the identified peak. This may be done by dividing the cumulative total of the calculated time difference by the cumulative count of photons, Hi, received during the time portion. A measure of noise, HNoise, may be subtracted from the cumulative count of photons, Hi. It is important to note that the formula presented in equation <NUM>, is just one of the many principles approaches that could be considered when using inter-bin delta.

It should be appreciated that gathering more data, such as second central moment of timestamps, may provide more useful information about the signal, e.g. material or relative orientation, while keeping memory requirements lower than of a full histogram.

Claim 1:
A method of calculating time of flight of a lidar signal, the method comprising:
i) transmitting a light pulse to start (<NUM>) an acquisition time period having a plurality of successive time portions;
ii) for each successive time portion, if a photon is received (<NUM>):
iia) recording (<NUM>) a time at which the photon is received;
iib) updating (<NUM>) a histogram to record a cumulative count of photons received during the time portion;
iic) calculating (<NUM>) a time difference between the time at which the photon is received and a reference point of the time portion;
iid) updating (<NUM>) a cumulative total of the calculated time difference for the time portion;
iii) repeating steps i) to iid) for a succession of N transmitted light pulses;
iv) identifying (<NUM>) a peak in a time portion of the histogram; and
v) estimating (<NUM>) a time of flight from the cumulative total of the calculated time difference for the time portion of the histogram containing the identified peak.