Patent Description:
Time-of flight (ToF) imaging techniques are used in many depth mapping systems (also referred to as 3D mapping or 3D imaging). In direct ToF techniques, a light source, such as a pulsed laser, directs pulses of optical radiation toward the scene that is to be mapped, and a high-speed detector senses the time of arrival of the radiation reflected from the scene. The depth value at each pixel in the depth map is derived from the difference between the emission time of the outgoing pulse and the arrival time of the reflected radiation from the corresponding point in the scene, which is referred to as the "time of flight" of the optical pulses. The radiation pulses that are reflected back and received by the detector are also referred to as "echoes.

Single-photon avalanche diodes (SPADs) are detectors capable of capturing individual photons with very high time-of-arrival resolution, on the order of a few tens of picoseconds. They may be fabricated in dedicated semiconductor processes or in standard CMOS technologies. Arrays of SPAD sensors, fabricated on a single chip, have been used experimentally in 3D imaging cameras.

<CIT> relates to a laser radar device and method for generating laser image. The document discloses a system controller which determines whether the quality of either a distance image or a light intensity image satisfies reference quality, and, when the quality of either the distance image or the light intensity image dos not satisfy the reference quality, outputs a speed change command to lower a speed, an altitude change command to change a submarine altitude (depth), or the like to a navigation control unit, thereby changing a physical relative relation between a measurement plane and a device in question. As an alternative, the system controller changes a beam scanning rate f, a beam divergence, or the like within limits at which the amount of variation in a spatial resolution does not exceed a permissible amount.

<CIT> relates to a measuring device for optically measuring a distance to a target object includes an emitter device for emitting an optical measuring beam to the target object, a capturing device including a detection surface for detecting an optical beam returned by the target object, and an evaluation device. The detection surface has a plurality of pixels, each pixel having at least one SPAD (single photon avalanche diode) and each of the plurality of pixels is connected to the evaluation device. The emitting device and the capturing device are configured in such a manner that the optical measurement beam returned by the target object simultaneously illuminates a plurality of pixels. The evaluation device is configured in order to determine the distance between the measuring device and the target object based on the evaluation of detection signals of several pixels.

<CIT> relates to a distance-measuring apparatus to prevent saturation of a received signal of a pulse laser light and achieve highly accurate distance measurement.

Embodiments of the present invention that are described hereinbelow provide improved LiDAR systems and methods for ToF-based ranging and depth mapping.

There is therefore provided, in accordance with an embodiment of the invention, an electro-optical device, including a laser, which is configured to emit toward a scene pulses of optical radiation, and an array of single-photon detectors, which are configured to receive the optical radiation that is reflected from points in the scene and to fire, upon detecting a photon, an output signal indicative of a respective time of arrival of the detected photon, such that up to a saturation level, a number of the single-photon detectors firing in response to a pulse of the optical radiation scales roughly linearly with a power of the pulse. A controller is coupled to drive the laser to emit a sequence of pulses of the optical radiation toward each of a plurality of points in the scene and to find respective times of flight for the points responsively to output signals fired by the single-photon detectors, while controlling the power of the pulses emitted by the laser by counting the number of the single-photon detectors firing in response to each pulse, and reducing the power of a subsequent pulse in the sequence when the number is greater than a predefined threshold.

In a disclosed embodiment, the single-photon detectors are single-photon avalanche detectors (SPADs).

In some embodiments, the controller is configured to reduce the power of the subsequent pulse in the sequence by at least one predetermined attenuation factor. In one such embodiment, the controller is configured to attenuate the power of subsequent pulses in multiple steps of a predetermined ratio until the number of the detectors outputting the signals falls below the predefined threshold.

In some embodiments, the device includes a scanner configured to scan the pulses of optical radiation across the scene. Typically, the controller is configured to construct a depth map of the scene based on the respective time of flight found at the plurality of points while scanning the pulses. In one embodiment, the controller is configured to activate a respective subset of the detectors at a location in the array that receives the reflected optical radiation at each point in synchronization with scanning the pulses. Additionally or alternatively, the controller is configured to set the power of the pulses at some of the points to the reduced power that was used at a preceding point while scanning the pulses.

In a disclosed embodiment, the controller includes an adder, which is coupled to receive the signals from the detectors and to output a numerical signal indicative of the number of the detectors outputting signals. A comparator is configured to compare the numerical signal from the adder to the predetermined threshold and to generate a command signal when the signal exceeds the predetermined threshold. A pulse generator generates, in response to the command signal, an adaptive power control signal, which controls a drive current of the laser responsively to the adaptive power control signal.

There is also provided, in accordance with an embodiment of the invention, a method for sensing, which includes emitting a sequence of pulses of optical radiation toward each of a plurality of points in the scene. The optical radiation that is reflected from points in the scene is received in an array of detectors, which fire, upon detecting a photon, an output signal indicative of a respective time of arrival of the detected photon, such that up to a saturation level, a number of the single-photon detectors firing in response to a pulse of the optical radiation scales roughly linearly with a power of the pulse. Respective times of flight are found for the points responsively to output signals fired by the single-photon detectors. The power of the emitted pulses is controlled by counting the number of the single-photon detectors firing in response to each pulse, and reducing the power of a subsequent pulse in the sequence when the number is greater than a predefined threshold.

The quality of measurement of the distance to each point in a scene using a LiDAR is often compromised in practical implementations by a number of environmental, fundamental, and manufacturing challenges. An example of environmental challenges is the presence of uncorrelated background light, such as solar ambient light, in both indoor and outdoor applications, typically reaching a spectral irradiance of <NUM> W/(m<NUM>-nm). Fundamental challenges are related to losses incurred by optical signals upon reflection from the surfaces in the scene, especially due to low-reflectivity surfaces and limited optical collection aperture, as well as electronic and photon shot noises. These limitations often generate inflexible trade-off relationships that can push the designer to resort to solutions involving large optical apertures, high optical power, narrow field-of-view (FoV), bulky mechanical construction, low frame rate, and the restriction of sensors to operation in controlled environments.

LiDAR systems in uncontrolled environments can suffer from problems of dynamic range due to the varying distances of reflectivities of obj ects from which the laser pulses reflect. High-intensity laser pulses are needed for distant and low-reflectivity objects, but will cause saturation of the detectors receiving radiation reflected from nearby and high-reflectivity objects. This saturation, in turn, causes a temporal bias error in the signal output by the detectors, which indicates the time of arrival of the pulse, and thus distort the corresponding time-of-flight determination. The bias error may reach 1ns, corresponding to an error in depth mapping of <NUM>. Saturation of the detector array may also distort measurements of the reflectivity of the scene.

Embodiments of the present invention that are described herein provide an electro-optical device functioning as a LiDAR, which is configured to adjust its laser power to prevent this sort of temporal bias error in the determination of time of arrival due to the saturation. Specifically, the laser power is adjusted based on monitoring the number of detectors responding to a given laser pulse, as an indicator of possible saturation. This adaptive adjustment of laser power can be useful in extending the dynamic range of the LiDAR, i.e., the range of distances over which the LiDAR gives reliable readings. Preventing saturation of the detector array also supports accurate readings of the scene reflectivities measured by the LiDAR.

In the disclosed embodiments, the electro-optical device comprises a laser, an array of detectors, and a controller, which is coupled to the laser and to the detector array. The controller drives the laser to emit a sequence of pulses of optical radiation towards the scene, from which the pulses are reflected to the array of detectors. Each detector detecting a pulse emits a signal to the controller, with the signal indicating the time of arrival of the pulse. In order to scan the points in the scene, the electro-optical device may also comprise a scanner, coupled to the controller.

The controller monitors the number of detectors from which it receives a signal for a given laser pulse, in order to detect a possible saturation of the detectors. As will be detailed below, the saturation of the responding detectors can result in distortion of the signal indicating the time of arrival, and this distortion introduces a bias error in the depth mapping. Thus, when the number of detectors outputting signals in response to a given pulse is greater than a predefined threshold, the controller reduces the power of subsequent pulses in the sequence in order to mitigate the saturation and thus achieve a more accurate ToF measurement.

The detection of saturation is based on the fact that the laser pulse reflected from the scene extends over an area of the detector array and typically has Gaussian or other centrally-peaked spatial and temporal distributions of irradiance. Each detector in the array will typically respond more efficiently to the first photon of the reflected radiation that reaches it from a given laser pulse due to the effects of so-called dead time. (Dead time is defined as the recovery time a single-photon detector requires after having detected a photon in order to gain back its nominal detection efficiency. ) Ideally, the arrival times of these photons will have a statistical spread corresponding to the spatial and temporal spreads of the pulse. As the laser intensity increases, the number of detectors "firing" (i.e., emitting an output signal indicating the detection of a photon) will increase; but the distribution should remain the same as long as there is no saturation. Thus, up to the saturation level, the number of firing detectors will scale roughly linearly with the peak power of the pulse.

Saturation, on the other hand, will be characterized by an excessive number of firing detectors, as well as distortion of the apparent temporal distribution (and hence of the time of arrival measurement) as multiple photons are incident on at least some of the firing detectors. Temporal distortion is caused by dead time, as the probability of detecting a photon from the trailing portion of an optical pulse depends on the probability of not having detected any photon in the leading portion of the same pulse. Statistically, as the mean number of incident photons per pulse on each detector increases, the resulting temporal distribution of detected arrival times is skewed towards the leading edge of the pulse.

Thus, the controller compares the number of firing detectors in response to each laser pulse to a certain threshold, which may be set empirically. If the comparison indicates that the detector array is not saturated, the controller derives the time-of-flight data from the signals received from the detector array, and calculates the scene depth. If, on the other hand, the comparison indicates a saturation of the detector array, the controller transmits to the laser a command to send the next pulse at a lower power. If the comparison in response to this next pulse still indicates saturation, the controller keeps attenuating the power of the laser in subsequent pulses until the detector array is no longer saturated. At this point the signals from the detector array are accepted as valid indications of time of arrival, and the scene depth is calculated using the time-of-flight data based on these signals.

For the case in which the distance to a point in the scene is unknown (for instance, for the first depth measurement or in a scene that is changing dynamically), the first pulse toward a given point of the scene is emitted at the maximum emission power of the laser in order to receive a valid signal even if the point is distant or has low reflectivity. The controller then attenuates the power for subsequent pulses as necessary when the number of firing detectors is over threshold. The power may be attenuated in steps of a fixed, predetermined ratio by, for example, halving the laser power at each step. Alternatively, the controller may dynamically change the amount by which the power is attenuated by considering, for instance, the dead time of the single-photon detectors, the spatial extent of the unsaturated region in the detector array, and/or the extent of the scene.

The above measurement sequence, in which the first pulse is emitted at the maximum emission power, may be repeated for each point of the scene. Alternatively, the final emission power directed to a given point may be used for neighboring points, with a reset to maximum emission power performed at a distance of a preset number of scan points from the given point.

In some embodiments, the detector array comprises single-photon avalanche diodes (SPADs), which can be operated effectively in this manner. The description that follows will therefore relate specifically, for the sake of concreteness and clarity, to SPAD arrays, but the principles of the present invention may similarly be applied to detectors of other types.

In order to synchronize the detection of the received pulses with the scanning of the scene, the controller may dynamically activate a subset of the SPADs in the SPAD array such that this subset encompasses the present location of the reflected laser pulse that is imaged onto the array. (The region of sensitivity of the SPAD array may be selected and scanned along with the illumination spot by appropriately setting the bias voltages of the SPADs in synchronization with the scanning of the laser beam, as described, for example, in <CIT>. ) The size of the subset may be chosen to allow for some scanning of the laser beam without the need to re-define the subset. Periodically, the controller defines a new subset of SPADs, raises the initial laser pulse again to a maximum level, and starts anew the sequence of testing for saturation and subsequent adjustment of power level of the laser.

In an environment where additional electro-optical devices emit high-power laser pulses, one of these pulses may accidentally reach the detector array of the LiDAR and saturate it. The controller of the LiDAR may interpret this saturation to be due to the emitted laser power of the LiDAR itself, and will consequently lower its laser power. In an embodiment, for avoiding such an erroneous action, the LiDAR may be equipped with the capability of making its decisions to lower the laser power based on two or three consecutively received laser pulses, rather than just one.

Although the embodiments described hereinbelow use a single threshold on the number of firing detectors in deciding whether to lower the laser pulse intensity, in an alternative embodiment, multiple thresholds can be defined for enabling several levels of response. Each threshold may be assigned a different ratio of power attenuation, and the power is attenuated by the ratio assigned to the highest exceeded threshold. Attenuating the laser power by an amount dependent on the exceeded threshold will, in the case of high level of saturation, adjust the laser power faster to an acceptable level. In the case of a low saturation level, a too-drastic attenuation of the laser power is avoided.

<FIG> is a schematic illustration of an electro-optical device <NUM> with adaptive transmission power control, in accordance with an embodiment of the invention. Electro-optical device <NUM> comprises a light source <NUM>, a receiver <NUM>, and a controller <NUM>, which is coupled to light source <NUM> and to receiver <NUM>. (The term "light" as used herein refers to optical radiation, which may be in any of the visible, infrared and ultraviolet ranges. ) Light source <NUM> comprises a laser <NUM> and a scanner <NUM>. Receiver <NUM> comprises collection optics <NUM> and a SPAD array <NUM>.

Controller <NUM> drives laser <NUM> to emit pulses of optical radiation, which are projected by scanner <NUM> as an emitted beam of pulses <NUM> and scanned, under the control of controller <NUM>, across a scene <NUM>. Scene <NUM> is shown here, for the sake of simplicity, as an abstract flat surface, but in general, the scene that is mapped has a more complex and possibly dynamic topology. A portion of emitted beam of pulses <NUM> is reflected towards receiver <NUM> as a received beam of pulses <NUM>. Received beam of pulses <NUM> is collected by collection optics <NUM> and imaged to a location <NUM> on SPAD array <NUM>.

At the beginning of the depth measurement sequence for a given point in scene <NUM>, controller <NUM> drives laser <NUM> to emit a pulse at a predefined maximum emission power. Controller <NUM>, in conjunction with driving scanner <NUM>, activates a subset of SPADs (not shown) around location <NUM> in SPAD array <NUM>, so that received beam of pulses <NUM> is completely within this subset of SPADs. Controller <NUM> receives the signals output by the activated subset of SPADs, and determines whether the number of firing SPADs in the subset is saturated or not by comparing the number of firing SPADs to a predetermined threshold.

When the number exceeds the threshold, controller <NUM> determines that the SPAD array is saturated, and commands laser <NUM> to emit the next pulse at a lower emission power. Again controller <NUM> compares the number of firing SPADs to the predetermined threshold, and adjusts the emission power of laser <NUM>, until such a level of emission power is reached that the number of firing SPADs in the subset is below the saturation threshold. At this point in the sequence of pulses, controller <NUM> accepts the signals from the subset of SPADs as valid signals for a time-of-flight measurement, and estimates the depth of the illuminated point in scene <NUM> on this basis. Controller <NUM> may instruct laser <NUM> to emit multiple pulses at this level and make multiple, successive depth measurements on this basis, at the same point or subsequent points of the scan.

As scanner <NUM> continues to scan the emitted beam of pulses <NUM> over scene <NUM>, controller <NUM> periodically resets the emission power of laser <NUM> to the maximum. For example, the laser power may be reset after location <NUM> on SPAD array <NUM> has moved by <NUM>-<NUM> SPADs. The process of power adjustment described above is then repeated.

Although controller <NUM> is shown, for the sake of simplicity, as a separate, unitary element, in practice controller <NUM> may comprise distributed processing components that are integrated with SPAD array <NUM>. The SPAD array typically comprises integrated sensing and processing circuits, as described, for example, in <CIT>, as well as in the above-mentioned <CIT>.

<FIG> is a schematic illustration of emitted beam of pulses <NUM> and received beam of pulses <NUM> (referring to the elements shown in <FIG>), in accordance with an embodiment of the invention. Received beam of pulses <NUM> is shifted from the emitted beam of pulses <NUM> by the amount of time of flight tTOF. This figure illustrates the possible effect of saturation on measured time of flight, which is mitigated by the techniques described herein.

For demonstrating the effect of the saturation, the signal received by controller <NUM> from the activated subset of SPADs in location <NUM> from a single received pulse <NUM> is shown schematically. A frame <NUM> shows two examples of signals: A signal <NUM> from a non-saturated subset of SPADs, and a signal <NUM> from a saturated subset of SPADs. Signal <NUM> is obtained when pulse <NUM> is sufficiently low, due to an appropriate match between the distance to the measured point on scene <NUM> and the emission power of the emitted pulse. Signal <NUM> is obtained when the power of the emitted pulse resulting in received pulse <NUM> is too high, due to too high an emission power of laser <NUM> for a given distance to the measured point of scene <NUM>. The narrow and shifted profile of signal <NUM> as compared to signal <NUM> is due to most or all of the activated SPADs firing in a very short time due to the leading photons of a high-irradiance received pulse <NUM>. The difference between the times of arrival determined from signals <NUM> and <NUM> is a bias error <NUM>, whose magnitude depends on the power of received pulse <NUM>. The value of bias error <NUM> may reach 1ns, which translates into a 1ns error in the determination of time of arrival, which, in turn, translates into an error of <NUM> in the determination of the depth of scene <NUM> at the measured point.

<FIG> is a block diagram that schematically illustrates SPAD array <NUM> and associated control and processing circuits, in accordance with an embodiment of the invention. The illustrated circuits process signals from SPAD array <NUM> and control laser <NUM> (<FIG>). The control and processing circuits shown in <FIG> can be considered to be components of controller <NUM> that is shown in <FIG>.

A detector chip <NUM> comprises SPAD array <NUM>, along with elements of controller <NUM> including a pipeline adder <NUM>, a (digital) comparator <NUM>, a pulse generator <NUM>, digital signal processing (DSP) circuitry <NUM>, and readout circuitry <NUM>. A laser driver chip <NUM> comprises a drive current controller <NUM>. A processor <NUM> is coupled to detector chip <NUM> (with coupling to comparator <NUM>, to pulse generator <NUM>, and to readout circuitry <NUM> explicitly shown), to laser driver chip <NUM>, and to scanner <NUM> (not shown). Within detector chip <NUM>, pipeline adder <NUM> is coupled to selectively read out SPADs <NUM> that are within the current subset <NUM> of SPAD array <NUM>, and feeds the results to comparator <NUM>, and to DSP circuitry <NUM>. Comparator <NUM> is further coupled, either via processor <NUM> or directly, to pulse generator <NUM> which, in turn, is coupled to laser drive controller <NUM> and to processor <NUM>. DSP circuitry <NUM> is further coupled to readout circuitry <NUM>, which, in turn, is coupled to a processor <NUM>.

Alternatively, processor <NUM> is coupled to laser driver current controller <NUM>, thus obviating the need for coupling pulse generator <NUM> to laser driver current controller <NUM>. In order to reduce the timing latency between comparator <NUM> and laser driver current controller <NUM>, pulse generator <NUM> can be directly coupled to laser driver current controller <NUM>. Pulse generator <NUM> provides an interface function by which detector chip <NUM> outputs a voltage or current pulse with a finite duration that can be easily detected by laser driver current controller <NUM> and processor <NUM>. The interface function of pulse generator <NUM> avoids possible problems that could result from the voltage pulses at the output of comparator <NUM> being too short for propagating outside detector chip <NUM>.

Further referring to the elements shown in <FIG>, emitted beam of pulses <NUM> is emitted, by a command from controller <NUM>, by laser <NUM>, and reflected by scene <NUM> as received beam of pulses <NUM>. Received beam of pulses <NUM> illuminates the area of SPAD array <NUM> at location <NUM>. While controlling scanner <NUM> to define the direction of emitted beam of pulses <NUM>, controller <NUM> also activates the corresponding subset <NUM> of SPAD array <NUM> around location <NUM>, and configures the inputs to pipeline adder <NUM> to be connected to SPADs <NUM> in subset <NUM>.

When a pulse of received beam of pulses <NUM> is received by SPAD array <NUM>, pipeline adder <NUM> adds the number of SPADs in subset <NUM> that have fired (not explicitly shown among SPADs <NUM>), and transmits the number as a numerical signal <NUM> to an input of comparator <NUM>. The other input to comparator <NUM> is a threshold <NUM> determined by processor <NUM>. If comparator <NUM> determines that signal <NUM> exceeds signal <NUM>, i.e., the number of firing SPADs in subset <NUM> is higher than the threshold determined by the controller (indicating saturation of subset <NUM>), the comparator issues a command signal <NUM> to pulse generator <NUM>. Pulse generator <NUM>, upon receiving signal <NUM>, issues an adaptive power control signal <NUM> to drive current controller <NUM>, which, in response, attenuates a laser drive current <NUM> by an amount determined by processor <NUM>. Once processor <NUM>, by monitoring the above signals, determines that the number of firing SPADs in subset <NUM> is not saturated and consequently no further attenuating of laser drive current <NUM> is required, it reads an output signal <NUM> from readout circuitry <NUM>, which now represents a valid time-of-flight signal.

Referring to emitted beam of pulses <NUM> and received beam of pulses <NUM> in <FIG>, a first emitted pulse 36a is emitted by laser <NUM> at its highest power, generating a first received pulse 40a. In the present example, first received pulse 40a saturates the number of firing SPADs in subset <NUM>. This causes the power of a second emitted pulse 36b to be attenuated by a predetermined attenuation factor, such as a factor of two, through the above-mentioned circuitry and signal processing, causing a second received pulse 40b to be a factor of two lower in power than first received pulse 40a. In the embodiment shown in <FIG>, second received pulse 40b still saturates the number of firing SPADs in subset <NUM>, requiring another halving of the pulse power. The resulting third received pulse 40c no longer saturates the number of firing SPADs in subset <NUM>, and signal <NUM> is accepted by processor <NUM> as a valid signal of time of flight. In the pictured example, three additional emitted pulses 36d are emitted by laser <NUM> at the same power as emitted pulse 36c, resulting in received pulses 40d, for increasing the signal-to-noise ratio (SNR) in time-of-flight signal <NUM>.

ToF measurements of high-intensity pulses showing signs of saturation may simply be discarded, as explained above, but alternatively, these saturated measurements may be incorporated into the depth map at certain points. For example, at an edge in a scene between nearby and more distant targets, reflections of high-intensity pulses from the closer target will cause saturation, but may still provide useful ToF information with regard to the more distant target. To handle this sort of situation, ToF measurements for longer distances can be accumulated following the high-intensity pulses even in cases of saturation. These high-intensity measurements can be combined with lower-intensity, non-saturated measurements of shorter distances in order to enhance edge resolution in the depth map.

Although the disclosed embodiment uses a factor of two for attenuating the power of successive emitted pulses <NUM>, other factors, as well as other sequences, for attenuating the power of emitted pulses <NUM> can be used in alternative embodiments of the invention. Alternatively, processor <NUM> may dynamically change the amount by which the power is attenuated by considering, for instance, the dead time of the SPADs, the spatial extent of the unsaturated region in subset <NUM>, and/or the extent of scene <NUM>.

Claim 1:
An electro-optical device (<NUM>), comprising:
a laser (<NUM>), which is configured to emit toward a scene (<NUM>) pulses (<NUM>) of optical radiation;
an array (<NUM>) of single-photon detectors (<NUM>), which are configured to receive the optical radiation that is reflected from points in the scene (<NUM>) and to fire, upon detecting a photon, an output signal indicative of a respective time of arrival of the detected photon, such that up to a saturation level, a number of the single-photon detectors firing in response to a pulse of the optical radiation scales roughly linearly with a power of the pulse; and
a controller (<NUM>), which is coupled to drive the laser (<NUM>) to emit a sequence of the pulses (<NUM>) of the optical radiation toward each of a plurality of points in the scene (<NUM>) and to find respective times of flight for the points responsively to output signals fired by the single-photon detectors, while controlling the power of the pulses (<NUM>) emitted by the laser (<NUM>) by counting the number of the single-photon detectors (<NUM>) firing in response to each pulse, and when the number of the single-photon detectors (<NUM>) firing in response to a given pulse is greater than a predefined threshold, reducing the power of a subsequent pulse in the sequence.