PATENT DOCUMENT

Publication Number: US-11852727-B2
Application Number: US-201716769346-A
Country: US
Kind Code: B2

Title: Time-of-flight sensing using an addressable array of emitters

Abstract:
Optical sensing apparatus ( 20 ) includes an array ( 28 ) of emitters ( 50 ), which emit pulses of optical radiation at different, respective times in response to a control input applied to the array. A receiver ( 26 ) includes a plurality of detectors ( 40 ), which output signals indicative of times of arrival of photons at the detectors. Optics ( 30, 32 ) project the optical radiation from the emitters onto respective locations in a scene and image the respective locations onto corresponding pixels of the receiver. A controller ( 44 ) controls the emitters to emit the output pulses in a predefined spatio-temporal sequence, and collects and processes the signals output by corresponding pixels in synchronization with the spatio-temporal sequence so as to measure respective times of flight of the pulses to and from the respective locations in the scene.

Claims:
The invention claimed is: 
     
       1. Optical sensing apparatus, comprising:
 an array of emitters, which are configured to emit pulses of optical radiation at different, respective times in response to a control input applied to the array; 
 a receiver, comprising a plurality of pixels comprising single-photon sensors, which are configured to output signals indicative of times of arrival of photons at the detectors, and at least one time-to-digital converter (TDC), which outputs a digital value indicative of a delay between a pulse emitted by one of the emitters and receipt of a photon at a corresponding pixel of the receiver; 
 optics, which are configured to project the optical radiation from the emitters onto respective locations in a scene and to image the respective locations onto corresponding pixels of the receiver, each pixel comprising one or more of the detectors; and 
 a controller, which is coupled to control the emitters to emit the output pulses in a predefined spatio-temporal sequence, and to collect and process the signals output by corresponding pixels in synchronization with the spatio-temporal sequence so as to measure respective times of flight of the pulses to and from the respective locations in the scene. 
 
     
     
       2. The apparatus according to  claim 1 , wherein the emitters comprise vertical-cavity surface-emitting lasers (VCSELs). 
     
     
       3. The apparatus according to  claim 1 , wherein in accordance with the predefined spatio-temporal sequence, each of the emitters is driven to emit a series of the pulses, and wherein the controller is configured to accumulate, for each pixel, a respective histogram of digital values output by the at least one TDC in response to the series of the pulses, and to derive a respective time of flight from the histogram. 
     
     
       4. The apparatus according to  claim 3 , wherein the controller is configured to modify the spatio-temporal sequence by varying a number of the pulses in the series of the pulses emitted by each of the emitters. 
     
     
       5. The apparatus according to  claim 1 , wherein both the emitters and the single-photon sensors are arranged in respective matrices comprising multiple rows and multiple columns, wherein the at least one TDC comprises multiple TDCs with each TDC coupled to a different, respective row of the single-photon sensors, and
 wherein the spatio-temporal sequence is defined so that at least first and second emitters, in different, respective rows of the array emit the pulses concurrently, while no more than a single emitter in each of the rows is operative at any given time. 
 
     
     
       6. The apparatus according to  claim 1 , wherein the spatio-temporal sequence is defined so that at least first and second emitters emit respective first and second pulses with an interval between the pulses that is less than half an average of the times of flight of the pulses to and from the respective locations in the scene. 
     
     
       7. The apparatus according to  claim 6 , wherein the at least first and second emitters belong respectively to first and second groups of the emitters, such that the emitters in each of the groups are spaced apart over the array and emit the pulses simultaneously. 
     
     
       8. The apparatus according to  claim 1 , wherein the controller is configured, after measuring first times of flight in accordance with an initial spatio-temporal sequence, to modify the spatio-temporal sequence and measure second times of flight in accordance with the modified spatio-temporal sequence. 
     
     
       9. Optical sensing apparatus, comprising:
 an array of emitters, which are configured to emit pulses of optical radiation at different, respective times in response to a control input applied to the array; 
 a receiver, comprising a plurality of detectors, which are configured to output signals indicative of times of arrival of photons at the detectors; 
 optics, which are configured to project the optical radiation from the emitters onto respective locations in a scene and to image the respective locations onto corresponding pixels of the receiver, each pixel comprising one or more of the detectors; and 
 a controller, which is coupled to control the emitters to emit the output pulses in a predefined spatio-temporal sequence, and to collect and process the signals output by corresponding pixels in synchronization with the spatio-temporal sequence so as to measure respective times of flight of the pulses to and from the respective locations in the scene, 
 wherein the controller is configured, after measuring first times of flight in accordance with an initial spatio-temporal sequence, to modify the spatio-temporal sequence and measure second times of flight in accordance with the modified spatio-temporal sequence, and 
 wherein the spatio-temporal sequence is modified so as to measure the second times of flight with a resolution that is finer than the resolution of the first times of flight. 
 
     
     
       10. A method for optical sensing, comprising:
 emitting pulses of optical radiation from an array of emitters at different, respective times in a predefined spatio-temporal sequence; 
 projecting the optical radiation from the emitters onto respective locations in a scene; 
 imaging the respective locations onto corresponding pixels of a receiver comprising a matrix of single-photon sensors; 
 outputting signals from the receiver indicative of times of arrival of photons at the corresponding pixels, 
 wherein outputting the signals comprises outputting a digital value indicative of a delay between a pulse emitted by one of the emitters and receipt of a photon at a corresponding pixel of the receiver; and 
 collecting and processing the signals output by corresponding pixels in synchronization with the spatio-temporal sequence so as to measure respective times of flight of the pulses to and from the respective locations in the scene. 
 
     
     
       11. The method according to  claim 10 , wherein the emitters comprise vertical-cavity surface-emitting lasers (VCSELs), and wherein the pulses have a duration that is less than 1 ns. 
     
     
       12. The method according to  claim 10 , wherein in accordance with the predefined spatio-temporal sequence, each of the emitters is driven to emit a series of the pulses, and wherein collecting and processing the signals comprises accumulating, for each pixel, a respective histogram of digital values indicative of the delay in response to the series of the pulses, and deriving a respective time of flight from the histogram. 
     
     
       13. The method according to  claim 12 , and comprising modifying the spatio-temporal sequence by varying a number of the pulses in the series of the pulses emitted by each of the emitters. 
     
     
       14. The method according to  claim 10 , wherein both the emitters and the single-photon sensors are arranged in respective matrices comprising multiple rows and multiple columns, and wherein outputting the digital value comprises coupling a respective time-to-digital converter (TDC) to read out the digital value from each row of the single-photon sensors, and
 wherein the spatio-temporal sequence is defined so that at least first and second emitters, in different, respective rows of the array emit the pulses concurrently, while no more than a single emitter in each of the rows is operative at any given time. 
 
     
     
       15. The method according to  claim 10 , wherein the spatio-temporal sequence is defined so that at least first and second emitters emit respective first and second pulses with an interval between the pulses that is less than half an average of the times of flight of the pulses to and from the respective locations in the scene. 
     
     
       16. The method according to  claim 15 , wherein the at least first and second emitters belong respectively to first and second groups of the emitters, such that the emitters in each of the groups are spaced apart over the array and emit the pulses simultaneously. 
     
     
       17. The method according to  claim 10 , and comprising, after measuring first times of flight in accordance with an initial spatio-temporal sequence, modifying the spatio-temporal sequence and measuring second times of flight in accordance with the modified spatio-temporal sequence. 
     
     
       18. The method according to  claim 17 , wherein the spatio-temporal sequence is modified so as to measure the second times of flight with a resolution that is finer than the resolution of the first times of flight.

Description:
FIELD OF THE INVENTION 
     The present invention relates generally to range sensing, and particularly to devices and methods for depth mapping based on time-of-flight measurement. 
     BACKGROUND 
     Time-of-flight (ToF) imaging techniques are used in many depth mapping systems (also referred to as 3D mapping or 3D imaging). In direct ToF techniques, a light source, such as a pulsed laser, directs pulses of optical radiation toward the scene that is to be mapped, and a high-speed detector senses the time of arrival of the radiation reflected from the scene. The depth value at each pixel in the depth map is derived from the difference between the emission time of the outgoing pulse and the arrival time of the reflected radiation from the corresponding point in the scene, which is referred to as the “time of flight” of the optical pulses. The radiation pulses that are reflected back and received by the detector are also referred to as “echoes.” 
     Single-photon avalanche diodes (SPADs), also known as Geiger-mode avalanche photodiodes (GAPDs), are detectors capable of capturing individual photons with very high time-of-arrival resolution, on the order of a few tens of picoseconds. They may be fabricated in dedicated semiconductor processes or in standard CMOS technologies. Arrays of SPAD sensors, fabricated on a single chip, have been used experimentally in 3D imaging cameras. Charbon et al. provide a review of SPAD technologies in “SPAD-Based Sensors,” published in  TOF Range - Imaging Cameras  (Springer-Verlag, 2013). 
     SUMMARY 
     Embodiments of the present invention that are described hereinbelow provide improved apparatus and methods for direct ToF sensing. 
     There is therefore provided, in accordance with an embodiment of the invention, optical sensing apparatus, including an array of emitters, which are configured to emit pulses of optical radiation at different, respective times in response to a control input applied to the array. A receiver includes a plurality of detectors, which are configured to output signals indicative of times of arrival of photons at the detectors. Optics are configured to project the optical radiation from the emitters onto respective locations in a scene and to image the respective locations onto corresponding pixels of the receiver, each pixel including one or more of the detectors. A controller is coupled to control the emitters to emit the output pulses in a predefined spatio-temporal sequence, and to collect and process the signals output by corresponding pixels in synchronization with the spatio-temporal sequence so as to measure respective times of flight of the pulses to and from the respective locations in the scene. 
     In a disclosed embodiment, the emitters include vertical-cavity surface-emitting lasers (VCSELs). 
     Additionally or alternatively, the detectors include single-photon sensors, and the receiver includes at least one time-to-digital converter (TDC), which outputs a digital value indicative of a delay between a pulse emitted by one of the emitters and receipt of a photon at a corresponding pixel of the receiver. In some embodiments, each of the emitters is driven to emit a series of the pulses in accordance with the predefined spatio-temporal sequence, and the controller is configured to accumulate, for each pixel, a respective histogram of digital values output by the at least one TDC in response to the series of the pulses, and to derive a respective time of flight from the histogram. In one embodiment, the controller is configured to modify the spatio-temporal sequence by varying a number of the pulses in the series of the pulses emitted by each of the emitters. 
     Further additionally or alternatively, both the emitters and the single-photon sensors are arranged in respective matrices including multiple rows and multiple columns, wherein the at least one TDC includes multiple TDCs with each TDC coupled to a different, respective row of the single-photon sensors, and the spatio-temporal sequence is defined so that at least first and second emitters, in different, respective rows of the array emit the pulses concurrently, while no more than a single emitter in each of the rows is operative at any given time. 
     In further embodiments, the spatio-temporal sequence is defined so that at least first and second emitters emit respective first and second pulses with an interval between the pulses that is less than half an average of the times of flight of the pulses to and from the respective locations in the scene. In a disclosed embodiment, the at least first and second emitters belong respectively to first and second groups of the emitters, such that the emitters in each of the groups are spaced apart over the array and emit the pulses simultaneously. 
     In some embodiments, the controller is configured, after measuring first times of flight in accordance with an initial spatio-temporal sequence, to modify the spatio-temporal sequence and measure second times of flight in accordance with the modified spatio-temporal sequence. In one embodiment, the spatio-temporal sequence is modified so as to measure the second times of flight with a resolution that is finer than the resolution of the first times of flight. 
     There is also provided, in accordance with an embodiment of the invention, a method for optical sensing, which includes emitting pulses of optical radiation from an array of emitters at different, respective times in a predefined spatio-temporal sequence. The optical radiation is projected from the emitters onto respective locations in a scene. The respective locations are imaged onto corresponding pixels of a receiver, which outputs signals indicative of times of arrival of photons at the corresponding pixels. The signals output by corresponding pixels are collected and processed in synchronization with the spatio-temporal sequence so as to measure respective times of flight of the pulses to and from the respective locations in the scene. 
     The present invention will be more fully understood from the following detailed description of the embodiments thereof, taken together with the drawings in which: 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG.  1    is a schematic side view of a depth mapping device, in accordance with an embodiment of the invention; 
         FIG.  2    is a block diagram that schematically illustrates an integrated emitter array, in accordance with an embodiment of the invention; 
         FIGS.  3 A-C  are schematic frontal views of an emitter array, showing successive stages in an emission sequence, in accordance with an embodiment of the invention; 
         FIG.  4    is a plot that schematically illustrates the timing of pulses in the emission sequence of  FIGS.  3 A-C , in accordance with an embodiment of the invention; and 
         FIGS.  5 A and  5 B  are schematic representation of a scene that is mapped by a depth mapping device, illustrating different, respective operational modes of an emitter array in the device, in accordance with an embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTION OF EMBODIMENTS 
     In direct ToF depth mapping systems that are known in the art, the data acquisition rate is limited by the distance to the target that is to be mapped: The light source emits a bright pulse of radiation, and the system then waits for a time no less than the time of flight of the photons to the target and back to the receiver before the next pulse can be fired. In other words, the system waits a fixed amount of time, which corresponds to the maximum working distance, i.e., the maximum distance to a target object that could be measured by the system. (If the pulse repetition period were to be less than the time of flight, the receiver might not be able to distinguish between the echoes of successive pulses, leading to problems of aliasing in the ToF measurements.) Considering that laser sources used in such systems typically have pulse widths less than 1 ns, while time of flight (in air) grows at 6 ns per meter of distance to the target, the limitation on the pulse rate means that the light source operates at a very low duty cycle. Therefore, the light source may have to emit very intense pulses in order to achieve good resolution and signal/noise ratio with acceptable measurement throughput. 
     Embodiments of the present invention that are described herein address these limitations by using an array of emitters, which can be driven to emit pulses of optical radiation at different, respective times in response to a control input applied to the array. Arrays of this sort can be produced, for example, by integrating emitters with control circuits in a single chip. In one such embodiment, a III-V semiconductor substrate on which the emitters are fabricated is bonded to a silicon substrate on which control circuits for the emitters are fabricated. This structure makes it possible to address and fire the emitters in the array individually in substantially any desired spatio-temporal sequence. 
     In the disclosed embodiments, optics project the optical radiation from the emitters onto respective locations in a scene and image these locations onto corresponding pixels of a receiver. The receiver comprises an array of detectors, which output signals indicative of times of arrival of photons at the detectors. Each pixel comprises one or more of the detectors (i.e., each location in the scene onto which an emitter beam is projected can be imaged back onto a single detector or onto a group of neighboring detectors, whose output signals are then pooled into one pixel). Thus, each pixel detects the time of arrival of the photons transmitted from a particular, corresponding emitter in the array. 
     This arrangement makes it possible for a controller to control the emitters to emit their output pulses in a predefined spatio-temporal sequence, and to collect and process the signals output by the corresponding pixels in synchronization with the spatio-temporal sequence so as to measure respective times of flight of the pulses to and from the respective locations in the scene. The term “spatio-temporal sequence” refers to the spatial order in which the particular emitters fire their pulses and the intervals between the successive pulses, which may be fired by the same or different emitters. 
     Various spatio-temporal sequences may be defined in order to optimize performance and meet throughput and resolution requirements. For example, the spatio-temporal sequence may be defined such that multiple emitters, in different locations in the array, emit their pulses simultaneously. Additionally or alternatively, the sequence may be defined so that different emitters in different locations emit their respective pulses with an interval between the pulses that is substantially less than the expected time of flight based on the range of working distances at which measurements are to be made. For example, the interval between pulses is typically less than half the average of the times of flight of the pulses to and from the respective locations in the scene. In both cases, there is little risk of aliasing, since the resulting echoes will be captured concurrently by different, respective pixels of the receiver. This arrangement is beneficial both in increasing the acquisition throughput of ToF data and increasing the overall duty cycle of the emitter array, and thus mitigating problems that otherwise arise due to emission of very bright pulses with high peak power. 
     Furthermore, the spatio-temporal sequence may be modified in the course of system operation in order to optimize ToF data acquisition. For example, the sequence may be chosen originally in order to acquire ToF data rapidly at coarse resolution, and then modified to acquire the ToF data at finer resolution, either over the entire field of view of the system or only within a certain reduced region of interest. “Resolution” in this context may mean either or both of transverse resolution, corresponding to the density of the pixels in the resulting depth map, and depth resolution, corresponding to the precision of the measured ToF values. (Depth resolution varies statistically as a function of the number of pulses emitted by a given emitter and sensed at the corresponding pixel in order to generate a ToF measurement for the respective location in the scene.) Alternatively or additionally, the spatio-temporal sequence may be switched over time in order to defeat possible interference due to other radiation sources in the vicinity. 
       FIG.  1    is a schematic side view of a depth mapping device  20 , in accordance with an embodiment of the invention. In the pictured embodiment, device  20  is used to generate depth maps of a scene including an object  22 , which in this example is a part of the body of a user of the device. To generate the depth map, an illumination assembly  24  directs pulses of light toward object  22 , and a receiver  26  measures the ToF of the photons reflected from the object. (The term “light,” as used in the present description and in the claims, refers to optical radiation, which may be in any of the visible, infrared, and ultraviolet ranges.) 
     Illumination assembly  24  typically comprises an array of emitters, such as a pulsed laser array  28 . The lasers emit short pulses of light, with pulse duration in the nanosecond or sub-nanosecond range, and with on-board control that enables the individual lasers to be fired in a desired spatio-temporal sequence with sub-nanosecond precision. Array  28  may comprise, for example, an addressable VCSEL array. Collection optics  30  direct the light toward object  22 . 
     Alternatively, other pulse durations and repetition frequencies may be used, depending on application requirements. 
     Receiver  26  comprises objective optics  32 , which image object  22  onto a sensing array  34 , so that photons emitted by illumination assembly  24  and reflected from object  22  are incident on the sensing array. In the pictured embodiment, sensing array  34  comprises a sensor chip  36  and a processing chip  38 , which are coupled together, for example, using chip stacking techniques that are known in the art. Sensor chip  36  comprises an array of high-speed single-photon detectors, arranged in a matrix of rows and columns. 
     In some embodiments, the photodetectors in sensor chip  36  comprise an array of SPADs  40 , each of which outputs a signal indicative of the times of incidence of photons on the SPAD following emission of pulses by illumination assembly  24 . Processing chip  38  comprises an array of processing circuits  42 , which are coupled respectively to the sensing elements. Both of chips  36  and  38  may be produced from silicon wafers using well-known CMOS fabrication processes, based on SPAD sensor designs that are known in the art, along with accompanying drive circuits, logic and memory. 
     Alternatively, the designs and principles of detection that are described herein may be implemented, mutatis  mutandis , using other circuits, materials and processes. For example, sensing array  34  may comprise an array of avalanche photodiodes with suitable supporting and processing circuits, as are known in the art. All such alternative implementations are considered to be within the scope of the present invention. 
     Receiver  26  outputs signals that are indicative of respective times of arrival of the photons received at each pixel (corresponding to a SPAD  40  or group of neighboring SPADs), or equivalently, from each location in the scene that is being mapped. These output signals are typically in the form of respective digital values of the times of arrival that are generated by processing circuits  42 , although other signal formats, both digital and analog, are also possible. For example, in some embodiments, processing circuits  42  comprise one or more time-to-digital converters (TDCs), which output digital values indicative of the delays between pulses emitted by respective emitters in array  28  and receipt of a photon at a corresponding pixel of receiver  26 . Circuits  42  may comprise a respective dedicated TDC for each pixel. Alternatively, for appropriate choices of the spatio-temporal pattern of pulse emission, the TDCs can be shared by multiple pixels, for example with a single TDC per row (or equivalently, per column) of the detector array. 
     A controller  44  reads out the individual pixel values and processes the times of arrival, in synchronization with the spatio-temporal sequence of emission by the corresponding emitters, in order to generate an output depth map, comprising the ToF measured at each pixel to and from the respective locations in the scene, or equivalently, the measured depth value. In some embodiments, the spatio-temporal sequence is defined such that each of the emitters in array  28  is driven to emit a series of pulses, and sensing array  34  outputs a digital time-of-arrival value from the appropriate pixel in response to each emitted pulse. Controller  44  accumulates, for each pixel, a respective histogram of the digital values output from the pixel in response to the series of the pulses, and derives the respective time of flight from the histogram, for example by taking the mode of the histogram. This histogram-based approach is useful in smoothing out the statistical variations and noise in photon arrival time, and the depth resolution typically increases with the number of pulses captured at each pixel. Controller  44  conveys the depth map to a receiving device  46 , such as a display or a computer or other processor, which segments and extracts high-level information from the depth map, but these features are beyond the scope of the present disclosure. 
     Although the present description relates to controller  44  and processing chip  38  as separate entities, with a certain division of functions between the controller and processing chip, in practice these entities and their functions may be combined and implemented monolithically on the same integrated circuit. Alternatively, other divisions of functionality between these entities will also be apparent to those skilled in the art and are considered to be within the scope of the present invention. Therefore, in the present description and in the claims, the term “controller” should be understood as encompassing all implementations of the functionalities that are attributed to controller  44  and processing chip  38 , whether carried out in hardware logic or in software. 
       FIG.  2    is a block diagram that schematically illustrates integrated emitter array  28 , in accordance with an embodiment of the invention. VCSELs  50  in array  28  are produced by epitaxial growth on a suitable III-V semiconductor substrate  52 , such as a GaAs substrate. In a separate step, control circuits for the VCSELs are formed on a silicon substrate  54 , using a CMOS process, for example. III-V substrate  52  is then bonded to silicon substrate  54 , with each VCSEL  50  in alignment with its respective control circuits. In the pictured embodiment, drive circuits  58  on substrate  54  include a driver for each VCSEL  50 , with a separate anode and/or cathode connection to enable independent control of the VCSEL. An addressing circuit  56  triggers drive circuits  58  to fire the VCSELs in accordance with a predefined spatio-temporal sequence, which is input to addressing circuit  56 , for example, by controller  44 . 
     Reference is now made to  FIGS.  3 A-C  and  4 , which schematically illustrate features of a spatio-temporal sequence of emission from array  28 , in accordance with an embodiment of the invention.  FIGS.  3 A-C  are frontal views of emitter array  28 , showing successive stages in the emission sequence, while  FIG.  4    is a plot showing the timing of pulses in the emission sequence. 
     In the pictured sequence, emitters  50  are divided into groups, labeled  50   a ,  50   b , . . . ,  50   j , . . . . The emitters in each group belong to different, respective rows of the array and are spaced apart along a given column. The emitters in each group emit their respective pulses concurrently, with no more than a single emitter in each of the rows operative at any given time. The three groups that are shown in each of  FIGS.  3 A-C  (for example, emitters  50   a , emitters  50   b  and emitters  50   c  in  FIG.  3 A ) are also spaced apart over array  28 . In the pictured sequence, for example, the emitters in each of these groups are located in different rows and in columns that are widely separated. Thus, the respective pixels of sensing array  34  that will receive the reflected photons from these emitters are also widely spaced within each group and among the different groups. 
     As a result of this spatial distribution, emitters  50   a ,  50   b  and  50   c  (and similarly the emitter groups in  FIGS.  3 B and  3 C ) can be driven to emit their respective pulses with an interval between the pulses that is substantially shorter than the times of flight of the pulses to and from the respective locations in the scene being mapped by device  20 . The wide spacing of the emitters within and between the groups ensures that there will not be significant crosstalk between the pixels, and thus no significant aliasing on this account. For example, the interval between the pulses emitted from the different groups may be less than half the average time of flight, or even substantially less, as shown in  FIG.  4   . The wide spacing of the emitters is also useful in spreading the emitted power from array  28  widely over the scene being mapped, rather than focusing in a single location at any given time. 
       FIG.  4    shows an example of this sort of temporal scheme, with a train  62  of optical pulses  60   a ,  60   b  and  60   c  emitted periodically by the groups of emitters  50   a ,  50   b  and  50   c , respectively. Pulses  60   a ,  60   b ,  60   c  typically have respective pulse widths of 1 ns or less, and the interval between the pulses in each train  62  is likewise on the order of 1 ns. Pulse train  62  is repeated with a period T, which is chosen to be greater than the expected time of flight to and from locations in the scene being mapped, for example 10 ns for nearby targets. The tight spacing of the pulses emitted by the different groups of emitters is useful in increasing the duty cycle and reducing the peak emission power of array  28 , as well as increasing the measurement throughput of device  20 . Alternatively, other pulse widths and repetition periods may be used, depending on application requirements and choice of components. 
     Pulse train  62  is typically repeated multiple times for emitters  50   a ,  50   b  and  50   c , in order to enable controller  44  to build a suitable histogram of photon arrival times at each pixel in these groups. By virtue of the choice of emitters in these groups, with each group drawn from a separate set of rows of laser array  28 , the histograms can be accumulated using a single, respective TDC serving each row of sensor array  34 . To identify possible aliasing due to distant objects, the repetition period T may be dithered, in which case aliased histogram peaks (due to distant reflections from a previous pulse train) will be dithered, as well, and can be eliminated from the depth map on this basis. 
     After applying the desired series of pulse trains  62  to the groups of emitters  50   a ,  50   b  and  50   c , a similar series of pulse trains is applied to emitters  50   d ,  50   e , and  50   f , and then to emitters  50   g ,  50   h  and  50   j , and so forth until the entire array  28  has been covered. Alternatively, in some embodiments (as illustrated below in  FIG.  5 A /B, for example), the spatio-temporal sequence may use only a subset of emitters  50  in array  28 . Further alternatively, other spatial groupings and temporal sequences may be applied, as long as the separation between the groups in space and time is sufficient to avoid significant crosstalk between different pixels. 
     Although  FIGS.  3 A-C  show an orderly rolling sequence of emission groups, other sorts of spatial patterns, including patterns that are irregular in spatial and/or temporal distribution, may be applied to array  28 . Such irregular patterns (possibly including pseudo-random patterns) are useful in avoiding crosstalk and interference due to other depth mapping devices and other radiation sources in the vicinity of device  20 . 
     In some embodiments, SPADs  40  in sensing array  34  are actuated in synchronization with the spatio-temporal sequence of firing emitters  50  in array  28 . In other words, each SPAD is actuated only when the corresponding emitter is operative, and is effectively dormant otherwise. This mode of operation is useful in reducing noise in the ToF measurements. When applied in conjunction with the sort of spatio-temporal sequence that is illustrated in  FIGS.  3 A-C , the areas of active pixels and their acquisition triggers will roll smoothly across sensing array  34 , thus facilitating efficient, low-noise acquisition of ToF values. 
       FIGS.  5 A and  5 B  are schematic representation of a scene  70  that is mapped by device  20 , illustrating different, respective operational modes of the device, in accordance with an embodiment of the invention.  FIG.  5 A  illustrates an initial spatio-temporal sequence, in which only a subset of emitters  50  are actuated (for example, half of the emitters, as illustrated by the respective locations  72  in scene  70  that are illuminated by the emitters). Using this spatio-temporal sequence, controller  44  acquires an initial depth map of the scene with coarse transverse resolution, but with faster throughput than would be achieved if all of the emitters were to be used. Additionally or alternatively, the initial spatio-temporal sequence may include only a relatively small number of pulses emitted by each of the emitters, so that the depth resolution of the initial depth map is also relatively coarse. 
     After measuring the times of flight in accordance with the initial spatio-temporal sequence, controller  44  proceeds to re-measure the times of flight in accordance with a modified spatio-temporal sequence, as illustrated in  FIG.  5 B . Here, for example, a processor analyzing the initial depth map may have recognized an object of interest in scene  70 , and therefore instructs controller  44  to acquire times of flight with finer resolution over a limited area  74  within the scene that contains the object of interest. All of emitters  50  that emit beams within area  74  are thus actuated, enabling device  20  to acquire ToF values at respective, densely-spaced locations  76 , while the emitters outside this area are not used. Additionally or alternatively, the modified spatio-temporal sequence may include a larger number of pulses by each of the emitters illuminating area  74 . By reducing the effective field of view to area  74 , device  20  is able to generate a local depth map with finer resolution in the transvers and/or depth dimension, while maintaining high throughput. 
     In the modified spatio-temporal sequence illustrated in  FIG.  5 B , emitters  50  outside area  74  are not actuated in order to avoid wasting the resources of device  20  (such as the output power from array  28 ) on areas of scene  70  that are not of interest. This sort of conservation of resources can be further enhanced by deactivating emitters  50  even within area  74  if they are not able to give usable ToF values. For example, controller  44  may deactivate the emitter (or emitters) irradiating a bright spot  78  in scene  70  upon determining that sensing array  34  is unable to provide usable ToF values in this area due to saturation. 
     Thus, generally speaking, the parameters of the spatio-temporal sequence, including the choice of emitters and number of pulses applied by each emitter, may be selected and modified in order to give the optimal tradeoff between field of view, resolution and throughput. Although the embodiments described above refer, for the sake of concreteness and clarity, to a specific sort of device architecture that is capable of implementing this approach, the principles of the present invention may similarly be applied using other controllable emitter and sensor arrays, as will be apparent to those skilled in the art after reading the above disclosure. All such alternative implementations are considered to be within the scope of the present invention. 
     It will thus be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.

Metadata:
Filing Date: 20171218
Publication Date: 20231226
Grant Date: 20231226
Priority Date: 20171218
Inventors: LAFLAQUIÈRE, ARNAUD
DRADER, MARC
Assignee: APPLE INC
CPC Classifications: [{"code": "G01S17/89", "inventive": true, "first": true, "tree": "[]"}, {"code": "G01S7/483", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4815", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4865", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/10", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/894", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/89", "inventive": true, "first": true, "tree": "[]"}, {"code": "G01S7/4815", "inventive": true, "first": true, "tree": "[]"}, {"code": "G01S17/89", "inventive": true, "first": true, "tree": "[]"}, {"code": "G01S17/10", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4815", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/484", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/894", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/484", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/10", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/894", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4815", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/10", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/483", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4865", "inventive": true, "first": false, "tree": "[]"}]
Family ID: 60957452