PATENT DOCUMENT

Publication Number: US-10955234-B2
Application Number: US-201916532517-A
Country: US
Kind Code: B2

Title: Calibration of depth sensing using a sparse array of pulsed beams

Abstract:
Depth sensing apparatus includes a radiation source, which is configured to emit a first plurality of beams of light pulses toward a target scene. An array of a second plurality of sensing elements is configured to output signals indicative of respective times of incidence of photons on the sensing element, wherein the second plurality exceeds the first plurality. Light collection optics are configured to image the target scene onto the array of sensing elements. Processing and control circuitry is coupled to receive the signals from the array and is configured to search over the sensing elements in order to identify, responsively to the signals, respective regions of the array on which the light pulses reflected from the target scene are incident, and to process the signals from the identified regions in order determine respective times of arrival of the light pulses.

Claims:
The invention claimed is: 
     
       1. Depth sensing apparatus, comprising:
 a radiation source, which is configured to emit a first number of beams of light pulses toward a target scene; 
 an array of a second number of sensing elements, configured to output signals indicative of respective times of incidence of photons on the sensing element, wherein the first number is greater than one, and the second number exceeds the first number; 
 light collection optics configured to image the target scene sparsely onto the array of sensing elements so that only a fraction of the sensing elements output the signals in response to the light pulses reflected from the target scene, without receiving the light pulses reflected from the target scene at the other sensing elements; and 
 processing and control circuitry, which is coupled to receive the signals from the array and is configured to search over the sensing elements in order to identify, responsively to the signals, the sensing elements in respective regions of the array on which the light pulses reflected from the target scene are incident, and to process the signals from the identified sensing elements in the respective regions in order determine respective times of arrival of the light pulses. 
 
     
     
       2. The apparatus according to  claim 1 , wherein the radiation source comprises at least one vertical-cavity surface-emitting laser (VCSEL). 
     
     
       3. The apparatus according to  claim 2 , wherein the at least one VCSEL comprises an array of VCSELs. 
     
     
       4. The apparatus according to  claim 1 , wherein the sensing elements comprise single-photon avalanche diodes (SPADs). 
     
     
       5. The apparatus according to  claim 1 , wherein the processing and control circuitry is configured to group the sensing elements in each of the identified regions together to define super-pixels, and to process together the signals from the sensing elements in each of the super-pixels in order to determine the respective times of arrival. 
     
     
       6. The apparatus according to  claim 5 , wherein the processing and control circuitry comprises multiple processing units, wherein each of the processing units is coupled to process the signals from a respective one of the super-pixels. 
     
     
       7. The apparatus according to  claim 1 , wherein the processing and control circuitry is configured, after identifying the respective regions, to actuate only the sensing elements in each of the identified regions, while the remaining sensing elements in the array are inactive. 
     
     
       8. The apparatus according to  claim 1 , wherein the processing and control circuitry is configured to identify the respective regions of the array on which the light pulses reflected from the target scene are incident by selecting a set of candidate regions in which the reflected light pulses are likely to be incident, and performing an iterative search over the array starting with each of the candidate regions while operating the radiation source and receiving the signals to find the regions of the array onto which the light pulses reflected from the target scene are incident. 
     
     
       9. The apparatus according to  claim 8 , wherein the processing and control circuitry is configured to identify the candidate regions based on nominal design values together with assembly tolerances and operational tolerances of the apparatus. 
     
     
       10. Depth sensing apparatus, comprising:
 a radiation source, which is configured to emit a first number of beams of light pulses toward a target scene; 
 an array of a second number of sensing elements, configured to output signals indicative of respective times of incidence of photons on the sensing element, wherein the first number is greater than one, and the second number exceeds the first number; 
 light collection optics configured to image the target scene onto the array of sensing elements; and 
 processing and control circuitry, which is coupled to receive the signals from the array and is configured to search over the sensing elements in order to identify, responsively to the signals, respective regions of the array on which the light pulses reflected from the target scene are incident, and to process the signals from the identified regions in order determine respective times of arrival of the light pulses, 
 wherein the processing and control circuitry is configured to identify the respective regions of the array on which the light pulses reflected from the target scene are incident by selecting a set of candidate regions in which the reflected light pulses are likely to be incident, and performing an iterative search over the array starting with each of the candidate regions while operating the radiation source and receiving the signals to find the regions of the array onto which the light pulses reflected from the target scene are incident, and 
 wherein the processing and control circuitry is configured to perform the iterative search by shifting repeatedly from those sensing elements from which no signal is received to neighboring sensing elements, until a number of the regions from which the signals are received exceeds a preset threshold. 
 
     
     
       11. Depth sensing apparatus, comprising:
 a radiation source, which is configured to emit a first number of beams of light pulses toward a target scene; 
 an array of a second number of sensing elements, configured to output signals indicative of respective times of incidence of photons on the sensing element, wherein the first number is greater than one, and the second number exceeds the first number; 
 light collection optics configured to image the target scene onto the array of sensing elements; and 
 processing and control circuitry, which is coupled to receive the signals from the array and is configured to search over the sensing elements in order to identify, responsively to the signals, respective regions of the array on which the light pulses reflected from the target scene are incident, and to process the signals from the identified regions in order determine respective times of arrival of the light pulses, 
 wherein the processing and control circuitry is configured to identify the respective regions of the array onto which the light pulses reflected from the target scene are incident by finding an initial set of candidate regions of the array on which the light pulses reflected from the target scene are incident, calculating a model, based on the initial set, that predicts locations of additional regions of the array on which the light pulses reflected from the target scene are expected to be incident, and searching over the locations predicted by the model while operating the radiation source and receiving the signals in order to identify additional regions of the array on which the light pulses reflected from the target scene are incident. 
 
     
     
       12. The apparatus according to  claim 11 , wherein the processing and control circuitry is configured to adjust the model iteratively until a number of the regions from which the signals are received exceeds a preset threshold. 
     
     
       13. The apparatus according to  claim 12 , wherein the processing and control circuitry is configured to adjust the model by adding the identified additional regions to the initial set to produce a new set of the regions, and updating the model based on the new set. 
     
     
       14. The apparatus according to  claim 11 , wherein the model is selected from a group of types of models consisting of a homographic model, a quadratic model, and a low-order spline. 
     
     
       15. A method for depth sensing, comprising:
 driving a radiation source to emit a first number of beams of light pulses toward a target scene; 
 imaging the target scene sparsely onto an array of a second number of sensing elements, configured to output signals indicative of respective times of incidence of photons on the sensing element, wherein the first number is greater than one, and the second number exceeds the first number, so that only a fraction of the sensing elements output the signals in response to the light pulses reflected from the target scene, without receiving the light pulses reflected from the target scene at the other sensing elements; 
 searching over the sensing elements in order to identify, responsively to the signals, the sensing elements in respective regions of the array on which the light pulses reflected from the target scene are incident; and 
 processing the signals from the identified the sensing elements in the respective regions in order determine respective times of arrival of the light pulses. 
 
     
     
       16. The method according to  claim 15 , wherein processing the signals comprises grouping the sensing elements in each of the identified regions together to define super-pixels, and to processing the signals from the sensing elements in each of the super-pixels together in order to determine the respective times of arrival. 
     
     
       17. The method according to  claim 15 , wherein processing the signals comprises, after identifying the respective regions, actuating only the sensing elements in each of the identified regions, while the remaining sensing elements in the array are inactive. 
     
     
       18. The method according to  claim 15 , wherein searching over the sensing elements comprises:
 identifying the respective regions of the array on which the light pulses reflected from the target scene are incident by selecting a set of candidate regions in which the reflected light pulses are likely to be incident; and 
 performing an iterative search over the array starting with each of the candidate regions while operating the radiation source and receiving the signals to find the regions of the array onto which the light pulses reflected from the target scene are incident. 
 
     
     
       19. The method according to  claim 18 , wherein identifying the respective regions comprises selecting the candidate regions based on nominal design values together with assembly tolerances and operational tolerances. 
     
     
       20. The method according to  claim 15 , wherein searching over the sensing elements comprises:
 identifying the respective regions of the array onto which the light pulses reflected from the target scene are incident by finding an initial set of candidate regions of the array on which the light pulses reflected from the target scene are incident; 
 calculating a model, based on the initial set, that predicts locations of additional regions of the array on which the light pulses reflected from the target scene are expected to be incident; and 
 searching over the locations predicted by the model while operating the radiation source and receiving the signals in order to identify additional regions of the array on which the light pulses reflected from the target scene are incident.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application claims the benefit of U.S. Provisional Patent Application 62/803,612, filed Feb. 11, 2019, and U.S. Provisional Patent Application 62/809,647, filed Feb. 24, 2019. This application is related to U.S. patent application Ser. No. 16/532,513, filed Aug. 6, 2019, entitled “Depth sensing using a sparse array of pulsed beams”. All of these related applications are incorporated herein by reference. 
    
    
     FIELD OF THE INVENTION 
     The present invention relates generally to systems and methods for depth mapping, and particularly to beam sources and sensor arrays used in time-of-flight sensing. 
     BACKGROUND 
     Existing and emerging consumer applications have created an increasing need for real-time three-dimensional (3D) imagers. These imaging devices, also known as depth sensors, depth mappers, or light detection and ranging (LiDAR) sensors, enable the remote measurement of distance (and often intensity) to each point in a target scene—referred to as target scene depth—by illuminating the target scene with an optical beam and analyzing the reflected optical signal. A commonly-used technique to determine the distance to each point on the target scene involves transmitting one or more pulsed optical beams towards the target scene, followed by the measurement of the round-trip time, i.e. time-of-flight (ToF), taken by the optical beams as they travel from the source to the target scene and back to a detector array adjacent to the source. 
     Some ToF systems use single-photon avalanche diodes (SPADs), also known as Geiger-mode avalanche photodiodes (GAPDs), in measuring photon arrival time. For example, U.S. Pat. No. 9,997,551, whose disclosure is incorporated herein by reference, describes a sensing device that includes an array of SPAD sensing elements. Each sensing element includes a photodiode, including a p-n junction, and a local biasing circuit, which is coupled to reverse-bias the p-n junction at a bias voltage greater than the breakdown voltage of the p-n junction by a margin sufficient so that a single photon incident on the p-n junction triggers an avalanche pulse output from the sensing element. A bias control circuit is coupled to set the bias voltage in different ones of the sensing elements to different, respective values. 
     U.S. Patent Application Publication 2017/0176579, whose disclosure is incorporated herein by reference, describes the use of this sort of variable biasing capability in selectively actuating individual sensing elements or groups of sensing elements in a SPAD array. For this purpose, an electro-optical device includes a laser light source, which emits at least one beam of light pulses, a beam steering device, which transmits and scans the at least one beam across a target scene, and an array of sensing elements. Each sensing element outputs a signal indicative of a time of incidence of a single photon on the sensing element. (Each sensing element in such an array is also referred to as a “pixel.”) Light collection optics image the target scene scanned by the transmitted beam onto the array. Circuitry is coupled to actuate the sensing elements only in a selected region of the array and to sweep the selected region over the array in synchronization with scanning of the at least one beam. 
     SUMMARY 
     Embodiments of the present invention that are described hereinbelow provide improved depth mapping systems and methods for operating such systems. 
     There is therefore provided, in accordance with an embodiment of the invention, depth sensing apparatus, including a radiation source, which is configured to emit a first plurality of beams of light pulses toward a target scene. An array of a second plurality of sensing elements is configured to output signals indicative of respective times of incidence of photons on the sensing element, wherein the second plurality exceeds the first plurality. Light collection optics are configured to image the target scene onto the array of sensing elements. Processing and control circuitry is coupled to receive the signals from the array and is configured to search over the sensing elements in order to identify, responsively to the signals, respective regions of the array on which the light pulses reflected from the target scene are incident, and to process the signals from the identified regions in order determine respective times of arrival of the light pulses. 
     In some embodiments, the radiation source includes at least one vertical-cavity surface-emitting laser (VCSEL), and may include an array of VCSELs. Additionally or alternatively, the sensing elements include single-photon avalanche diodes (SPADs). 
     In some embodiments, the processing and control circuitry is configured to group the sensing elements in each of the identified regions together to define super-pixels, and to process together the signals from the sensing elements in each of the super-pixels in order to determine the respective times of arrival. In a disclosed embodiment, the processing and control circuitry includes multiple processing units, wherein each of the processing units is coupled to process the signals from a respective one of the super-pixels. Additionally or alternatively, the processing and control circuitry is configured, after identifying the respective regions, to actuate only the sensing elements in each of the identified regions, while the remaining sensing elements in the array are inactive. 
     In some embodiments, the processing and control circuitry is configured to identify the respective regions of the array on which the light pulses reflected from the target scene are incident by selecting a set of candidate regions in which the reflected light pulses are likely to be incident, and performing an iterative search over the array starting with each of the candidate regions while operating the radiation source and receiving the signals to find the regions of the array onto which the light pulses reflected from the target scene are incident. In one embodiment, the processing and control circuitry is configured to identify the candidate regions based on nominal design values together with assembly tolerances and operational tolerances of the apparatus. The processing and control circuitry may be configured to perform the iterative search by shifting repeatedly from those sensing elements from which no timing signal is received to neighboring sensing elements, until a number of the regions from which the signals are received exceeds a preset threshold. 
     In other embodiments, the processing and control circuitry is configured to identify the respective regions of the array onto which the light pulses reflected from the target scene are incident by finding an initial set of candidate regions of the array on which the light pulses reflected from the target scene are incident, calculating a model, based on the initial set, that predicts locations of additional regions of the array on which the light pulses reflected from the target scene are expected to be incident, and searching over the locations predicted by the model while operating the radiation source and receiving the signals in order to identify additional regions of the array on which the light pulses reflected from the target scene are incident. In a disclosed embodiment, the processing and control circuitry is configured to adjust the model iteratively until a number of the regions from which the signals are received exceeds a preset threshold. The processing and control circuitry may be configured to adjust the model by adding the identified additional regions to the initial set to produce a new set of the regions, and updating the model based on the new set. 
     Alternatively or additionally, the model is selected from a group of types of models consisting of a homographic model, a quadratic model, and a low-order spline. 
     There is also provided, in accordance with an embodiment of the invention, a method for depth sensing, which includes driving a radiation source to emit a first plurality of beams of light pulses toward a target scene. The target scene is imaged onto an array of a second plurality of sensing elements, configured to output signals indicative of respective times of incidence of photons on the sensing element, wherein the second plurality exceeds the first plurality. A search is performed over the sensing elements in order to identify, responsively to the signals, respective regions of the array on which the light pulses reflected from the target scene are incident. The signals from the identified regions are processed in order determine respective times of arrival of the light pulses. 
     The present invention will be more fully understood from the following detailed description of the embodiments thereof, taken together with the drawings in which: 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a schematic side view of a depth mapping system, in accordance with an embodiment of the invention; 
         FIG. 2A  is a schematic side view of a radiation source used in the depth mapping system of  FIG. 1 , in accordance with an embodiment of the invention; 
         FIG. 2B  is a schematic frontal view of an array of emitters used in the radiation source of  FIG. 2A , in accordance with an embodiment of the invention; 
         FIG. 2C  is a schematic frontal view of an array of emitters that can be used in the radiation source of  FIG. 2A , in accordance with another embodiment of the invention; 
         FIG. 3A  is a schematic representation of a pattern of spots projected onto a target scene, in accordance with an embodiment of the invention; 
         FIG. 3B  is a schematic frontal view of a ToF sensing array, in accordance with an embodiment of the invention; 
         FIG. 3C  is a schematic detail view of a part of the ToF sensing array of  FIG. 3B , onto which images of the spots in a region of the target scene of  FIG. 3A  are cast, in accordance with an embodiment of the invention; 
         FIGS. 4A and 4B  are schematic frontal views of a ToF sensing array showing sets of super-pixels that are selected for activation and readout in two different time periods, in accordance with an embodiment of the invention; 
         FIG. 5  is a block diagram that schematically illustrates circuitry for processing of signals from a super-pixel, in accordance with an embodiment of the invention; 
         FIG. 6  is a flowchart that schematically illustrates a method for identifying the pixels on a SPAD array that receive laser spots, in accordance with an embodiment of the invention; and 
         FIG. 7  is a flowchart that schematically illustrates a method for identifying the pixels on a SPAD array that receive laser spots, in accordance with another embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTION OF EMBODIMENTS 
     Overview 
     In some of the embodiments described in the above-mentioned U.S. Patent Application Publication 2017/0176579, SPADs are grouped together into “super-pixels,” wherein the term “super-pixel” refers to a group of mutually-adjacent pixels along with data processing elements that are coupled directly to these pixels. At any instant during operation of the system, only the sensing elements in the area or areas of the array that are to receive reflected illumination from a beam are actuated, for example by appropriate biasing of the SPADs in selected super-pixels, while the remaining sensing elements are inactive. The sensing elements are thus actuated only when their signals provide useful information. This approach reduces the background signal, thus enhancing the signal-to-background ratio, and lowers both the electrical power needs of the detector array and the number of data processing units that must be attached to the SPAD array. 
     One issue to be resolved in a depth mapping system of this sort is the sizes and locations of the super-pixels to be used. For accurate depth mapping, with high signal/background ratio, it is important that the super-pixels contain the detector elements onto which most of the energy of the reflected beams is imaged, while the sensing elements that do not receive reflected beams remain inactive. Even when a static array of emitters is used, however, the locations of the reflected beams on the detector array can change, for example due to thermal and mechanical changes over time, as well as optical effects, such as parallax. 
     In response to this problem, some embodiments of the present invention provide methods for calibrating the locations of the laser spots on the SPAD array. For this purpose, processing and control circuitry receives timing signals from the array and searches over the sensing elements in order to identify the respective regions of the array on which the light pulses reflected from the target scene are incident. Detailed knowledge of the depth mapping system may be used in order to pre-compute likely regions of the reflected laser spots to be imaged onto the SPAD array. A random search in these regions will converge rapidly to the correct locations of the laser spots on the array. Alternatively or additionally, a small subset of the locations of laser spots can be identified in an initialization stage. These locations can be used in subsequent iterative stages to predict and verify the positions of further laser spots until a sufficient number of laser spots have been located. 
     Even following meticulous calibration, it can occur in operation of the depth mapping system that some of the pixels or super-pixels on which laser spots are expected to be imaged fail to output usable timing signals. In some cases, ancillary image data can be used to identify areas of the scene that are problematic in terms of depth mapping, and to recalibrate the super-pixel locations when necessary. This ancillary image data can be provided, for example, by a color image sensor, which captures two-dimensional (2D) images in registration with the SPAD array. 
     The emitter arrays used in the embodiments described below are “sparse,” in the sense that the number of pulsed beams of optical radiation that are emitted toward a target scene is substantially less than the number of pixels (i.e., SPADs or other sensing elements) in the array that receives the radiation reflected from the scene. The illumination power available from the emitter array is projected onto a correspondingly sparse grid of spots in the scene. The processing and control circuitry in the apparatus then receives and processes signals only from the pixels onto which these spots are imaged in order to measure depth coordinates. 
     The pixels onto which the spots are imaged are referred to herein as the “active pixels,” and the “super-pixels” are made up of groups of adjacent active pixels, for example 2×2 groups. The pixels in the array that fall between the active pixels are ignored, and need not be actuated or read out at all, as they do not contribute to the depth measurement and only increase the background level and noise. Alternatively, a different number, such as one, two, three or more pixels, may be included in a super-pixel. Furthermore, although the embodiments described herein relate specifically to rectangular super-pixels, the group of SPAD pixels in a super-pixel may have a different shape, such as, for example, diamond shape, triangular, circular, or irregular. The exact location of the spot within the SPAD pixels varies slightly depending on the distance to the scene due to a small amount of parallax. At any given time, the signals from the SPAD pixels of the super-pixel are processed together in measuring for a given laser spot both its strength (intensity) and its time of flight. Additionally, the signals from the SPAD pixels may be processed as individual signals for determining the location of the laser spot within the super-pixel. 
     An advantage of using a sparse emitter array is that the available power budget can be concentrated in the small number of projected spots, rather than being spread over the entire field of view of the sensing array. As a result of this concentration of optical power in a small number of spots, the signal levels from the corresponding active pixels—and thus the accuracy of ToF measurement by these pixels—are enhanced. This signal enhancement is particularly beneficial for long-range depth measurements and for depth mapping in conditions of strong ambient light, such as outdoors. 
     The concentration of optical power in a sparse array of spots can be further enhanced by arranging the emitters in multiple banks, and actuating these banks in alternation. The laser beams generated by the emitters are typically collimated by a collimating lens and may be replicated by a diffractive optical element (DOE) in order to increase the number of projected spots. The pulses of optical radiation emitted from the different banks of the emitters are incident, after reflection from the target scene, on different, respective sets of the active pixels. The processing and control circuitry can then receive and process the signals from the active pixels in these respective sets in synchronization with actuation of the corresponding banks of emitters. Thus, during any given period in the operation of the apparatus, the processing and control circuitry need receive and process the signals only from one active set of sensing elements, while all other sets remain inactive. This sort of multi-bank, synchronized operation makes it possible to time-multiplex processing resources among the different sets of sensing elements, and thus reduce circuit complexity and power consumption. 
     Because the spots reflected from the target scene are imaged sparsely onto the SPAD array, the number of possible super-pixels is much larger than the number of laser spots, and only a small fraction of the total number of pixels in the SPAD array should be active at any given time and coupled to a processing unit for the purpose of measuring time-of-flight. Therefore, information is required as to which of the SPAD super-pixels to activate at any given time. 
     A mapping of SPAD pixels to processing units, i.e., the assignment of SPAD pixels to super-pixels, may be determined initially during a factory calibration. However, temperature changes during operation, as well as mechanical shocks, may alter the mechanical parameters of the mapping, thus modifying the positions of the laser spots on the SPAD array and necessitating recalibration during operation in the field. An exhaustive search could be used to determine which of the SPAD pixels to connect to the processing units, wherein all pixels are searched to detect laser spots; but this approach suffers from at least two basic problems:
         Some laser spots may not fall on objects in the scene or may fall on objects that absorb the laser wavelength, thus returning no pulse. Therefore a search may not always be successful.   As the distribution of laser spots is very sparse when compared to the number of pixels of the SPAD array, exhaustive search will require a large number of exposures and will take a long time.       

     The embodiments of the present invention that are described herein address these problems by providing improved methods for calibrating the locations of the laser spots on the SPAD array. These methods can be applied not only in the sorts of arrays that are shown in the figures and described hereinbelow, but also in other SPAD-based systems, such as systems comprising multiple banks of SPADs, as well as SPADs of various sizes, and systems using various sorts of emitters and emitter arrays, including emitters whose beams are replicated by a DOE. The present methods can then be extended, mutatis mutandis, to multi-bank systems, by activating the SPAD pixels and performing the calibration bank by bank. 
     In a disclosed embodiment, detailed knowledge of the depth mapping system is utilized to pre-compute likely regions of the reflected laser spots to be imaged onto the SPAD array. A search in these regions, for example a random search, will converge rapidly to the correct locations of the laser spots on the array. 
     Another disclosed embodiment uses a two-stage solution: in an initialization stage, a small subset of the locations of laser spots is identified, and in a subsequent iterative stage, the positions of further laser spots are predicted by a model and verified. Iterative steps of spot detection are utilized to refine the model and add locations, until a sufficient number of laser spots have been located. 
     System Description 
       FIG. 1  is a schematic side view of a depth mapping system  20 , in accordance with an embodiment of the invention. Depth mapping system  20  comprises a radiation source  21 , which emits M individual beams (for example, M may be on the order of 500). The radiation source comprises multiple banks of emitters arranged in a two-dimensional array  22  (as shown in detail in  FIG. 2B ), together with beam optics  37 . The emitters typically comprises solid-state devices, such as vertical-cavity surface-emission lasers (VCSELs) or other sorts of lasers or light-emitting diodes (LEDs). The beam optics typically comprise a collimating lens and may comprise a diffractive optical element (DOE, not shown), which replicates the actual beams emitted by array  22  to create the M beams that are projected onto the scene  32 . (For example, an array of four banks of pixels with  16  VCSELs in a 4×4 arrangement in each bank may be used to create 8×8 beams, and a DOE may split each beam into 3×3 replicas to give a total of 24×24 beams.) For the sake of simplicity, these internal elements of beam optics  37  are not shown. 
     A receiver  23  in system  20  comprises a two-dimensional SPAD array  24 , together with J processing units  28  and select lines  31  for coupling the processing units to the SPADs, along with a combining unit  35  and a controller  26 . SPAD array  24  comprises a number of detector elements N that is much larger than M, for example, 100×100 pixels or 200×200 pixels. The number J of processing units  28  depends on the number of pixels of SPAD array  24  to which each processing unit is coupled, as will be further described with reference to  FIG. 4 . 
     Array  22 , together with beam optics  37 , emits M pulsed beams of light  30  towards a target scene  32 . Although beams  30  are depicted in  FIG. 1  as parallel beams of constant width, each beam diverges as dictated by diffraction. Furthermore, beams  30  diverge from each other so as to cover a required area of scene  32 . Scene  32  reflects or otherwise scatters those beams  30  that impinge on the scene. The reflected and scattered beams are collected by objective optics  34 , represented by a lens in  FIG. 1 , which form an image of scene  32  on array  24 . Thus, for example, a small region  36  on scene  32 , on which a beam  30   a  has impinged, is imaged onto a small area  38  on SPAD array  24 . 
     A Cartesian coordinate system  33  defines the orientation of depth mapping system  20  and scene  32 . The x-axis and the y-axis are oriented in the plane of SPAD array  24 . The z-axis is perpendicular to the array and points to scene  32  that is imaged onto SPAD array  24 . 
     For clarity, processing units  28  are shown as if separate from SPAD array  24 , but they are commonly integrated with the SPAD array. Similarly, combining unit  35  is commonly integrated with SPAD array  24 . Processing units  28 , together with combining unit  35 , comprise hardware amplification and logic circuits, which sense and record pulses output by the SPADs in respective super-pixels, and thus measure the times of arrival of the photons that gave rise to the pulses, as well as the strengths of the optical pulses impinging on SPAD array  24 . 
     As further described below in reference to  FIG. 4 , processing units  28  together with combining unit  35  may assemble histograms of the times of arrival of multiple pulses emitted by array  22 , and thus output signals that are indicative of the distance to respective points in scene  32 , as well as of signal strength. Circuitry that can be used for this purpose is described, for example, in the above-mentioned U.S. Patent Application Publication 2017/0176579. Alternatively or additionally, some or all of the components of processing units  28  and combining unit  35  may be separate from SPAD array  24  and may, for example, be integrated with controller  26 . For the sake of generality, controller  26 , processing units  28  and combining unit  35  are collectively referred to herein as “processing and control circuitry.” 
     Controller  26  is coupled to both radiation source  21  and receiver  23 . Controller  26  actuates the banks of emitters in array  22  in alternation to emit the pulsed beams. The controller also provides to the processing and combining units in receiver  23  an external control signal  29 , and receives output signals from the processing and combining units. The output signals may comprise histogram data, and may be used by controller  26  to derive both times of incidence and signal strengths, as well as a precise location of each laser spot that is imaged onto SPAD array  24 . 
     To make optimal use of the available sensing and processing resources, controller  26  identifies the respective areas of SPAD array  24  on which the pulses of optical radiation reflected from corresponding regions of target scene  32  are imaged by lens  34 , and chooses the super-pixels to correspond to these areas. The signals output by sensing elements outside these areas are not used, and these sensing elements may thus be deactivated, for example by reducing or turning off the bias voltage to these sensing elements. Methods for choosing the super-pixels initially and for verifying and updating the selection of super-pixels are described, for example, in the above-mentioned provisional patent applications. 
     External control signal  29  controls select lines  31  so that each processing unit  28  is coupled to a respective super-pixel, comprising four SPAD pixels, for example. The control signal selects the super-pixels from which the output signals are to be received in synchronization with the actuation of the corresponding banks of emitters. Thus, at any given time, processing units  28  and combining unit  35  read and process the signals only from the sensing elements in the areas of SPAD array  24  that receive the reflected pulses from scene  32 , while the remaining sensing elements in the array are inactive. The processing of the signals from SPAD array  24  is further described in reference to  FIG. 4 . For the sake of simplicity, the detailed structures of emitter array  22  and SPAD array  24  are not shown in  FIG. 1 . 
     For clarity, the dimensions of emitter array  22  and SPAD array  24  have been exaggerated in  FIG. 1  relative to scene  32 . The lateral separation of emitter array  22  and SPAD array  24 , referred to as the “baseline,” is in reality much smaller than the distance from emitter array  22  to scene  32 . Consequently a chief ray  40  (a ray passing through the center of objective optics  34 ) from scene  32  to SPAD array  24  is nearly parallel to rays  30 , leading to only a small amount of parallax. 
     Of those M super-pixels that are activated and coupled to the J processing units  28 , either all of them or a subset of m super-pixels, wherein m&lt;M, will receive a reflected laser beam  30 . The magnitude of m depends on two factors:
         1. The calibration of SPAD array  24 , i.e., the choice of the M super-pixels, and   2. The number of laser beams  30  that are actually reflected from scene  32 .
 
The value M may correspond to the total number of emitters when all of the emitters are actuated together, or to the number of emitters in each bank when the banks are actuated in alternation, as in the present embodiments.
       

     Even if all M laser beams  30  were to be reflected from scene  32 , m will be less than M if SPAD array  24  is not properly calibrated. (Calibration procedures described in the above-mentioned provisional patent applications can be used to maximize m.) Consequently, controller  26  will receive signals indicating times of arrival and signal strengths from only m processing units  28 . Controller  26  calculates from the timing of the emission of beams  30  by VCSEL array  22  and from the times of arrival measured by the m processing units  28  the time-of-flight of the m beams, and thus maps the distance to the corresponding m points on scene  32 . 
     Controller  26  typically comprises a programmable processor, which is programmed in software and/or firmware to carry out the functions that are described herein. Alternatively or additionally, controller  26  comprises hard-wired and/or programmable hardware logic circuits, which carry out at least some of the functions of the controller. Although controller  26  is shown in the figure, for the sake of simplicity, as a single, monolithic functional block, in practice the controller may comprise a single chip or a set of two or more chips, with suitable interfaces for receiving and outputting the signals that are illustrated in the figure and are described in the text. 
     One of the functional units of controller  26  is a depth processing unit (DPU)  27 , which processes signals from both processing units  28  and combining unit  35 , as will be further described below. DPU  27  calculates the times of flight of the photons in each of beams  30 , and thus maps the distance to the corresponding points in target scene  32 . This mapping is based on the timing of the emission of beams  30  by emitter array  22  and from the times of arrival (i.e., times of incidence of reflected photons) measured by processing units  28 . Controller  26  typically stores the depth coordinates in a memory, and may output the corresponding depth map for display and/or further processing. 
     Emitter Array 
       FIG. 2A  is a schematic side view of radiation source  21 , in accordance with an embodiment of the invention. VCSEL array  22  comprises an integrated circuit chip on which multiple banks of VCSELs are formed (as shown in  FIG. 2B , for example). The VCSELs emit respective beams  30  toward optics  37 , which collimate and project the beams toward the target scene. Optics  37  optionally comprise a diffractive optical element (DOE), which splits the optical radiation emitted by each of the VCSELs into multiple beams  30 , for example a 3×3 array of beams. 
     To enable selection and switching among the different banks, array  22  is mounted on a driver chip  50 , for example, a silicon chip with CMOS circuits for selecting and driving the individual VCSELs or banks of VCSELs. The banks of VCSELS in this case may be physically separated, for ease of fabrication and control, or they may be interleaved on the VCSEL chip, with suitable connections to driver chip  50  to enable actuating the banks in alternation. Thus, beams  30  likewise irradiate the target scene in a time-multiplexed pattern, with different sets of the beams impinging on the respective regions of the scene at different times. 
       FIG. 2B  is a schematic frontal view of array  22  used in beam source  21 , in accordance with an embodiment of the invention. Array  22  in this example comprises eight banks  52 , with seventy-two emitters  54 , such as VCSELs, in each bank. In this case, array  22  generates  578  beams. 
       FIG. 2C  is a schematic frontal view of an array  60  of vertical emitters  54  that can be used in beam source  21  in place of array  22 , in accordance with another embodiment of the invention. In this case four banks  62   a ,  62   b ,  62   c  and  62   d  of emitters  54  are interleaved as alternating vertical stripes on a substrate  64 , such as a semiconductor chip: Each bank comprises a number of stripes that alternate on the substrate with the stripes in the other banks. Alternatively, other interleaving schemes may be used. 
     As further alternatives to the pictured embodiments, array  22  may comprise a larger or smaller number of banks and emitters. Typically, for sufficient coverage of the target scenes with static (non-scanned) beams, array  22  comprises at least four banks  52  or  62 , with at least four emitters  54  in each bank, and possibly with a DOE for splitting the radiation emitted by each of the emitters. For denser coverage, array  22  comprises at least eight banks  52  or  62 , with twenty emitters  54  or more in each bank. These options enhance the flexibility of system  20  in terms of time-multiplexing of the optical and electrical power budgets, as well as processing resources. 
     Super-Pixel Selection and Actuation 
       FIG. 3A  is a schematic representation of a pattern of spots  70  of optical radiation that are projected onto target scene  32 , in accordance with an embodiment of the invention. Each spot  70  is cast by a corresponding beam  30  ( FIG. 1 ). In the present embodiments, different groups of spots  70  are projected onto scene  32  in alternation, corresponding to the alternating actuation of the corresponding banks  52  of emitters  54  ( FIG. 2B ). 
       FIG. 3B  is a schematic frontal view of SPAD array  24  onto which target scene  32  is imaged, in accordance with an embodiment of the invention. The sensing elements, such as SPADs, in array  24  are too small to be seen in this figure. Rather,  FIG. 3B  shows the locations of spots  72  that are reflected from target scene  70  and imaged onto array  24  by lens  34 . In other words, each spot  72  is the image on array  24  of a corresponding spot  70  that is projected onto scene  32  by emitter array  22 . Lens  34  images a region  74  of target scene  32  ( FIG. 3A ), including spots  70  that the area contains, onto a corresponding area  76  on array  24 . 
       FIG. 3C  is a schematic detail view of area  76  of array  24 , showing the locations of spots  72  that are imaged onto the array, in accordance with an embodiment of the invention. These spots  72  may be imaged at the same time, if they originate from the same bank of emitters, or at different, alternating times if they are from different banks. In this view shown in  FIG. 3C , it can be seen that array  24  comprises a matrix of sensing elements  78 , such as SPADs. (As noted earlier, sensing elements  78  in an array are also referred to as “pixels.”) Controller  26  assigns each processing unit  28  to a super-pixel  80  comprising a 2×2 group of the sensing elements  78 . In this example, it is assumed that during an initial calibration stage, spots  72  were imaged onto array  24  at locations  72   a . Controller  26  thus selected the sensing elements  78  to assign to each super-pixel  80  so as to maximize the overlap between the corresponding spot  72  and the super-pixel, and thus maximize the signal received from each super-pixel. 
     At some later stage, however, spots  72  shifted to new locations  72   b  on array  24 . This shift may have occurred, for example, due to mechanical shock or thermal effects or due to other causes. Spots  72  at locations  72   b  no longer overlap with super-pixels  80  in area  76 , or overlap only minimally with the super-pixels. Sensing elements  78  on which the spots are now imaged, however, are inactive and are not connected to any of processing units  28 . To rectify this situation, controller  26  may recalibrate the locations of super-pixels  80 , as described in the above-mentioned provisional patent applications. 
       FIGS. 4A and 4B  are schematic frontal views of SPAD array  24  showing sets of super-pixels  80   a  and  80   b  that are selected for activation and readout in two different time periods, in accordance with an embodiment of the invention. In the embodiment, the selection of the set of super-pixels is synchronized with the selection of banks of emitters. Specifically, assuming array  60  ( FIG. 2C ) is used for generating the spots on the target scene, super-pixels  80   a  will be used when bank  62   a  is actuated, and super-pixels  80   b  will be used when bank  62   b  is actuated (and so forth for banks  62   c  and  62   d ). Thus, during any given period in the operation of the system  20 , processing units  28  serve only the one active set of super-pixels  80 , while all other sets remain inactive, in an integrated time-multiplexing scheme. 
       FIG. 5  is a block diagram that schematically illustrates the processing of signals from a super-pixel  80 , in accordance with an embodiment of the invention. In the pictured embodiment, super-pixel  80  comprises four sensing elements  78  of SPAD array  24 . Each processing unit  28  comprises one or more time-to-digital converters (TDC)  143 , wherein the TDCs are hardware elements translating the avalanche events (signals from a SPAD pixel due to a detected photon) from each sensing element  78  to time-of-arrival information. Each processing unit  28  further comprises, for each TDC  143 , a weight  144 , and may comprise a histogram unit (not shown), wherein the time-of-arrival information is aggregated into histograms, generally over thousands of pulses from VCSEL array  22 . In the present embodiment, however, the histograms are aggregated centrally for super-pixel, without individual histogram units in processing units  28 . 
     In  FIG. 5 , each processing unit  28  is coupled to a single sensing element  78 , thus requiring one TDC  143 . Alternatively, processing unit  28  may be coupled to two or more sensing elements  78 , and will then comprise a number of TDC&#39;s  143  equal to the number of pixels. For example, if each processing unit  28  is coupled to four SPAD pixels  24 , the number of TDCs  143  per processing unit will be four, and J=4*M. Additionally or alternatively, as noted earlier, processing units  28  may be switched among different sensing elements  78 , which are activated at different, alternating times, in synchronization with the alternating actuation of the corresponding banks  52  of emitters  54  ( FIG. 2B ). 
     The time-of-arrival information from the four processing units  28  is aggregated by combining unit  35 , using weights  144 , to yield a single histogram  146  for super-pixel  80 . This combined histogram  146  is sent to DPU  27 , which in turn detects, based on histogram  146 , whether any object or structure was detected in scene  32  by super-pixel  80  and, if so, reports its depth information based on time-of-flight data. 
     Additionally, the respective numbers of events reported by the four processing units  28  may be separately summed in combining unit  35  over a predefined interval of arrival times to yield an indication of the received signal strength for that interval for each sensing element  78 . Typically the interval is configured to start after the end of a so-called “stray pulse” and continue to the end of the histogram. A stray pulse is a pulse that is generated within system  20  as a result of, for example, an imperfect coating of an optical surface, which causes a reflection of the pulses emitted by VCSEL array  22  directly back into the optical path to SPAD array  24 . It is typically an undesired pulse, but one that is very difficult to eliminate altogether. The stray pulse may be utilized for calibrating the timing signals as follows: A time of arrival of a stray pulse is recorded and subtracted from a subsequent timing signal due to a laser pulse that has been reflected by scene  32 . This subtraction yields a relative time-of-flight for the received laser pulse, and compensates for any random firing delays of VCSEL array  22 , as well as for most of the VCSEL and SPAD drifts related to temperature changes. 
     These four indicators of signal strength are also transferred to DPU  27  (in conjunction with combined histogram  146 ). The indicators may be used by DPU  27  to determine a precise location of the spot on sensing elements  78 . 
     In some embodiments, the four units of TDC  143 , as well as combining unit  35 , reside in the same chip as SPAD array  24 , while the rest of signal processing, including DPU  27 , resides in separate controller  26 . A major reason for generating single combined histogram  146  for super-pixel  80  is to reduce the information that is transferred from SPAD array  24  to DPU  27  and to controller  26 . The partitioning into two separate units reflects the fact that SPAD array  24  and the associated units perform primarily optical and analog functions, while controller  26  performs mostly digital and software-driven operations. 
     Super-Pixel Calibration by Search in Precomputed Regions 
       FIG. 6  is a flowchart that schematically illustrates a method for identifying the pixels in a sensing array that receive laser spots, in accordance with an embodiment of the invention. This method is described, for the sake of convenience and clarity, with reference to SPAD array  24  and the other elements of system  20  ( FIG. 1 ). The method can be performed, for example, each time system  20  is turned on. Alternatively, the method can be carried out prior to an initial use of system  20 , and the results can be stored for future use. The method can then be repeated periodically and/or when system performance indicates that recalibration may be required. 
     Alternatively, however, the principles of this method may be applied, mutatis mutandis, in other depth mapping systems of similar configuration. For example, VCSEL array  22  could be replaced by a single laser (or a small number of lasers), with a beamsplitting element, such as a diffractive optical element (DOE), to split the laser output into multiple beams. As another example, other types of sensing arrays, comprising other sorts of detector elements, could be used in place of SPADs. The method of  FIG. 7 , as described below, is similarly applicable not only to system  20 , but to other depth mapping systems, as well. 
     In the method of  FIG. 6 , three input steps: a nominal design value step  150 , an assembly tolerance step  152 , and an operational tolerance step  154 , provide inputs for a pre-computation step  156 . Design value step  150  provides the nominal system design values for depth mapping system  20  ( FIG. 1 ); assembly tolerance step  152  provides the assembly tolerances of the depth mapping system; and operational tolerance step  154  provides expected operational tolerances, such as variations of ambient temperature and the effects of a mechanical shock on the depth mapping system. 
     The above inputs include multiple parameters. For example, a typical focal length of collection lens  34  has a nominal value of 2 mm, an assembly tolerance of 0.1 mm and an operational tolerance of 0.05 mm. Each tolerance is normally distributed around zero, with a standard deviation equal to the above tolerance. The probability distribution of the focal length is a normal distribution combined from the two normal distributions and centered at the nominal value of 2 mm. An additional example of a parameter is the baseline between VCSEL array  22  and SPAD array  24 . The multiple parameters, such as the two examples described above, allow controller  26  to model accurately the optical path taken by the laser pulses and thus calculate the locations where the spots impinge on SPAD array  24 . 
     Based on these inputs, controller  26  calculates a search region for each of the M laser spots expected on SPAD array  24  ( FIG. 1 ), in a pre-computation step  156 . Each search region includes a group of pixels for which a probability of receiving a laser beam reflected from scene  32  is estimated to be higher than a preset threshold, such as 99.9%. As an example of the calculations performed by controller  26 , an increase of 1% in the focal length of collection lens  34  magnifies the image on SPAD array  24  by 1%, thus shifting the spots in an outward radial direction. The probability distribution of this parameter and all other parameters of the input translates to a region around each of the nominal spot locations on SPAD array  24  in which there is a probability higher than 99.9% to find the spot. 
     Once the search regions have been chosen in pre-computation step  156 , controller  26 , in a random iterative search step  158 , fires a succession of pulses of beams  32  from VCSEL array  22  ( FIG. 1 ), and at the same time performs random searches within the search regions to identify the M super-pixels that receive the pulsed beams. Alternatively, controller  26  may apply other search strategies, not necessarily random, within the search regions. During step  158 , each processing unit  28  is coupled to receive signals from a different pixel following each laser pulse or sequence of multiple pulses, and controller  26  checks, using DPU  27 , which pixels have output signals due to an incident photon, and which have not. Based on the results, controller  26  selects the pixels to include in each super-pixel as those on which the photons were found to be incident. In simulations, the search was found to converge within a succession of 8-10 repeated sequences of pulsed beams  32  and thus identify the M super-pixels of SPAD array  24  that receive the M beams. 
     Once controller  26  has found the M super-pixels, it finishes the search and assigns, in an assignment step  160 , these super-pixels for use in 3D mapping of scene  32  by depth mapping system  20 . 
     Two-Stage Solution for Super-Pixel Calibration 
       FIG. 7  is a flowchart that schematically illustrates a two-stage method for identifying the super-pixels in SPAD array  24  ( FIG. 1 ) that receive laser spots, in accordance with another embodiment of the invention. The first stage starts by providing, in an input step  200 , a small number m 0  of potential process candidates, wherein, in this context, the term “process candidate” is used for those SPAD super-pixels likely to receive laser spots. A typical number for potential process candidates is either a fixed number, such as m 0 =5, or a percentage, such as 10%, of the number M. These potential process candidates may be obtained, for example, from a previous use of depth mapping system  20 . 
     These potential candidates are coupled to respective processing units  28 , in a candidate processing step  202 . In a first detection step  204 , controller  26  fires a sequence of pulses of beams  32  from VCSEL array  22  and queries processing units  28  and combining unit  35  to find out how many of the m 0  process candidates on SPAD array  24  reported “hits,” i.e., output signals indicating that they had received photons. In a first comparison step  206 , controller  26  checks whether the number of reported hits in first detection step  204  exceeds a first preset threshold, for example 8% of M (if initially 10% of M were selected as process candidates). 
     If the number of hits was below the threshold, controller  26  searches, in a search step  208 , for hits in the areas around the process candidates by firing successive pulsed beams  32  from VCSEL array  22  and performing a single pixel search around the candidates. After new hits have been identified, the previous process candidates in process candidate step  202  are replaced by the new hits, and steps  204  and  206  are repeated until the number of detected hits in first comparison step  206  exceeds the first preset threshold. 
     The detected hits in first comparison step  206  are used by controller  26  to build a model in a modeling step  210 . The model expresses the deviation of the locations of the hits in SPAD array  24  relative to their nominal locations, i.e., the locations where the reflected laser beams were expected to be incident on SPAD array  24  according to the design geometry of system  20 , for example. The model may be a quadratic model, a simplified pinhole camera model, or a homographic model, for example, and it may take into account system tolerances as previously described. 
     A homographic model h is an eight-parameter transformation (h 1 , . . . , h 8 ), mapping a point p=(x,y) to another point p′=(x′,y′) through the relation: 
     
       
         
           
             
               ( 
               
                 
                   x 
                   ′ 
                 
                 , 
                 
                   y 
                   ′ 
                 
               
               ) 
             
             = 
             
               ( 
               
                 
                   
                     
                       
                         h 
                         1 
                       
                       ⁢ 
                       x 
                     
                     + 
                     
                       
                         h 
                         2 
                       
                       ⁢ 
                       y 
                     
                     + 
                     
                       h 
                       3 
                     
                   
                   
                     
                       
                         h 
                         7 
                       
                       ⁢ 
                       x 
                     
                     + 
                     
                       
                         h 
                         8 
                       
                       ⁢ 
                       y 
                     
                     + 
                     1 
                   
                 
                 , 
                 
                   
                     
                       
                         h 
                         4 
                       
                       ⁢ 
                       x 
                     
                     + 
                     
                       
                         h 
                         5 
                       
                       ⁢ 
                       y 
                     
                     + 
                     
                       h 
                       6 
                     
                   
                   
                     
                       
                         h 
                         7 
                       
                       ⁢ 
                       x 
                     
                     + 
                     
                       
                         h 
                         8 
                       
                       ⁢ 
                       y 
                     
                     + 
                     1 
                   
                 
               
               ) 
             
           
         
       
     
     The coordinates x and y refer to Cartesian coordinate system  33  of  FIG. 1 . Such a model represents the correct spot locations on SPAD array  24  in a case where scene  34  comprises a plane (for instance, a wall). 
     A quadratic model is given by:
 
 x′=a   1   +b   1   x+c   1   y+d   1   x   2   +e   1   y   2   +f   1   xy  
 
 y′=a   2   +b   2   x+c   2   y+d   2   x   2   +e   2   y   2   +f   2   xy  
 
     The equations of a simplified pinhole camera model are as follows: Given a point (x,y,z) in Cartesian coordinate system  33 , we first compute the undistorted image coordinates: 
     
       
         
           
             
               
                 x 
                 u 
               
               = 
               
                 
                   f 
                   ⁢ 
                   
                     X 
                     Z 
                   
                 
                 + 
                 
                   c 
                   x 
                 
               
             
             , 
             
                 
             
             ⁢ 
             
               
                 y 
                 u 
               
               = 
               
                 
                   f 
                   ⁢ 
                   
                     Y 
                     Z 
                   
                 
                 + 
                 
                   
                     c 
                     y 
                   
                   . 
                 
               
             
           
         
       
     
     We then apply a distortion operation to obtain the final image coordinates:
 
 x   d   =c   x +( x   u   −c   x )· p ( r ), y   d   =c   y +( y   u   −c   y )· p ( r ),
 
where
 
 r =√{square root over (( x   u   −c   x ) 2 +( y   u   −c   y ) 2 )}
 
and p(r)=1+k 1 r 2 +k 2 r 4 +k 3 r 6  is a distortion polynomial. The parameters of the model are, therefore, the following constants: f, c x , c y , k 1 , k 2 , k 3  (see G. Bradski and A. Kaehler,  Learning OpenCV,  1 st  edition, O&#39;Reilly Media, Inc., Sebastopol, Calif., 2008).
 
     Additionally or alternatively, other models, such as splines or more elaborate models that describe the optics of system  20  to a higher degree of complexity, may be employed. 
     Based on one of the above models, controller  26  predicts the locations of a number of new process candidates, by applying the model to the nominal locations of a number of additional pixels where other reflected laser beams were expected to be incident, in a candidate addition step  212 , making now up a total of m 1  process candidates. Typically, m 1  increases in each iteration at candidate addition step  212 . In a second detection step  214  controller  26  fires an additional sequence of pulses of beams  30  from VCSEL array  22  and queries how many of the m 1  process candidates on SPAD array  24  have reported hits. 
     In a second comparison step  216 , controller  26  compares the relative number of hits (the ratio between the hits and the total number M of pulsed beams  30 ) to a second preset threshold. This latter threshold is typically set to a high value, corresponding to a situation in which the large majority of beams  30  are successfully received by corresponding super-pixels. If the relative number of hits is less than the second preset threshold, controller  26  adjusts the model in modeling step  210  based on the detected hits. The adjustment of the model includes recalculating the model coefficients, as well as, where required, an increase in the complexity of the model. In an iterative process, of 5-8 loops, for example, controller  26  adds new process candidates based on the model in candidate addition step  212 , queries hits in second detection step  214 , and compares their relative number to the second preset threshold in second comparison step  216 . As long as the relative number of hits does not exceed the second preset threshold, controller  26  keeps looping back to model step  210 , improving the model based on the new hits. 
     Once the relative number of detected hits exceeds the second preset threshold, controller  26  finishes the search and assigns, in an assignment step  218 , the detected hits for use in 3D mapping of scene  32  by depth mapping system  20 . 
     In case the number of process candidates at step  214  does not increase at a given stage of the iteration and is still too low, controller  26  may initiate a single-pixel offset search in an offset search step  222 . In offset search step  222 , a search for the yet undetected laser spots is performed with a single-pixel offset around their expected locations. 
     It will be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.

Metadata:
Filing Date: 20190806
Publication Date: 20210323
Grant Date: 20210323
Priority Date: 20190211
Inventors: ROTH, ZEEV
SHOVAL, Mendy
LAIFENFELD, MOSHE
ZINO, Sagy
OGGIER, THIERRY
Assignee: APPLE INC
CPC Classifications: [{"code": "H10F30/225", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/894", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/497", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4863", "inventive": true, "first": true, "tree": "[]"}, {"code": "G01S7/484", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4817", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4816", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4815", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4865", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/486", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/89", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01B11/22", "inventive": true, "first": true, "tree": "[]"}, {"code": "G01S17/10", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4815", "inventive": true, "first": true, "tree": "[]"}, {"code": "G01S7/4817", "inventive": true, "first": false, "tree": "[]"}, {"code": "H01S5/183", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/497", "inventive": true, "first": false, "tree": "[]"}, {"code": "H01S5/32", "inventive": false, "first": false, "tree": "[]"}, {"code": "G01S17/89", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/484", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4816", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S7/4865", "inventive": true, "first": false, "tree": "[]"}, {"code": "H01S5/32", "inventive": false, "first": false, "tree": "[]"}, {"code": "G01B11/22", "inventive": true, "first": true, "tree": "[]"}, {"code": "H01S5/183", "inventive": true, "first": false, "tree": "[]"}, {"code": "H01L31/107", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/89", "inventive": true, "first": false, "tree": "[]"}, {"code": "G01S17/10", "inventive": true, "first": false, "tree": "[]"}]
Family ID: 67777399