3D active depth sensing with laser pulse train bursts and a gated sensor

This disclosure provides systems, methods, and apparatuses for sensing a scene. In one aspect, a device may illuminate the scene using a sequence of two or more periods. Each period may include a transmission portion during which a plurality of light pulses are emitted onto the scene. Each period may include a non-transmission portion corresponding to an absence of emitted light. The device may receive, during each transmission portion, a plurality of light pulses reflected from the scene. The device may continuously accumulate photoelectric charge indicative of the received light pulses during an entirety of the sequence. The device may transfer the accumulated photoelectric charge to a readout circuit after an end of the sequence.

CROSS-REFERENCE TO RELATED APPLICATIONS

This Patent Application is related to the following United States utility patents: U.S. Pat. No. 9,774,801 B2, entitled “SOLID STATE IMAGE SENSOR WITH ENHANCED CHARGE CAPACITY AND DYNAMIC RANGE,” filed on Dec. 5, 2014, and assigned to the assignee hereof; and U.S. Pat. No. 9,332,200 B1, entitled “PIXEL READOUT ARCHITECTURE FOR FULL WELL CAPACITY EXTENSION,” filed on Dec. 5, 2014, and assigned to the assignee hereof. The disclosure of these patents are considered part of and are incorporated by reference in this Patent Application.

TECHNICAL FIELD

This disclosure relates generally to depth sensing systems and specifically to improving the speed and accuracy with which active depth systems generate depth information.

DESCRIPTION OF THE RELATED TECHNOLOGY

A device may determine distances of objects or surfaces in a scene using various active-or-passive depth sensing techniques. Passive depth sensing systems determine distances to objects in a scene based on ambient light reflected from the objects. Active depth sensing systems determine distances to objects in a scene by emitting pulses of light into a scene and analyzing corresponding light pulses reflected from objects in the scene. Some active depth sensing systems also may determine distances to objects in a scene by projecting a structured light (SL) pattern onto a scene and analyzing changes of the SL pattern reflected from objects in the scene. Active depth sensing systems typically employ either time-of-flight (ToF) techniques or SL techniques.

Some active depth sensing systems use an array of vertical cavity surface emitting lasers (VCSELs) that each emit a point of light (or “dot”) through a diffractive optical element (DOE). The DOE is often a grating-type with a periodic structure to diffract the light emitted from the VCSELs so as to form 2D array patterns (or “dot patterns”) on the scene. Some systems may then “stitch” the 2D arrays to form a pseudo-3D structured light pattern. The inherent pincushion distortion associated with such stitching may limit the resolution of the resultant image.

To eliminate this distortion and achieve higher-resolution images, an active depth sensing system may use a single DFB laser, rather than a VCSEL array, to emit a single point of light (or “dot”) through a coded DOE (rather than a grating-type DOE). The coded DOE can shape and split the single point of light so as to project a distribution of light (or “dot pattern”) onto the scene. At continuous wave or long pulse (e.g., on the order of milliseconds) operation, the light emitted from one DFB laser may have a lower power (and thus be less bright) than the light emitted from a VCSEL array, and may therefore be more susceptible to interference from ambient light (such as sunlight) than VCSEL arrays.

SUMMARY

One innovative aspect of the subject matter described in this disclosure can be used as a method of sensing a scene. In some implementations, the method can include illuminating the scene using a sequence of two or more periods, each period including a transmission portion during which a plurality of light pulses are emitted onto the scene and including a non-transmission portion corresponding to an absence of emitted light. The method also can include receiving, during each transmission portion, a plurality of light pulses reflected from the scene, and continuously accumulating photoelectric charge indicative of the received light pulses during an entirety of the sequence. The method also can include transferring the accumulated photoelectric charge to a readout circuit after an end of the sequence.

Another innovative aspect of the subject matter described in this disclosure can be implemented in an apparatus. In some implementations, the apparatus can include a memory and a processor coupled to the memory. The processor can be configured to illuminate the scene using a sequence of two or more periods, each period including a transmission portion during which a plurality of light pulses are emitted onto the scene and including a non-transmission portion corresponding to an absence of emitted light. The processor also can be configured to receive, during each transmission portion, a plurality of light pulses reflected from the scene and to continuously accumulate photoelectric charge indicative of the received light pulses during an entirety of the sequence. The processor also can be configured to transfer the accumulated photoelectric charge to a readout circuit after an end of the sequence.

Another innovative aspect of the subject matter described in this disclosure can be implemented in a device. The device can include means for illuminating a scene using a sequence of two or more periods, each period including a transmission portion during which a plurality of light pulses are emitted onto the scene and including a non-transmission portion corresponding to an absence of emitted light. The device also can include means for receiving, during each transmission portion, a plurality of light pulses reflected from the scene, and means for continuously accumulating photoelectric charge indicative of the received light pulses during an entirety of the sequence. The device also can include means for transferring the accumulated photoelectric charge to a readout circuit after an end of the sequence.

DETAILED DESCRIPTION

The following description is directed to certain implementations for the purposes of describing the innovative aspects of this disclosure. However, a person having ordinary skill in the art will readily recognize that the teachings herein can be applied in a multitude of different ways. Aspects of the present disclosure relate to light projectors and include a 3D active depth sensing system that emits light in laser pulse train bursts and receives reflections of the emitted light at a gated sensor.

An active depth sensing system may emit light in a predefined distribution of points (or another suitable shape of focused light) into a scene, and the reflected light may be received by the active depth sensing system. Depths of objects in the scene may be determined by comparing the distribution of the received light and the distribution of the emitted light. In comparing the distributions, a portion of the predefined distribution for the emitted light may be identified in the received light. In the present disclosure, an active depth sensing system that projects a distribution of light (e.g., structured light (SL), such as a distribution of light points, a flood light, and/or other shapes) is referred to as a SL system (with a SL projector).

Denser distributions of light (such as additional light points or more instances of focused light in an area than for sparser distributions of light) may result in a higher resolution of a depth map or a greater number of depths that may be determined. However, the intensity of individual light points are lower for denser distributions than for sparser distributions, and thus denser distributions of light may be more susceptible to interference from ambient light than sparser distributions of light. Thus, a sparser distribution may be more suitable for daylight scenes (with more interference), and a denser distribution may be more suitable for indoor or nighttime scenes (with less interference).

Many devices use a SL system in different types of lighting (with different amounts of interference). For example, a smartphone may include an active depth sensing system for face recognition, and the smartphone may be used indoors and outdoors. Many devices also include a flood illuminator. A flood illuminator may project a diffuse light onto a scene so that enough light exists in the scene for an image sensor to capture one or more images of the scene. In one example, a device (such as a smartphone) that performs face recognition may first determine if a face to be recognized (and/or identified) exists in the scene. In some implementations, the device may capture a two-dimensional (2D) image using flood illumination and then use the 2D image in conjunction with a three-dimensional (3D) image to recognize (and/or identify) a face (if any) in the image. Specifically, a light projector of the device may include a flood illuminator to project IR light onto a scene so that an IR sensor may capture the scene, and the device may determine from the capture if a face exists in the scene. If a face is determined to exist in the scene, the device may then use an active depth sensing system (e.g., via one or more light projectors) for face recognition and/or liveness confirmation. In some implementations, the device may use a proximity sensor to determine whether a face is present in the scene. In some aspects, a user of the device may determine when to turn on the light projector. In some other aspects, the device may be configured to turn on the projector automatically. In these ways, the device may use a 2D image (e.g., captured via flood illumination) in conjunction with a 3D image to enhance the performance of the device.

As discussed above, an active depth sensing system may use a single DFB laser to emit a single point of light through a coded DOE, which may reduce distortion and achieve higher-resolution images as compared with a VCSEL array. However, due to a damage threshold for the DFB laser, at continuous wave or long pulse (e.g., on the order of milliseconds) operation, the light emitted from the DFB laser may have a lower power (and thus be less bright) than the light emitted from the VCSEL array, and may therefore be more susceptible to ambient light (such as sunlight).

Rather than emitting light in a continuous wave long pulse light, aspects of the present disclosure describe a device including a DFB laser configured to output a sequence of relatively short bursts of light pulses (a “pulse burst”) that are each followed by a relatively long cooling period. Each pulse burst may include a small number of short light pulses. By repeating this pulse train burst and cooling cycle enough times, the device described herein may enable the single DFB laser to safely operate at a high enough power to illuminate a scene with brightness comparable to an array of VCSELs. Thus, SNR degradation from ambient light (i.e., noise) for the device may be reduced. In this way, aspects of the present disclosure may be used to enable, or otherwise improve, active depth sensing applications, such as, and not limited to: facial lock, facial recognition, facial modeling, animated emoji, user avatars, video conferencing, 3D modeling (such as automotive 3D modeling), 3D modeling for augmented reality (AR), gesture recognition, macro room imaging, among other appropriate applications. As one non-limiting example, aspects of the present disclosure may enable a mobile phone to capture a face of a user of the mobile phone (e.g., to enable the user to make a mobile payment) even while the user and the mobile phone are in bright conditions outdoors.

In the following description, numerous specific details are set forth, such as examples of specific components, circuits, and processes to provide a thorough understanding of the present disclosure. The term “coupled” as used herein means connected directly to or connected through one or more intervening components or circuits. Also, in the following description and for purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that these specific details may not be required to practice the teachings disclosed herein. In other instances, well-known circuits and devices are shown in block diagram form to avoid obscuring teachings of the present disclosure. Some portions of the detailed descriptions which follow are presented in terms of procedures, logic blocks, processes, and other symbolic representations of operations on data bits within a computer memory. In the present disclosure, a procedure, logic block, process, or the like, is conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system.

Aspects of the present disclosure are applicable to any suitable electronic device (such as security systems, smartphones, tablets, laptop computers, vehicles, drones, or other devices) including or coupled to one or more active depth sensing systems. While described below with respect to a device having or coupled to one light projector, aspects of the present disclosure are applicable to devices having any number of light projectors, and are therefore not limited to specific devices.

The term “device” is not limited to one or a specific number of physical objects (such as one smartphone, one controller, one processing system and so on). As used herein, a device may be any electronic device with one or more parts that may implement at least some portions of this disclosure. While the below description and examples use the term “device” to describe various aspects of this disclosure, the term “device” is not limited to a specific configuration, type, or number of objects. Additionally, the term “system” is not limited to multiple components or specific implementations. For example, a system may be implemented on one or more printed circuit boards or other substrates, and may have movable or static components. While the below description and examples use the term “system” to describe various aspects of this disclosure, the term “system” is not limited to a specific configuration, type, or number of objects.

FIG. 1shows a block diagram of an example SL system100. A SL system may emit light in a distribution of points (or another suitable shape of focused light). For purposes of discussion herein, the distribution of points may be referred to as a “pattern,” a “SL pattern,” a “dot pattern,” or the like, and the pattern may be predefined or random. The points of light may be projected on to a scene, and the reflections of the points of light may be received by the SL system. Depths of objects in a scene may be determined by comparing the pattern of the received light and the pattern of the emitted light. In comparing the patterns, a portion of the predefined distribution for the emitted light may be identified in the received light. A SL system may project a distribution of light (such as a distribution of light points or other shapes) using a SL projector.

The SL system100may be used to generate depth information for a scene106. For example, the scene106may include a face, and the SL system100may be used for identifying or authenticating the face. The SL system100may include a projector102and a receiver108. The projector102may be referred to as a “transmitter,” “projector,” “emitter,” and so on, and should not be limited to a specific transmission component. Throughout the following disclosure, the terms emitter, projector and transmitter may be used interchangeably. The receiver108may be referred to as a “detector,” “sensor,” “sensing element,” “photodetector,” and so on, and should not be limited to a specific receiving component.

While the disclosure refers to the distribution as a light distribution, any suitable signals at other frequencies may be used (such as radio frequency waves, sound waves, etc.). Further, while the disclosure refers to the distribution as including a plurality of light points, the light may be focused into any suitable size and dimensions. For example, the light may be projected in lines, squares, or any other suitable dimension. In addition, the disclosure may refer to the distribution as a codeword distribution, where a defined portion of the distribution (such as a predefined patch of light points) is referred to as a codeword. If the distribution of the light points is known, the codewords of the distribution may be known. However, the distribution may be organized in any way, and the present disclosure should not be limited to a specific type of distribution or type of signal or pulse.

The projector102may be configured to project or emit a distribution104of light points onto the scene106. The white circles in the distribution104may indicate where no light is projected for a possible point location, and the black circles in the distribution104may indicate where light is projected for a possible point location. In some example implementations, the projector102may include one or more light sources124(such as one or more lasers), a lens126, and a light modulator128. The projector102also may include an aperture122from which the emitted light escapes the projector102. In some implementations, the projector102may further include a diffractive optical element (DOE) to diffract the emissions from one or more light sources124into additional emissions. In some aspects, the light modulator128(to adjust the intensity of the emission) may comprise a DOE. In projecting the distribution104of light points onto the scene106, the projector102may emit one or more lasers from the light source124through the lens126(and/or through a DOE or light modulator128) and onto the scene106. The projector102may be positioned on the same reference plane as the receiver108, and the projector102and the receiver108may be separated by a distance called the baseline (112).

In some example implementations, the light projected by the projector102may be IR light. IR light may include portions of the visible light spectrum and/or portions of the light spectrum that is not visible to the naked eye. In one example, IR light may include near infrared (NIR) light, which may or may not include light within the visible light spectrum, and/or IR light (such as far infrared (FIR) light) which is outside the visible light spectrum. The term IR light should not be limited to light having a specific wavelength in or near the wavelength range of IR light. Further, IR light is provided as an example emission from the projector102. In the following description, other suitable wavelengths of light may be used. For example, light in portions of the visible light spectrum outside the IR light wavelength range or ultraviolet light. Alternatively, other signals with different wavelengths may be used, such as microwaves, radio frequency signals, and other suitable signals.

The scene106may include objects at different depths from the SL system (such as from the projector102and the receiver108). For example, objects106A and106B in the scene106may be at different depths. The receiver108may be configured to receive, from the scene106, reflections110of the emitted distribution104of light points. To receive the reflections110, the receiver108may capture an image. When capturing the image, the receiver108may receive the reflections110, as well as (i) other reflections of the distribution104of light points from other portions of the scene106at different depths and (ii) ambient light. Noise also may exist in the captured image.

In some example implementations, the receiver108may include a lens130to focus or direct the received light (including the reflections110from the objects106A and106B) on to the sensor132of the receiver108. The receiver108also may include an aperture120. Assuming for the example that only the reflections110are received, depths of the objects106A and106B may be determined based on the baseline112, displacement and distortion of the light distribution104(such as in codewords) in the reflections110, and intensities of the reflections110. For example, the distance134along the sensor132from location116to the center114may be used in determining a depth of the object106B in the scene106. Similarly, the distance136along the sensor132from location118to the center114may be used in determining a depth of the object106A in the scene106. The distance along the sensor132may be measured in terms of number of pixels of the sensor132or a distance (such as millimeters).

In some example implementations, the sensor132may include an array of photodiodes (such as avalanche photodiodes) for capturing an image. To capture the image, each photodiode in the array may capture the light that hits the photodiode and may provide a value indicating the intensity of the light (a capture value). The image therefore may be the capture values provided by the array of photodiodes.

In addition, or in the alternative, the sensor132may include a complementary metal-oxide semiconductor (CMOS) sensor. To capture the image by a photosensitive CMOS sensor, each pixel of the sensor may capture the light that hits the pixel and may provide a value indicating the intensity of the light. In some example implementations, an array of photodiodes may be coupled to the CMOS sensor. In this manner, the electrical impulses generated by the array of photodiodes may trigger the corresponding pixels of the CMOS sensor to provide capture values.

The sensor132may include at least a number of pixels equal to the number of possible light points in the distribution104. For example, the array of photodiodes or the CMOS sensor may include a number of photodiodes or a number of pixels, respectively, corresponding to the number of possible light points in the distribution104. The sensor132logically may be divided into groups of pixels or photodiodes (such as 4×4 groups) that correspond to a size of a bit of a codeword. The group of pixels or photodiodes also may be referred to as a bit, and the portion of the captured image from a bit of the sensor132also may be referred to as a bit. In some example implementations, the sensor132may include the same number of bits as the distribution104.

If the light source124emits IR light (such as NIR light at a wavelength of, e.g., 940 nm), the sensor132may be an IR sensor to receive the reflections of the NIR light. The sensor132also may be configured to capture an image using a flood illuminator (not shown for simplicity). As illustrated, the distance134(corresponding to the reflections110from the object106B) is less than the distance136(corresponding to the reflections110from the object106A). Using triangulation based on the baseline112and the distances134and136, the differing depths of objects106A and106B in the scene106may be determined in generating depth information for the scene106. Determining the depths may further include determining a displacement or a distortion of the distribution104in the reflections110.

Although a number of separate components are illustrated inFIG. 1, one or more of the components may be implemented together or include additional functionality. All described components may not be required for a SL system100, or the functionality of components may be separated into separate components. Additional components not illustrated also may exist. For example, the receiver108may include a bandpass filter to allow signals having a determined range of wavelengths to pass onto the sensor132(thus filtering out signals with a wavelength outside of the range). In this manner, some incidental signals (such as ambient light) may be prevented from interfering with the captures by the sensor132. The range of the bandpass filter may be centered at the transmission wavelength for the projector102. For example, if the projector102is configured to emit NIR light with a wavelength of 940 nm, the receiver108may include a bandpass filter configured to allow NIR light having wavelengths within a range of, e.g., 920 nm to 960 nm. Therefore, the examples described regardingFIG. 1are for illustrative purposes, and the present disclosure should not be limited to the example SL system100.

For a light projector (such as the projector102), the light source may be any suitable light source. In some example implementations, the light source124may include one or more distributed feedback (DFB) lasers. In some other example implementations, the light source124may include one or more vertical-cavity surface-emitting lasers (VCSELs).

A DOE is a material situated in the projection path of the light from the light source. The DOE may be configured to split a light point into multiple light points. For example, the material of the DOE may be a translucent or a transparent polymer with a known refractive index. The surface of the DOE may include peaks and valleys (varying the depth of the DOE) so that a light point splits into multiple light points when the light passes through the DOE. For example, the DOE may be configured to receive one or more lights points from one or more lasers and project an intended distribution with a greater number of light points than emitted by the one or more lasers. While the Figures may illustrate the depth of a DOE changing along only one axis of the DOE, the Figures are only to assist in describing aspects of the disclosure. The peaks and valleys of the surface of the DOE may be located at any portion of the surface of the DOE and cause any suitable change in the depth of portions of the DOE, and the present disclosure should not be limited to a specific surface configuration for a DOE.

If the light source124includes an array of lasers (such as a VCSEL array), a portion of the distribution of light points may be projected by the array. A DOE may be used to replicate the portion in projecting the distribution of light points. For example, the DOE may split the projection from the array into multiple instances, and a pattern of the projection may be a repetition of the projection from the array. In some example implementations, the DOE may be configured to repeat the projection vertically, horizontally, or at an angle between vertical and horizontal relative to the projection. The repeated instances may be overlapping, non-overlapping, or any suitable configuration. While the examples describe a DOE configured to split the projection from the array and stack the instances above and below one another, the present disclosure should not be limited to a specific type of DOE configuration and repetition of the projection.

FIG. 2shows a block diagram of an example device200within which aspects of the present disclosure may be implemented. The device200may include or be coupled to an emitter201, a sensor202, a processor204, a memory206storing instructions208, and an active depth controller210(which may include one or more signal processors212). The emitter201may include or be coupled to a diffractive optical element (DOE)205, and may include or be coupled to a diffuser207. In some implementations, the emitter201may include multiple projectors. In addition, or in the alternative, the active depth sensing system may include a flood illuminator component separate from the emitter201. For purposes of discussion herein, the device200may be referred to as a “SL system” or an “active depth sensing system.” Further for purposes of discussion herein, the “active depth sensing system” may instead refer to just one or more components of the device200, such as the active depth controller210, the emitter201, the sensor202, the processor204, and/or any other appropriate components.

In some implementations, the emitter201may be a distributed feedback (DFB) laser for emitting light pulses onto the scene. The DOE205may enable the emitter201to emit a distribution of light such as a known DOE dot pattern, a codeword DOE projection, a random dot projection or distribution, or the like. The diffuser207may transition the device200between one or more operating modes. For example, in a transmission mode, the diffuser207may be switched on to allow the emitter201to emit light pulses into a scene, and in a non-transmission mode, the diffuser207may be switched off to prevent the emitter201from emitting light pulses into the scene.

In some implementations, the sensor202may be a gated, global shutter (GS) sensor configured to receive reflected light pulses from the scene. In some aspects, the sensor202may be a composite CMOS image sensor. In some aspects, the sensor202may be based on a monolithic pixel array architecture, for example, with Time-Division Multiplexed Read (TDMR) capabilities.

In some implementations, the active depth controller210may be a computation element for calculating depth information. In some aspects, the active depth controller210may be configured to control (or otherwise operate) one or both of the emitter201and the sensor202. In some aspects, the active depth controller210may be controlled, work in conjunction with, or otherwise be operated by one or more other components of the device200, such as the processor204and/or the memory206.

The device200may optionally include or be coupled to a display214and a number of input/output (I/O) components216. The sensor202may be, or otherwise may be coupled to, a camera, such as a single camera, a dual camera module, or a module with any number of other camera sensors (not shown for simplicity). The signal processor212may be configured to process captures from the sensor202. The device200may further include one or more optional sensors220(such as a gyroscope, magnetometer, inertial sensor, NIR sensor, and so on) coupled to the processor204. The device200also may include a power supply218, which may be coupled to or integrated into the device200. The device200may include additional features or components not shown.

The memory206may be a non-transient or non-transitory computer readable medium storing computer-executable instructions208to perform all or a portion of one or more operations described in this disclosure. The processor204may be one or more suitable processors capable of executing scripts or instructions of one or more software programs (such as instructions208) stored within the memory206. In some aspects, the processor204may be one or more general purpose processors that execute instructions208to cause the device200to perform any number of functions or operations. In additional or alternative aspects, the processor204may include integrated circuits or other hardware to perform functions or operations without the use of software. While shown to be coupled to each other via the processor204in the example ofFIG. 2, the processor204, the memory206, the active depth controller210, the optional display214, the optional I/O components216, and the optional sensors220may be coupled to one another in various arrangements. For example, the processor204, the memory206, the active depth controller210, the optional display214, the optional I/O components216, and/or the optional sensors220may be coupled to each other via one or more local buses (not shown for simplicity).

The display214may be any suitable display or screen allowing for user interaction and/or to present items (such as depth information or a preview image of the scene) for viewing by a user. In some aspects, the display214may be a touch-sensitive display. The I/O components216may be or include any suitable mechanism, interface, or device to receive input (such as commands) from the user and to provide output to the user. For example, the I/O components216may include (but are not limited to) a graphical user interface, keyboard, mouse, microphone and speakers, squeezable bezel or border of the device200, physical buttons located on device200, and so on. The display214and/or the I/O components216may provide a preview image or depth information for the scene to a user and/or receive a user input for adjusting one or more settings of the device200(such as adjusting an intensity of emissions by emitter201, determining or switching one or more operating modes of the device200, adjusting a field of emission of the emitter201, and so on).

The active depth controller210also may include, or may otherwise be coupled to, a signal processor212, which may be one or more processors to process captures from the sensor202. The active depth controller210may alternatively or additionally include a combination of specific hardware and the ability to execute software instructions.

The emitter201may vary its field of emission for different operating modes. In some example implementations, the emitter201may include a focusing apparatus for adjusting the size of the field of emission/transmission. In one example, mirrors attached to actuators (such as microelectromechanical systems (MEMS) actuators) may adjust a focus of the light emissions from the emitter201. In another example, an adjustable holographic optical element (HOE) may adjust the focus of the light emissions from the emitter201. In a further example, a formable diffractive optical element (DOE) (such as a piezoelectric material to adjust the shape) may be adjusted to focus the diffracted points of light emitted.

FIG. 3shows an example device300including an active depth sensing light projector. In some implementations, the device300may be one example of the device200ofFIG. 2. The device300further may include an IR sensor306to capture an image based on the reflections of light emitted from the active depth sensing light projector302or the flood illuminator304(with the active depth sensing light projector302and the illuminator304projecting IR light). In some example implementations, the active depth sensing light projector302may include one or more DFBs for emitting light pulses onto a scene. In some example implementations, the IR sensor306may be a gated global shutter (GS) sensor for receiving light pulses reflected from the scene. The active depth sensing light projector302and the IR sensor306may be separated by a baseline308.

An example device300may be a smartphone, with an earpiece310and a microphone312for conducting phone calls or other wireless communications. A smartphone also may include a display314with or without a notch including the active depth sensing light projectors302, illuminator304, and the IR sensor306. A flood illuminator304may project a diffuse IR light onto a scene for the IR sensor306to capture an image based on reflections of the diffuse IR light.

FIG. 4shows a timing diagram400depicting an example operation for sensing a scene. The example operation may be performed by a device (not shown for simplicity), such as the device300ofFIG. 3, the device200ofFIG. 2, or any suitable device. In some implementations, the device may be a mobile phone.

At time t0, the device may begin a frame exposure401. During the frame exposure401, the device may illuminate the scene using a sequence of two or more periods. As a non-limiting example, a first period is shown that starts at time to and ends at time t2, and a second period is shown that starts at time t2and ends at time t4. Each of the two or more periods may include a transmission portion and a non-transmission portion. For purposes of discussion herein, each transmission portion may be referred to as a “pulse burst,” and each non-transmission portion may be referred to as a “cooling period.”

During each transmission portion of the sequence, such as from time t2to time t3, the device may emit a plurality of light pulses onto the scene. Each of the light pulses may have a duration, which is shown as a time difference, tB−tA. In some implementations, the device may include one or more lasers (such as the light projector302ofFIG. 3) for emitting the plurality of light pulses. In some aspects, one or more of the lasers may be a single-mode DFB laser, and each of the plurality of emitted light pulses may be generated by the single-mode DFB laser. The device also may enable one or more sensors, such as the IR sensor306ofFIG. 3, during each of the transmission portions. The one or more sensors may receive a plurality of light pulses reflected from the scene during each of the transmission portions. In some example implementations, the received light pulses may be reflected from a face of a user of the device. In some aspects, the device may identify or authenticate the face of the user based on the received light pulses.

The sequence includes a number of non-transmission portions between pulse bursts. As a non-limiting example, a first non-transmission portion is shown from time t1to time t2. Each of the non-transmission portions may correspond to an absence of emitted light (such that the laser does not emit light pulses), for example, so that an operating temperature of the laser may decrease between pulse bursts. The device may prevent or disable the accumulation of photoelectric charge during each non-transmission portion, for example, by disabling the one or more sensors. In this manner, the device may prevent the one or more sensors from receiving ambient light (which may degrade SNR) during the non-transmission portions.

In some implementations, the one or more lasers may emit each of the light pulses at a selected transmit power level that exceeds a maximum specified power level for continuous or long pulse operation of the laser. As the one or more lasers emit the light pulses during the transmission portions, a temperature of the one or more lasers may approach or exceed a specified temperature value at which the one or more lasers may incur physical damage. A duration of each of the non-transmission portions may be selected to prevent damage to the one or more lasers, such as from the operating temperature of the one or more lasers being greater than the specified temperature value during a corresponding one of the transmission portions, by allowing the operating temperature of the one or more lasers to decrease to a level below the specified temperature value. In this manner, each of the non-transmission portions of the sequence may provide a cooling period for a preceding transmissions portion of the sequence. In some aspects, the duration of each non-transmission portion of the sequence may be an order of magnitude greater than the duration of each corresponding transmission portion of the sequence.

Since a gating duration for a traditional single DFB laser may be on the order of approximately tens of nanoseconds, a traditional GS sensor may not be capable of reliably turning on and off during emission that synchronizes with a given light pulse from the DFB laser. Aspects of the present disclosure may enable the GS sensor to be gated and synchronized with each pulse burst from the single DFB laser, for example, so that the device receives reflected light pulses during each transmission portion of the sequence and does not receive any or most of the light (for example, ambient light) during each non-transmission portion of the sequence. As one non-limiting example, the duration of each transmission portion may be approximately 1 μs, and the duration of each non-transmission portion may be approximately 10 μs. Each transmission portion may include approximately 25 light pulses, and the duration (or “pulse-width”) of each light pulse may be approximately 20 ns in an appropriate (relatively high) duty cycle, such as approximately 50%. In some aspects, the duty cycle also may adhere to a heat characteristic of the DFB laser. In some aspects, the light pulses may be emitted at a current of approximately 2-3 A. The DFB laser may be configured to emit a number of bursts (approximately 800-3200 in this example) sufficient to saturate the image. In some implementations, the number of pulses in each pulse burst and the number of pulse bursts may be dynamically adjusted based on system conditions, such as a measured SNR,

μsigσb⁢g,
where μsigrepresents the laser signal, and where σbgrepresents the ambient light noise. SNR may be higher when the device is indoors than when the device is exposed to sunlight outdoors. Thus, in some implementations, the device may dynamically adjust at least one of a number of the transmission portions and a number of the plurality of light pulses based on environmental light conditions for the device. For example, the device may dynamically reduce the number of light pulses or the number of transmission portions when the device is indoors (or in other low-light conditions). As another example, the device may dynamically increase the number of light pulses or the number of transmission portions when the device is outdoors (or in other bright-light conditions).

As a non-limiting example, the sequence may include approximately 500-2000 periods, where each period includes a transmission portion and a non-transmission portion. In part by switching the GS sensor on at the beginning of each pulse burst (such as at time t0) and switching the GS sensor off at the end of each pulse burst (such as at time t1), aspects of the present disclosure may allow the single DFB laser to safely operate at a high enough power to illuminate a scene with brightness comparable to an array of VCSELs (e.g., approximately 2-3 W).

The sequence of periods for the frame exposure401is shown to include a number, n, of additional periods. Each of the n additional periods may include a corresponding transmission portion and a corresponding non-transmission portion (not shown for simplicity). During an entirety of the sequence (e.g., beginning at time t0), the device may continuously accumulate photoelectric charge indicative of the received light pulses. For example, each of the one or more sensors may be a photodiode configured to continuously receive photons from the received light pulses during each transmission portion of the sequence. The device may be configured to integrate the photoelectric charge to one or more storage nodes of the device, such as the memory206ofFIG. 2. In some aspects, one or more of the storage nodes may be a charge-coupled device (CCD)-type of storage node (e.g., a storage capacitor). The sequence may continue until enough photoelectrons have been accumulated to form a single image frame, such as at time tn+1. In some implementations, the device may integrate the photoelectric charge in a manner similar to the GS pixel architecture disclosed in U.S. Pat. No. 9,332,200 B1.

The sequence (and the frame exposure401) may end after a final transmission portion (the “final pulse burst”) that starts at time tnand ends at time tn+1. In some implementations, the final transmission portion may not be followed by a corresponding non-transmission portion.

After the end of the sequence (e.g., at time tn+1), the device may transfer (or “extract”) the accumulated photoelectric charge to a readout circuit (not shown for simplicity). In some implementations, the device may readout the photoelectric charge in a manner similar to the GS pixel architecture disclosed in U.S. Pat. No. 9,332,200 B1. In some example implementations, the device may perform a number, m, of additional frame exposures after the frame exposure401. The device may operate the same or similar to the frame exposure401during each of the m additional frame exposures.

FIG. 5shows an example system500for sensing a scene. The example system500is shown to include a DFB, a GS Sensor, a Laser Drive, and a System Drive. The DFB may be an example implementation of the one or more lasers (e.g., a single DFB laser) described with respect toFIG. 4. The GS Sensor may be an example implementation of the one or more sensors (e.g., a GS sensor) described with respect toFIG. 4. The System Drive may be configured to turn the Laser Drive on by sending “On” triggers to the Laser Drive. When the Laser Drive is on, it may send laser driving pulses to the DFB, and the DFB may emit corresponding light pulses in a sequence of laser pulse bursts, as described with respect toFIG. 4. The System Drive also may be configured to enable the GS Sensor by sending an “On” trigger to the GS Sensor. The System Drive also may optionally send an “Off” trigger to switch the GS sensor off at the end of each laser pulse burst. In addition, or in the alternative, the GS sensor may be timed to turn off after a particular duration of the length of the pulse burst, such as approximately 1 μs. The System Drive may send the same system “On” trigger simultaneously (as indicated by Δt) to each of the Laser Drive and the GS Sensor. Thus, at the beginning of each laser pulse burst, the GS sensor may be triggered and turned on. As discussed with respect toFIG. 4, the DFB may thus safely emit the pulse train at a power level sufficient to illuminate the scene as brightly as an array of VCSELs, and at the same time, the system may minimize SNR degradation from ambient light noise by gating the GS sensor to synchronize with the on time for each burst of the pulse train.

FIG. 6shows a timing diagram600depicting an example operation for controlling an active depth sensing system. The example operation may be performed by a device (not shown for simplicity), such as the device300ofFIG. 3, the device200ofFIG. 2, or any suitable device. In some implementations, the device may be a mobile phone. The timing diagram600may be similar to the timing diagram400ofFIG. 4. For example, a frame exposure601, a pulse burst610, a pulse burst620, a pulse burst690, a time t0, a time t1, a time t2, a time t3, a time t4, a time t5, a time tn, a time tn+1, and m additional frame exposures may be the same or similar to the frame exposure401, the first pulse burst, the second pulse burst, the third pulse burst, the final pulse burst, the time t0, the time t1, the time t2, the time t3, the time t4, the time t5, the time tn, the time tn+1, and the m additional frame exposures ofFIG. 4. “Laser Pulse Bursts” are shown that are the same or similar to the laser pulse bursts shown inFIG. 4.FIG. 6also shows corresponding “Laser Driving Pulse Bursts” that trigger the laser pulse bursts, as described with respect toFIG. 5. As described with respect toFIG. 4andFIG. 5, a GS sensor may be triggered to turn on in-sync with the beginning of the laser pulse bursts (e.g., via the simultaneous “On” triggers at time t0). The GS sensor also may optionally be triggered to turn off in-sync with the end of the laser pulse bursts (e.g., via the “Off” trigger at time tA).

FIG. 7shows an illustrative flowchart depicting an example operation700for sensing a scene. The example operation700is performed by a device, such as the device200ofFIG. 2. The device illuminates the scene using a sequence of two or more periods, each period including a transmission portion during which a plurality of light pulses are emitted onto the scene and including a non-transmission portion corresponding to an absence of emitted light (701). Each of the emitted light pulses is emitted from a single-mode distributed feedback (DFB) laser at a selected transmit power level. In some implementations, a duration of each non-transmission portion is based at least in part on the selected transmit power level. In addition, or in the alternative, the duration of each non-transmission portion is selected to prevent damage to the laser based on the operating temperature of the laser being greater than the temperature value during a corresponding one of the transmission portions. In some aspects, the selected transmit power level exceeds a maximum specified power level for continuous operation of the laser.

The device receives, during each transmission portion, a plurality of light pulses reflected from the scene (702). In some implementations, a sensor may continuously receive photons during each transmission portion of the sequence, and may be prevented from receiving ambient light during each non-transmission portion of the sequence.

The device continuously accumulates photoelectric charge indicative of the received light pulses during an entirety of the sequence (703). In some implementations, the device may pause or disable the accumulation of photoelectric charge during each non-transmission portion of the sequence.

The device transfers the accumulated photoelectric charge to a readout circuit after an end of the sequence (704). In some implementations, the transferred accumulated photoelectric charge may be used to construct an image of the scene. In some aspects, the image may be a face of a user to be identified or authenticated by a mobile device such as a smartphone.