Time-of-flight imaging systems

Electronic devices may include time-of-flight image pixels. A time-of-flight image pixel may include first and second charge storage regions coupled to a photosensor and a transfer transistor with a gate terminal coupled to the first storage region. An electronic device may further include a light pulse emitter configured to emit pulses of light to be reflected by objects in a scene. Reflected portions of the emitted pulses of light may be captured along with background light by the time-of-flight image pixels. Time-of-flight image pixels may be configured sense the time-of-flight of the reflected portions of the emitted pulses. The electronic device may include processing circuitry configured to use the sensed time-of-flight of the reflected portions to generate depth images of a scene. Depth images may include depth-image pixel values that contain information corresponding to the distance of the objects in the scene from the electronic device.

BACKGROUND

This relates generally to imaging devices, and more particularly, to imaging devices that measure the flight time of reflected light pulses.

Image sensors are commonly used in electronic devices such as cellular telephones, cameras, and computers to capture images. In a typical arrangement, an electronic device is provided with an image sensor and a corresponding lens. Some electronic devices use arrays of image sensors and corresponding lenses to gather image data. This type of system, which is sometimes referred to as an array camera, may be used to capture depth information from a scene using a parallax effect based on a known physical offset between image sensors.

Depth information such as information about the distance of an object from an electronic device is also commonly captured using a dedicated range finder such as a radar system. In a typical range finder, light of a known frequency is emitted from the range finder in the direction of an object and is reflected off of the object in the direction of the range finder. Range finders typically have a light sensor that detects the reflected light. Distance information is then determined based on the time-of-flight of the light between the emission and detection of the light and the known speed of light.

Time-of-flight distance information is also sometimes extracted by a range finder from an emitted and reflected pulse of light by synchronizing the emission and the detection of the light. The light sensor is often configured to collect light for a predetermined amount of time after the emission of the light. Light reflected from a far away object may not return during the light collection period while light reflected from a nearby object may return and be collected during the light collection period. This is because the light reflected from the far away object travels a longer distance and therefore has a longer time-of-flight. Closer objects therefore appear brighter than relatively further objects. Distance information is therefore extracted from the brightness of an object.

Range finding systems of this type may capture depth information to a relatively larger distance than can be determined using a typical array camera. However, range finding systems of this type typically collect distance information for a single point, not a collection of points as in an image. In addition, range finding systems that determine depth information based on the brightness of reflected light may be confused by the presence of differences in the intrinsic brightness of objects in a typical real-world scene. Difficulties that arise in separating background intensity from reflected light pulse intensity can therefore be problematic when capturing images with depth information.

It would therefore be desirable to be able to provide improved imaging devices for capturing depth images.

BRIEF SUMMARY OF THE INVENTION

Various embodiments are described, illustrating electronic devices that include time-of-flight image pixels configured to measure the time of flight of an emitted light pulse for sensing distance information about objects in a scene. Emitted light pulses may be generated by a light pulse emitter on the electronic device and reflected from objects in the field-of-view of the time-of-flight image pixels. Time-of-flight image pixels may be configured to measure differences in time-of-flight between reflected portions of emitted light pulses using differences in brightness of the reflected portions. Time-of-flight image sensors may be configured to remove background light contamination of reflected portions of emitted light pulses.

A time-of-flight image pixel may include a photosensitive element such as a photodiode, and first and second charge storage regions coupled to the photosensitive element. A time-of-flight image pixel may include a first transfer transistor coupled between the photosensitive element and the first charge storage region and a second transfer transistor coupled between the photosensitive element and the second charge storage region. The second transfer transistor may include a gate terminal that is coupled to the first charge storage region.

A time-of-flight image pixel may include a third transfer transistor having first and second source/drain terminals. The first source/drain terminal of the third transfer transistor may be connected to the gate terminal of the second transfer transistor and the second source/drain terminal of the third transfer transistor may be connected to the first charge storage region.

A time-of-flight image pixel may include a fourth transfer transistor having a first source/drain terminal that is coupled to the gate terminal of the second transfer transistor and a reset transistor having a first source/drain terminal that is coupled to the second charge storage region and a second source/drain terminal coupled to a source/drain terminal of a source follower transistor having a gate terminal connected to the second charge storage region. If desired, a time-of-flight image pixel may include an additional reset transistor having a first source/drain terminal that is coupled to the photosensitive element.

If desired, the time-of-flight image pixel may include a reset transistor having a first source/drain terminal that is coupled to the second charge storage region, a source follower transistor having a gate terminal connected to the second charge storage region, and a row select transistor coupled to the source follower transistor.

The electronic device may further include a light pulse emission component such as a non-visible light pulse emitter configured to emit pulses of non-visible light. The electronic device may include an array of image sensors. The array of image sensors may include a red image sensor, a blue image sensor, a green image sensor or other image sensors. Each of the image sensors in the array of image sensors may include an array of time-of-flight image pixels. Time-of-flight image pixels may be configured to collect background light and reflected portions of the emitted pulses of non-visible light and to store charges generated by the background light on the first charge storage region and to store charges generated by the reflected portions of the emitted pulses of non-visible light on the second charge storage region.

The electronic device may include processing circuitry configured to extract depth information from a depth-image signal generated by the time-of-flight image pixels. The processing circuitry may be configured to combine image data from the red image sensor, the blue image sensor, and the green image sensor to form a color image.

During operation of the electronic device, time-of-flight image pixels may be configured to convert background light into electric charges and to transfer the electric charges from the photosensitive element to the first charge storage region. A light pulse emitter may be configured to emit a pulse of non-visible light. Time-of-flight image pixels may be configured to convert additional background light and a reflected portion of the emitted pulse of non-visible light into additional electric charges and to transfer a portion of the additional electric charges (e.g., the portion corresponding to the reflected portion of the emitted pulse of non-visible light) to the second charge storage region. Transferring the portion of the additional electric charges may include connecting the gate terminal of the second transfer transistor to the first charge storage region on which the electric charges are stored by activating the fourth transfer transistor.

During operation, the photosensitive element may be reset to remove a remaining portion of the additional electric charges from the photosensitive element before a subsequent pulse of non-visible light may be emitted from the light pulse emitter. Time-of-flight image pixels may be configured to convert further additional background light and a reflected portion of the subsequent emitted pulse of non-visible light into further additional electric charges and to transfer a portion of the further additional electric charges (e.g., the portion corresponding to the reflected portion of the subsequent emitted pulse of non-visible light) to the second charge storage region on which the portion of the additional electric charges is stored.

Time-of-flight image pixels may be configured to convert the portion of the additional electric charges and the portion of the further additional electric charges into a depth-image signal. Processing circuitry may be used to extract distance information from the depth-image signal and to process the distance information to form a portion of a depth image that includes depth-image pixel values that correspond to the distance of an object to the electronic device.

DETAILED DESCRIPTION

Digital camera modules are widely used in electronic devices such as digital cameras, computers, cellular telephones, or other electronic devices. These electronic devices may include image sensors that gather incoming light to capture an image. The image sensors may include arrays of image pixels. The image sensors may include arrays of time-of-flight image pixels for sensing the flight time of a light pulse emitted by a non-visible-light emitting component of the electronic device and reflected from an object. Image sensors may, if desired, include both image pixels and time-of-flight image pixels. Image pixels and time-of-flight image pixels in the image sensors may include photosensitive elements such as photodiodes that convert the incoming light into electric charges.

Time-of-flight image sensor pixels may include one or more charge storage regions for storing charges collected using photosensitive elements. Time-of-flight image sensors may be configured to store charges generated by background image light from a scene separately from charges generated by reflected light that was emitted by a non-visible-light emitting component of the electronic device. Charges generated by reflected light that was emitted by a non-visible-light emitting component of an electronic device may be converted into depth-image data. The depth-image data may be processed to form depth images (i.e., images in which the image data in each pixel of the image represents the distance to the object in that pixel). Image sensors may have any number of pixels such as image pixels and/or time-of-flight image pixels (e.g., hundreds or thousands or more). A typical image sensor may, for example, have hundreds of thousands or millions of pixels (e.g., megapixels).

FIG. 1is a diagram of an illustrative electronic device that includes time-of-flight image pixels and a light pulse emitter for capturing depth images. Electronic device10ofFIG. 1may be a portable electronic device such as a camera, a cellular telephone, a video camera, or other imaging device that captures digital image data. Camera module12may be used to convert incoming light into digital image data. Camera module12may include an array of lenses14and a corresponding array of image sensors16. Lenses14and image sensors16may be mounted in a common package and may provide image data to control circuitry such as storage and processing circuitry18.

Electronic device10may include one or more light emitting components such as visible light source22(e.g., a camera flash, an LED light source, etc.) and a non-visible-light pulse emitter (e.g., an infrared laser, a radio pulse emitter, or other source non-visible light capable of generating pulses of non-visible light) such as non-visible light emitter20. Visible light source22may be used to light a real-world scene during capture of image data. Non-visible-light emitter20(sometimes called light pulse emitter, pulse emitter, infrared emitter, emitter, etc.) may be used to emit a pulse of, for example, infrared light. Light emitted by pulse emitter20may be reflected off of objects in a real-world scene and detected using image sensor array16of camera module12. Circuitry18may be used to extract depth information (e.g., information about the distance of objects in a scene) from detected, reflected portions of light emitted by pulse emitter20.

Storage and processing circuitry18may include one or more integrated circuits (e.g., image processing circuits, microprocessors, storage devices such as random-access memory and non-volatile memory, etc.) and may be implemented using components that are separate from camera module12and/or that form part of camera module12(e.g., circuits that form part of an integrated circuit that includes image sensors16or an integrated circuit within module12that is associated with image sensors16). Image data that has been captured by camera module12may be processed and stored using circuitry18. Processed image data may, if desired, be provided to external equipment (e.g., a computer or other device) using wired and/or wireless communications paths coupled to processing circuitry18.

As shown inFIG. 2, image sensor array16may contain an array of individual image sensors having image pixels such as image sensor pixels30. In the example ofFIG. 2, image sensor array16includes four image sensors16(1,1),16(1,2),16(2,1), and16(2,2). This is merely illustrative. In general, array16may have any suitable number of image sensors (e.g., one image sensor, two or more image sensors, three or more image sensors, four or more image sensors, ten or more sensors, 16 image sensors, 20 or more image sensors, etc.).

Image sensors such as image sensors16(1,1),16(1,2),16(2,1), and16(2,2) may each be configured to receive light of a given color by providing each image sensor with a color filter. The color filters that are used for image sensor pixel arrays in the image sensors may, for example, be red filters that pass red light, blue filters that pass blue light, green filters that pass green light, and infrared filters that pass infrared light. Each filter may form a color filter layer that covers the image sensor pixel array of a respective image sensor in the array. Other filters such as white color filters, dual-band IR cutoff filters (e.g., filters that allow visible light and a range of infrared light emitted by LED lights), etc. may also be used.

Image sensors such as image sensors16(1,1),16(1,2),16(2,1), and16(2,2) may be formed on one or more separate semiconductor substrates. With one suitable arrangement, which is sometimes described herein as an example, the image sensors are formed on a common semiconductor substrate (e.g., a common silicon image sensor integrated circuit die). Each image sensor may be identical. For example, each image sensor may be a Video Graphics Array (VGA) sensor with a resolution of 480×640 sensor pixels (as an example). Other types of image sensor may also be used for the image sensors if desired. For example, images sensors with greater than VGA resolution or less than VGA resolution may be used, image sensor arrays in which the image sensors are not all identical may be used, etc.

Image sensors such as image sensors16(1,1),16(1,2),16(2,1), and16(2,2) of camera module12may include one or more time-of-flight image sensors having time-of-flight image pixels such as time-of-flight image pixels32. A time-of-flight image sensor may be used to capture depth-image light for generating depth information about a real-world scene. Depth-image data may be captured in the form of electric charges generated by photosensors such as photodiodes in time-of-flight image pixels32. These depth-image charges may be generated by detected portions of light emitted by emitter20ofFIG. 1and reflected from objects in a real-world scene.

In one preferred embodiment that is sometimes described herein as an example, light emitted by emitter20may include infrared image light and a time-of-flight image sensor may be implemented using an infrared image sensor (e.g., an image sensor with an associated infrared color filter or an image sensor with infrared sensitive time-of-flight image pixels).

Image data such as red image data, blue image data, green image data, time-of-flight image data or other image data may be processed by processing circuitry18. Time-of-flight image data may be processed by circuitry18to extract depth information about a scene from the image data (e.g., the distance of an object imaged by each time-of-flight image pixel32in electronic device10).

Processing circuitry18(e.g., processing circuitry integrated onto sensor array integrated circuit16and/or processing circuitry on one or more associated integrated circuits) may use the relative brightness of detected, reflected image light to determine the distance to the object in the field-of-view of each time-of-flight image pixel32. Time-of-flight image pixels32in image sensor array16may include multiple charge storage regions configured to store charges associated with reflected portions of light that was generated by emitter20(seeFIG. 1) separately from charges generated by background light.

Processing circuitry18(e.g., processing circuitry integrated onto sensor array integrated circuit16and/or processing circuitry on one or more associated integrated circuits) may also combine color image data (e.g., red, green, blue or other color image data) with depth-image data to form a three-dimensional color image of a scene. In some modes of operation, all of the image sensors on array16may be active (e.g., when determining 3-dimensional image depth information). In other modes of operation (e.g., color imaging), only a subset of the image sensors may be used. Other sensors may be inactivated to conserve power (e.g., their positive power supply voltage terminals may be taken to a ground voltage or other suitable power-down voltage and their control circuits may be inactivated or bypassed).

If desired, camera module12may include a single image sensor array with time-of-flight image pixels32. If desired, camera module12may include one or more image sensor arrays each having a mix of conventional image pixels30and time-of-flight image pixels32. However, this is merely illustrative. If desired, each image sensor of image sensor array16may include exclusively conventional image pixels30or exclusively time-of-flight image pixels32.

Image sensor array16may also include circuitry such as support circuitry24(e.g., row select and control driver circuitry). Support circuitry24may be used to issue reset signals, row select signals, etc. for the image pixels30and time-of-flight image pixels32of image sensor pixel arrays such as image sensors16(1,1),16(1,2),16(2,1), and16(2,2). Support circuitry24may likewise be used for reading out image data and depth-image data along output lines associated with image pixels and time-of-flight image pixels respectively.

Time-of-flight image pixels32may be configured to receive light reflected from objects in a real-world scene as shown inFIG. 3. In the example ofFIG. 3, a light pulse emitter such as non-visible light emitter20may be configured to emit one or more pulses of, for example, infrared light such as pulse26. Camera module12may be configured to receive a portion of light pulse26that is reflected from multiple objects such as objects28,30, and33having distances DC, DM, and DF from electronic device10respectively. As shown inFIG. 3, portions of light pulse26may travel from emitter20to objects28,30, and33along paths such as paths34,36, and38respectively.

Because distance DF to object33is larger than distances DM and DC to objects30and28respectively, path38may be longer than paths36and34. Because path34is longer than paths36and34, the portion of light pulse26that is reflected from object33will take a longer period of time to reach image sensor array16than portions of light pulse reflected from objects28and30. The time-of-flight of returning portions of light pulse26may therefore depend on distances such as distances DC, DM, and DF to objects in a real-world scene.

Time-of-flight image pixels32may be configured to sense relative differences in the time-of-flight of returning portions of light pulse26by sensing how much of returning portions of light pulse26return within a predetermined light collection period (sometimes called integration time or exposure time). Time-of-flight image pixels32may be configured to sense how much of returning portions of light pulse26return within a predetermined light collection period by determining the relative quantity of light sensed by time-of-flight image pixels32. However, variations in intrinsic brightness of objects in a scene due to variations in object color and illumination may be problematic when extracting information about the how much of returning portions of light pulse26return within the predetermined integration time.

For example, a real-world scene may contain other objects such as background object40and may be illuminated by light sources other than emitter20such as external light source42. External light source42may be the Sun, the Moon, a flame, an indoor or outdoor electric light (e.g., an incandescent of fluorescent) or other light source or combination of light sources. Object40may be located behind objects such as objects28,30, and33, in front of objects28,30, and33, may form a portion of objects28,30, and33or may be otherwise positioned in the field-of-view of camera module12. Objects28,30,33, and40may be uniform in color or may have portions that have different colors from other portions. Objects28,30,33, and40may be uniformly illuminated by light sources such as light source42or may have portions that are in bright light and portions that are in relative darkness.

Variations in intrinsic brightness of objects28,30,33, and40due to variations in color and illumination may cause differences in the quantity of light received by camera module12of device10that are unrelated to differences in distance (e.g., distances DC, DM, DF, or other object distance). Time-of-flight image pixels32may be configured to partially or completely remove intrinsic brightness (e.g., background light) from reflected portions of a light pulse emitted by emitter20.

The example ofFIG. 3in which device10determines distances to three objects is merely illustrative. In general, device10may determine distances to a number of objects equal to or less than the number of time-of-flight image pixels (e.g., tens, hundreds, thousands, millions or more) in image sensor array16. Device10may determine relative distances to multiple portions of a single object.

FIG. 4is a diagram showing how returning portions of an emitted light pulse that are reflected by objects in a scene may be affected by the distance to a reflecting object. As shown inFIG. 4, an emitted pulse of light such as pulse26may be emitted in the direction of objects in a real-world scene. Time-of-flight image pixels32may receive returning pulses of reflected light such as returning pulses50,52, and54. Returning pulses50,52, and54may, for example be returning portions of emitted pulse26that have been reflected from objects28,30, and33ofFIG. 3respectively.

Time-of-flight image pixels32may be configured to collect light for an exposure period (e.g., an exposure time, integration time, or light collection period) beginning at time T1and ending at time T1. In the example ofFIG. 4, time T1occurs after the beginning of pulse26and before the end of pulse26. This is merely illustrative. Pulse26may begin and end before time T1or pulse26may begin and end after time T1and before time T2if desired.

Because returning pulses50,52, and54travel along paths34,36, and38(seeFIG. 3), pulse50may return to device10before pulse52and pulse52may return to device10before pulse54. Because time-of-flight image pixels32collect light for a predetermined light collection period, time-of-flight image pixels32may only collect portions of a return pulse that arrive during the exposure period. Therefore, because path34is relatively short, portion56of return pulse50that returns during the exposure period may include substantially all of return pulse50. Because path36is relatively longer than path34, only a fractional portion such as portion58of return pulse52may be detected by a time-of-flight pixel32during the exposure period. Similarly, because path38is relatively long, a relatively smaller fractional portion60of return pulse52may be detected by a time-of-flight pixel32during the exposure period.

In this way, emitter20and time-of-flight image pixels such as pixels32may be used to gather depth information about the scene from relative brightness or detected, reflected light. However, as shown inFIG. 5, it may be beneficial to remove background light signals from detected signals before constructing a depth image. In the example ofFIG. 5, an image pixel such as a time-of-flight image pixel32may be configured to detect a background signal such as signal70in the absence of an emitted light pulse that includes only background light and associated noise such as photon noise, read noise or other noise. Due to the noise signal associated with background signal70, background signal70may be uncertain by an amount72. A light emitter such as emitter20may emit a light pulse26having a signal intensity that is smaller than the intensity of background signal70.

After emission of light pulse26, time-of-flight image pixels such as pixels32ofFIG. 2may detect reflected return pulses such as pulses50and54in addition to background signal70that is uncertain by an amount72. Because background signal70may be larger than pulse26, return pulses50and54may be smaller than the amount72of noise associated with signal70. For this reason, time-of-flight image pixels such as pixels32may be configured to remove background signal70from detected image light signals. Time-of-flight image pixels such as pixels32may be configured to sample background signal70and to readout only signal from subsequent integrations that is in excess to background signal70. Time-of-flight image pixels such as pixels32may therefore be configured to provide close object signal50and far object signal54with background signal70removed so that a relative difference74may be detected between signals such as signals50and54.

FIG. 6is a schematic diagram of an illustrative time-of-flight image pixel32. As shown inFIG. 6, time-of-flight image pixel32may include a photosensitive element such as photodiode PD and charge storage regions such floating diffusion regions FD_B and FD_F. Charge storage regions FD_B and FD_F may be implemented using a region of doped semiconductor (e.g., a doped silicon region formed in a silicon substrate by ion implantation, impurity diffusion, or other doping techniques). The doped semiconductor region (i.e., the floating diffusion FD) exhibits a capacitance that can be used to store the charge that has been transferred from photodiode PD. The signal associated with the stored charge on node FD_F may be conveyed processing circuitry such as processing circuitry18ofFIG. 1using readout circuitry82.

Photodiode PD may be implemented using a p-n junction formed from an interface such as interface80between doped semiconductor regions77and79(i.e., an interface between a p-type semiconductor and an n-type semiconductor) for converting captured light into electrical charge. Region77may be implemented using a region of p-type doped semiconductor and region79may be implemented using an n-type doped semiconductor. However, this is merely illustrative. If desired, region77may be implemented using a region of n-type doped semiconductor and region79may be implemented using a p-type doped semiconductor.

Time-of-flight pixel32may include reset transistors such as reset transistor85that receive a reset signal RSTG. Reset transistor85may include a source/drain terminal coupled to a reset voltage RSTD. Reset voltage RSTD may be, for example, a positive power supply voltage (sometimes denoted as Vaa), a ground voltage (sometimes denoted as AGND), etc. Time-of-flight pixel32may include transfer transistors (transfer gates) such as transfer transistor81that receives a transfer signal TXB for transferring electric charge from photodiode PD to charge storage region FD_B and transfer transistor83that receives a transfer signal TX for transferring electric charge from photodiode PD to charge storage region FD_F.

As shown inFIG. 6, transfer transistor83may have a gate terminal such as terminal86that is coupled to charge storage region FD_B. Time-of-flight image pixel32may include a switch such as switch84that allows gate86of transfer transistor83to be connected to charge storage region FD_B so that transfer signal TX may be equal to the voltage on charge storage region FD_B. Switch84may allow gate86to alternatively be connected to an additional transfer voltage so that transfer transistor83may receive a different transfer control signal such as transfer control signal TDX (e.g., a positive power supply voltage Vaa, a ground voltage AGND, etc.).

Signals associated with the charge converted by a photodiode or current generated by time-of-flight pixel32(sometimes referred to herein as depth-image data) may be conveyed to processing circuitry18of electronic device10(seeFIG. 1) through readout circuitry such as circuitry82that includes components such as row select transistors, source-follower transistors, or other components associated with time-of-flight pixel32. Some components of time-of-flight pixel32(e.g., row select transistors, charge storage regions, reset transistors, etc.) may be shared among multiple time-of-flight pixels. Image data that has been captured by time-of-flight pixels32may be processed and stored using processing circuitry18. Processed image data may, if desired, be provided to external equipment (e.g., a computer or other device) using wired and/or wireless communications paths coupled to processing circuitry18.

Various illustrative embodiments of time-of-flight image pixel32having a transfer transistor having a gate that is coupled to a charge storage region are shown inFIGS. 7,8, and9. As shown inFIG. 7, photodiode PD may be coupled to multiple charge storage regions such as floating diffusion regions FD_B and FD_F. Photodiode PD may be coupled to floating diffusion region FD_F via transfer transistor83and to floating diffusion region FD_B via transfer transistor81. Gate86of transfer transistor83may be coupled to a source/drain terminal of transfer transistor92and a source/drain terminal of transfer transistor94. In combination, transistors92and94may form switch84for selectively connecting gate86to floating diffusion region FD_B (e.g., by asserting transfer signal TXF and de-asserting transfer signal TXG) or to another transfer control signal such as signal TXD (e.g., by de-asserting transfer signal TXF and asserting transfer signal TXG).

As shown inFIG. 7, if desired, floating diffusion region FD_B may include an additional capacitor such as capacitor96for increasing the well depth of floating diffusion region FD_B (i.e., the amount of charge that can be stored). During exposure, light may be converted into electric charge by photodiode PD. Charges accumulated by photodiode PD may be transferred to floating diffusion region FD_B by activating transfer transistor81or to floating diffusion region FD_F by activating transfer transistor83. The quantity of charge transferred from photodiode PD to floating diffusion region FD_B may be determined by the voltage TX applied to gate86of transfer transistor83(e.g., whether gate86is coupled to floating diffusion region FD_B or to another voltage source).

As shown inFIG. 7, pixel32may be coupled to a readout circuit such as readout circuit82. Readout circuit82may include a source follower transistor such as transistor90having a gate terminal coupled to floating diffusion region FD_F for converting charge stored on floating diffusion region FD_F into a voltage to be readout along path100(e.g., to processing circuitry18ofFIG. 1). Readout circuit82may include reset transistor85. In the example ofFIG. 7, reset transistor85includes a first source/drain terminal connected to floating diffusion region FD_F and a second source/drain terminal connected to a reset voltage RSTD. Source follower transistor90may include a gate terminal connected to a first source/drain terminal of reset transistor85and a source/drain terminal that is connected to a second source/drain terminal of reset transistor85. However, this is merely illustrative.

If desired, source/drain terminals of source follower transistor90may be free of connections to source/drain terminals of reset transistor85as shown inFIG. 8. In the example ofFIG. 8, source follower transistor90may include a gate terminal connected to a source/drain terminal of reset transistor85and a source/drain terminal that is connected to a row select transistor such as row select transistor98. Row select transistor98may include a gate terminal that receives a row select signal RSEL that selects a row of pixels including pixel32. When row select signal RSEL is asserted, charge stored on floating diffusion region FD_F may be converted to a voltage signal by source follower transistor90and the pixel signal may be read out along path100.

It may be desirable to be able to reset photodiode PD without resetting floating diffusion region FD_F. As shown inFIG. 9, time-of-flight pixel32may therefore be provided with an additional reset transistor such as transistor102having a source/drain terminal coupled to photodiode PD and a second source/drain terminal coupled to a reset voltage PDD. Transistor102may receive a control signal PDG for resetting photodiode PD. Photodiode PD may be reset between subsequent light collection periods during which return pulses such as pulses50,52, and54ofFIG. 5are incident on photodiode PD.

Some of the charges generated by photodiode PD during exposure periods during which return pulses such as pulses50,52, and54are incident on photodiode PD may be transferred to floating diffusion region FD_F after each exposure period. Photodiode PD may then be reset using transistor102without resetting floating diffusion region FD_F so that additional charges may be collected due to additional return pulses. The additional charges may then be transferred to floating diffusion region FD_F where the original charges are still stored. In this way, multiple integrations of charges generated by light pulses reflected by objects after emission (e.g. by emitter20ofFIG. 1) may be accumulated on floating diffusion region FD_F prior to readout of pixel32.

During transfer of charges to floating diffusion region FD_F, gate86of transistor83may be coupled to floating diffusion region FD_B. Floating diffusion region FD_B may store charges transferred from photodiode PD following a previous exposure period in which no pulse was emitted by emitter20. Photodiode PD may be reset between transfer of charges to floating diffusion region FD_B and accumulation of charges due to return pulses such as pulses50,52, and54incident on photodiode PD.

FIGS. 10,11,12,13,14,15,16, and17show illustrative charge storage configurations of time-of-flight image pixel32during operation electronic device10. As shown inFIG. 10, before image data is acquired (e.g., before exposure of photodiode PD to light for conversion of light into electric charge), reset control signal RSTG, and transfer signals TXB and TX may be asserted to reset pixel32. This turns on reset transistor85and transfer transistors81and83to reset charge storage nodes FD_B and FD_F (also referred to as floating diffusion regions) to a power supply voltage Vaa as indicated by arrows110. The reset control signal RSTG may then be deasserted to turn off reset transistor85.

After the reset process is complete, as shown inFIG. 11, transfer gate control signals TX and TXB may be deasserted (e.g., transfer gates associated with transistors81and83may be coupled to a ground voltage AGND) as indicated by arrows114. With transfer control signals TX and TXB deasserted, photodiode PD may be exposed to background light BGL for a predetermined amount of time (e.g., the exposure time). Background light BGL may include light from a real-world scene in the absence of an emitted light pulse from electronic device10(e.g., a pulse of infrared light from emitter20ofFIG. 1). Photodiode PD may convert background light BGL into electric charges112.

As shown inFIG. 12, charges112may be transferred from photodiode PD to charge storage region FD_B. Charges112may be transferred to region FD_B by asserting transfer signal TXB (e.g., coupling a gate terminal of transistor81to supply voltage Vaa) thereby activating transistor81.

Following transfer of charges112to storage region FD_B, a light emitter associated with device10(e.g., emitter20ofFIG. 1) may emit a pulse of light that is reflected from objects in a scene onto photodiode PD. As shown inFIG. 13, reflected pulse light PL and background light BG may be incident on photodiode PD. Photodiode PD may convert pulse light PL and background light BG into electric charges116and112′ respectively while collecting light for the same predetermined exposure time used in collecting background light BGL as described in connection withFIG. 11. Because electric charges are indistinguishable from other electric charges, time-of-flight image pixel32may be configured to separate electric charges112′ and116using a previous measurement of the amount of charge produced by photodiode PD in response to background light BGL during the same exposure time (i.e., charges112stored on floating diffusion region FD_B).

As shown inFIG. 14, time-of-flight image pixel32may be configured to separate electric charges112′ and116by coupling gate86of transfer transistor83to charge storage region FD_B where charges112are stored. Coupling gate86of transistor83charge storage region FD_B where charges112may couple gate86to a voltage V(FD_B) determined by the amount of charges112stored on region FD_B. As shown inFIG. 14, this may allow charges116in excess of the amount of charges112stored on region FD_B to be transferred to floating diffusion region FD_F. Following transfer of charges116to charge storage region FD_F, charges112′ may remain on photodiode PD. Charges112′ and charges112may be a substantially equal amount of charge. Gate86may be coupled to charge storage region FD_B by activating transistor94(seeFIGS. 7,8, and9) by asserting transfer signal TXF). Activating transistor94may partially activate transistor83as indicated by arrow118ofFIG. 14to transfer charges116to storage region FD_F.

As shown inFIGS. 15,16, and17, the steps described above in connection withFIGS. 11,12,13, and14may be repeated following a reset of photodiode PD (i.e., a removal of charges112′ stored on photodiode PD). Photodiode PD may be reset, for example, by activating transistor102ofFIG. 9. Resetting photodiode PD by activating transistor102may allow additional charges generated from pulse light PL to be stored with charges116on storage region FD_F. However, this is merely illustrative. If desired, a time-of-flight signal based on charges116on storage region FD_F may be read out using a readout circuit such as circuit82ofFIG. 6and photodiode PD and storage region FD_F may both be reset before repeating the steps described above in connection withFIGS. 11,12,13, and14.

In configurations in which photodiode PD is reset without resetting floating diffusion region FD_F, photodiode PD may be subsequently exposed to background light BGL and pulse light PL from a subsequent pulse by emitter20while gate86is coupled to ground voltage AGND as shown inFIG. 15. Background light BGL and subsequent pulse light PL may be converted into electric charges112″ and116′ respectively.

As shown inFIG. 16, gate86of transistor83may be subsequently coupled to charge storage region FD_B in order to transfer charges116′ to charge storage region FD_F. Following transfer of charges116′ to charge storage region FD_F, charges112″ may remain on photodiode PD. Charges112″ and charges112may be a substantially equal amount of charge. Following transfer of charges116′ to charge storage region FD_F, charge storage region FD_F may include both charges116′ and116from subsequent pulses of emitter20.

As shown inFIG. 17, the steps described above in connection with FIGS.15and16may be repeated any number of times. Following transfer of charges116,116′ and charges associated with further pulses of light by emitter20to floating diffusion region FD_F, charges such as charges112″ . . . may remain on photodiode PD until a subsequent reset of photodiode PD or pixel32. Charges116,116′, . . . may be transferred to control circuitry such as storage and processing circuitry18ofFIG. 1for using a source follower transistor to convert charges116,116′, . . . into a time-of-flight signal associated with an object in the field-of-view of time-of-flight pixel32. Circuitry18may be used to combine time-of-flight signals from multiple pixels32(e.g., in an array of time-of-flight image pixels, multiple arrays of time-of-flight pixels, etc.) to form a depth image in which the value of each pixel in the depth image contains information relating to the distance to an object in that pixel.

FIG. 18is a flow chart of illustrative steps that may be used in acquiring depth images using an electronic device having time-of-flight image pixels and a light pulse emitter.

At step150, with the light pulse emitter off, charges may be collected using a photodiode such as photodiode PD associated with time-of-flight image pixels such as time-of-flight image pixel32in response to background light such as background light BGL for a predetermined exposure time.

At step152, charges collected using photodiode PD may be transferred to a charge storage region such as floating diffusion region FD_B by activating a transfer transistor coupled between the photodiode and the charge storage region.

At step154, charges may be again collected using photodiode PD for the same predetermined exposure time. During the predetermined exposure time, a pulse of, for example, infrared light may be emitted by a non-visible light emitter such as non-visible light emitter20ofFIG. 1. Charges collected by photodiode PD may be generated by photodiode PD in response to background light BGL and pulsed light from emitter20that has been reflected from objects in the field-of-view of time-of-flight image pixel32.

At step156, a second transfer transistor such as transfer transistor83coupled between photodiode PD and a second charge storage region such as floating diffusion region FD_F may be partially activated by coupling a gate such as gate terminal86of transistor83to floating diffusion region FD_B. Coupling gate terminal86to floating diffusion region FD_B may allow charges generated by photodiode PD in response to pulse light PL (sometimes called depth-image charges) to be transferred to charge storage region FD_F.

At step158, photodiode PD may be reset. If desired, charge storage regions FD_F and FD_B may also be reset. In configurations in which charge storage region FD_F is also reset, a voltage associated with charges stored on charge storage region FD_F may be read out to circuitry such as storage and processing circuitry18ofFIG. 1prior to resetting charge storage region FD_F. In configurations in which charge storage region FD_B is reset, steps150,152,154and156may be repeated for subsequent collection of depth-image charges.

As indicated by arrow162, if desired, following reset of photodiode PD, steps,154, and156may be repeated to collect subsequent measurements of depth-image information by collecting charges associated with subsequent pulses of light by emitter20. Repeating steps154and156may allow collection of a stronger depth-image signal without increasing the intensity of emitted light pulses from emitter20.

At step160, cumulative depth-image charges (i.e., all charges stored on floating diffusion region FD_F following multiple pulses of light from emitter20) may be read out from charge storage region FD_F to circuitry such as storage and processing circuitry18.

Circuitry18may be used to combine time-of-flight signals (depth-image signals) from multiple pixels32(e.g., in an array of time-of-flight image pixels, multiple arrays of time-of-flight pixels, etc.) to form a depth image in which the value of each pixel in the depth image contains information relating to the distance to an object in that pixel.

Various embodiments have been described illustrating electronic devices that include time-of-flight image pixels configured to measure the time of flight of an emitted light pulse for sensing distance information about objects in a scene. Emitted light pulses may be generated by a light pulse emitter on the electronic device and reflected from objects in the field-of-view of the time-of-flight image pixels. Time-of-flight image pixels may be configured to measure differences in time-of-flight between reflected portions of emitted light pulses using differences in brightness of the reflected portions. Time-of-flight image sensors may be configured to remove background light contamination of reflected portions of emitted light pulses.

A time-of-flight image pixel may include a photosensitive element such as a photodiode, and first and second charge storage regions coupled to the photosensitive element. A time-of-flight image pixel may include a first transfer transistor coupled between the photosensitive element and the first charge storage region and a second transfer transistor coupled between the photosensitive element and the second charge storage region. The second transfer transistor may include a gate terminal that is coupled to the first charge storage region.

A time-of-flight image pixel may include a third transfer transistor having first and second source/drain terminals. The first source/drain terminal of the third transfer transistor may be connected to the gate terminal of the second transfer transistor and the second source/drain terminal of the third transfer transistor may be connected to the first charge storage region.

A time-of-flight image pixel may include a fourth transfer transistor having a first source/drain terminal that is coupled to the gate terminal of the second transfer transistor and a reset transistor having a first source/drain terminal that is coupled to the second charge storage region and a second source/drain terminal coupled to a source/drain terminal of a source follower transistor having a gate terminal connected to the second charge storage region. If desired, a time-of-flight image pixel may include an additional reset transistor having a first source/drain terminal that is coupled to the photosensitive element.

If desired, the time-of-flight image pixel may include a reset transistor having a first source/drain terminal that is coupled to the second charge storage region, a source follower transistor having a gate terminal connected to the second charge storage region, and a row select transistor coupled to the source follower transistor.

The electronic device may further include a light pulse emission component such as a non-visible light pulse emitter configured to emit pulses of non-visible light. The electronic device may include an array of image sensors. The array of image sensors may include a red image sensor, a blue image sensor, a green image sensor or other image sensors. Each of the image sensors in the array of image sensors may include an array of time-of-flight image pixels. Time-of-flight image pixels may be configured to collect background light and reflected portions of the emitted pulses of non-visible light and to store charges generated by the background light on the first charge storage region and to store charges generated by the reflected portions of the emitted pulses of non-visible light on the second charge storage region.

The electronic device may include processing circuitry configured to extract depth information from a depth-image signal generated by the time-of-flight image pixels. The processing circuitry may be configured to combine image data from the red image sensor, the blue image sensor, and the green image sensor to form a color image.

During operation of the electronic device, time-of-flight image pixels may be configured to convert background light into electric charges and to transfer the electric charges from the photosensitive element to the first charge storage region. A light pulse emitter may be configured to emit a pulse of non-visible light. Time-of-flight image pixels may be configured to convert additional background light and a reflected portion of the emitted pulse of non-visible light into additional electric charges and to transfer a portion of the additional electric charges (e.g., the portion corresponding to the reflected portion of the emitted pulse of non-visible light) to the second charge storage region. Transferring the portion of the additional electric charges may include connecting the gate terminal of the second transfer transistor to the first charge storage region on which the electric charges are stored by activating the fourth transfer transistor.

During operation, the photosensitive element may be reset to remove a remaining portion of the additional electric charges from the photosensitive element before a subsequent pulse of non-visible light may be emitted from the light pulse emitter. Time-of-flight image pixels may be configured to convert further additional background light and a reflected portion of the subsequent emitted pulse of non-visible light into further additional electric charges and to transfer a portion of the further additional electric charges (e.g., the portion corresponding to the reflected portion of the subsequent emitted pulse of non-visible light) to the second charge storage region on which the portion of the additional electric charges is stored.

Time-of-flight image pixels may be configured to convert the portion of the additional electric charges and the portion of the further additional electric charges into a depth-image signal. Processing circuitry may be used to extract distance information from the depth-image signal and to process the distance information to form a portion of a depth image that includes depth-image pixel values that correspond to the distance of an object to the electronic device.

The foregoing is merely illustrative of the principles of this invention which can be practiced in other embodiments.