Patent Description:
The disclosure describes a time of flight camera configured to filter out erroneous readings resulting from pulses of light bouncing off multiple surfaces.

This disclosure relates to ways of improving performance of a depth detection system. The depth detection system can be configured to sequentially emit complementary illumination patterns onto a region being monitored by an imaging sensor of the depth detection system. The imaging sensor can act as a time of flight sensor to determine a distance between the depth detection system and objects within the region by measuring the time it takes for light forming the illumination patterns to reflect off the objects and return to the imaging sensor. Some of the light received at the imaging sensor can be indirect light that bounces off other surfaces before arriving at the imaging sensor. This can be especially problematic in room corners where more indirect light is likely to return to the imaging sensor. The reflections increase the amount of time it takes for the light to return to the imaging sensor, thereby reducing the accuracy of the sensor data. Some of this indirect light can be filtered out from consideration by the depth detection system by identifying light reflecting off portions of the region being monitored by the imaging sensor falling outside of a first illumination pattern when the first illumination pattern is active. This identified light can then be subtracted out of consideration when the second illumination pattern is active. Similarly, light falling outside of the second illumination pattern when the second illumination pattern is active can be subtracted from the first illumination pattern. In this way, more accurate depth detection information can be obtained.

Light sources that emit the complementary illumination patterns can be mounted to a common substrate to prevent the light sources from being out of alignment from one another. The common substrate can also help reduce any thermal effects that would result in the light sources being thrown out of alignment.

A depth detection system is disclosed and includes at least the following: a projection system, comprising: a projector housing having a rigid substrate, a first light source configured to emit light through a first plurality of light shaping components, the first light source being mounted to the rigid substrate, and a second light source configured to emit light through a second plurality of light shaping components, the second light source being mounted to the rigid substrate adjacent to the first light source; an imaging sensor proximate the projection system and configured to receive light emitted by the first and second light sources after being reflected off objects within a field of view of the imaging sensor; and a processor configured to calculate a distance between the depth detection system and the objects within the sensor field of view by measuring an amount of time for light emitted by the first and second light sources to reflect off the objects within the sensor field of view and return to the imaging sensor.

Another depth detection system is disclosed and includes the following: a plurality of light shaping components, comprising: a collimating optical element, a refractive optical element, a diffractive optical element, and a micro-lens array; a light source configured to emit light through the plurality of light shaping components; an imaging sensor configured to detect light emitted by the light source and reflected off objects within a field of view of the imaging sensor; and a processor configured to determine a distance between the depth detection system and the objects by filtering out sensor readings associated with light reflected off surfaces outside the field of view of the imaging sensor.

A depth detection system is disclosed and includes the following: a projection system, comprising: a projector housing having a rigid substrate, a first light source configured to emit light through a first plurality of light shaping components and produce a first illumination pattern, the first light source being mounted to the rigid substrate, and a second light source configured to emit light through a second plurality of light shaping components and produce a second illumination pattern complementary to the first illumination pattern, the second light source being mounted to the rigid substrate adjacent to the first light source; an imaging sensor proximate the projection system and configured to receive light emitted by the first and second light sources after being reflected off objects within a field of view of the imaging sensor; and a processor configured to calculate a distance between the depth detection system and the objects within the sensor field of view by measuring an amount of time for light emitted by the first and second light sources to reflect off the objects within the sensor field of view and return to the imaging sensor and filtering out sensor readings associated with light reflected off surfaces outside the field of view of the imaging sensor.

Document <CIT> discloses a time of flight camera system configured to resolve either or both of the direct path and multipath components of radiation received by the system, wherein two illumination patterns are simultaneously projected.

Other aspects and advantages of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings which illustrate, by way of example, the principles of the described embodiments.

Representative applications of methods and apparatus according to the present application are described in this section. These examples are being provided solely to add context and aid in the understanding of the described embodiments. It will thus be apparent to one skilled in the art that the described embodiments may be practiced without some or all of these specific details. In other instances, well known process steps have not been described in detail in order to avoid unnecessarily obscuring the described embodiments. Other applications are possible, such that the following examples should not be taken as limiting.

A depth detection system can be configured to characterize an environment within a field of view of the depth detection system. The resulting characterization can be used to determine a position and exterior shape of portions of objects facing the depth detection system. One type of depth detection system is a time of flight (TOF) camera. A TOF camera utilizes a projector for emitting modulated pulses of light and a sensor for receiving a portion of each of the pulses of light that reflects off the various objects within the sensor's field of view. A processor receiving readings from the sensor can determine the time taken for the light to travel from the sensor and bounce off one of the objects in the field of view and return to the sensor. Because the speed of light is known, the system can determine the distance between the depth detection sensor and the object based on that time. Unfortunately, while this method works well for determining distance when the light bounces off an object and returns directly back to the sensor, any light returning to the sensor that bounces off another object first can cause inaccuracies in the depth data.

One solution to this problem is to filter out indirectly reflected light received at the sensor to reduce inaccuracies. One way this can be accomplished is to adjust the manner in which the environment is illuminated with light. The light can be emitted by a projection system in alternating illumination patterns to sequentially illuminate different portions of the objects in the field of view. In some embodiments, the illumination pattern can be arranged in substantially parallel stripes, although different patterns are also possible. Each of the stripes can be separated by a gap having about the same thickness as each stripe. In this way, about half of the field of view can be illuminated any time an illumination pattern is emitted. It should be appreciated that different stripe and gap thicknesses can be used but that at some point during a series of different illumination patterns each portion of the field of view should be unilluminated. Any light returning from areas of the frame that should not be illuminated by a particular pattern of light can be used to identify reflected light. When a different illumination pattern illuminates that portion of the object from which reflected light was previously detected, the reflected light can be subtracted from the detected light to identify only that portion of the light that travels directly from the projection system to the object and back to the sensor. Any other light can be ignored for the purposes of making a depth map of the area with the sensor's field of view. In this way, the accuracy of the depth data can be substantially improved.

A projection system for performing the aforementioned method can include two or more light sources for generating the illumination patterns. In some embodiments, the projection system can be configured to operate very quickly in order to keep up with changing conditions. For example, in some embodiments, the light sources can be configured to emit more than <NUM> pulses per second. A sensor associated with the projection system can be configured to capture the light as it comes back and can have a global shutter that allows each of the pixels of the sensor to be read at the same time. In this way, any errors introduced due to sequentially reading the pixels can be avoided.

According to the invention, the light sources are incorporated within a single projector housing. Packaging the light sources in a single projector prevents the situation where one of two or more separate projection units gets bumped or jostled a different amount than the other units, which results in misalignment of the illumination patterns. While a slight change in alignment of a single projector configured to project multiple illumination patterns could result in a portion of the sensor field of view not being covered by the illumination pattern, the majority of the sensor field of view could remain covered without compromising alignment of the illumination patterns. According to the invention, a single projector housing includes a unitary rigid substrate with a low coefficient of thermal expansion that keeps the separation between the light sources consistent over a large range of temperatures. Each of the light sources can have different optics that direct the light into the various illumination patterns. In some embodiments, a projection system with a single light source can be used that has shifting optics. In such an embodiment, the optics can oscillate between two or more positions to create two or more illumination patterns from the single light source.

These and other embodiments are discussed below with reference to <FIG>; however, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes only and should not be construed as limiting.

<FIG> shows an exemplary depth detection system <NUM> in use. Depth detection system <NUM> includes a projection system <NUM> and a sensor <NUM>. Projection system <NUM> can be configured to emit light towards an object <NUM>. In some embodiments, the light emitted by projection system <NUM> can be infrared light or near infrared light. Since the light emitted by projection system <NUM> can be configured to cover a broad area corresponding to a field of view of sensor <NUM>, exemplary light wave <NUM> can bounce off of wall <NUM> and due to the angle of wall <NUM> light wave <NUM> can instead of reflecting back from wall <NUM> bounce off object <NUM> and then back to sensor <NUM> as depicted. This can be particularly problematic when object <NUM> has irregular surfaces (i.e. curved or cylindrical surfaces) that scatter light incident to object <NUM>. The scattering of the reflected light increases the likelihood of the reflected light arriving back at sensor <NUM> as depicted.

<FIG> shows how light incident to object <NUM> can be reflected by diffuse and/or specular reflection. While a flat surface is generally needed to generate specular reflection, a flat surface also tends to generate some diffuse reflection on account of scattering centers located below the surface of object <NUM>. Curved or varied surfaces generate even more diffuse reflections that scatter in many directions. One of the reasons the light reflected off wall <NUM> can be hard to distinguish from the direct light is that when the surface of wall <NUM> is relatively flat, a substantial amount of light wave <NUM> can be reflected as specular reflection from wall <NUM>, thereby causing the resulting diffuse reflection at object <NUM> from light wave <NUM> to have a similar intensity as the diffuse reflection at object <NUM> originating from light wave <NUM>. It should be noted that light going from the projector to object <NUM> and then bouncing off wall <NUM> back towards the sensor is not considered to be a problem where wall <NUM> is not in the sensor field of view. In such a case, the high angle of incidence of the light entering the sensor would not be detected by the sensor on account of the sensor only being configured to receive light arriving from a particular field of view. The high angle of incidence light can be prevented from reaching the sensor using a shroud or light gathering lens positioned over the sensor.

<FIG> shows examples of different types of objects illuminated by projection system <NUM>. The first column of images shows images generated using all of the light reflected off of the objects and captured by sensor <NUM>. The images in the second column show only the light reflected directly off the objects. The images in the third column show only the light reflected first off other objects (indirect light) prior to hitting the object in the sensor field of view. The first row of egg pictures provide an example of diffuse interreflections. The spherical shape of the eggs accentuates the amount of diffuse reflection generated by light striking the surface of each of the eggs. In particular, the indirect light image from the first row shows how the lower edges of the eggs capture a substantial amount of indirect light and could consequently appear to be farther away from the sensor. The second row of wooden block pictures provides an example of both diffuse and specular interreflection. The flat surfaces of the wooden blocks allow for a certain amount of specular reflection while the underlying wood grain structure and corners of the blocks create diffuse interreflection. Finally the third row of peppers shows how sub-surface scattering can cause only a small amount of light to be reflected directly back to sensor <NUM>. This limited amount of direct light can make filtering out the indirect light even more important for determining the actual distance between sensor <NUM> and the peppers. <FIG> was originally published as part of the article "<NPL>.

<FIG> shows projection system <NUM>, which includes projectors <NUM> and <NUM>. Projectors <NUM> and <NUM> can be used to emit complementary illumination patterns A & B. Illumination patterns A & B can be sequentially pulsed so that only one of the illumination patterns is active at any given time. According to the invention, the illumination patterns are pulsed in an alternating pattern (e.g. in an A, B, A, B pattern). The pulsed emissions can also be modulated to help distinguish the pulsed emission from other ambient light sources. Consequently, when illumination pattern A is active, any area outside of illumination pattern A should be devoid of light. However, generally a portion of illumination pattern A reflecting off other surfaces first and in certain environments other ambient light can be detected by sensor <NUM> reflecting off areas not being directly illuminated by illumination pattern A. This reflected light detected in the unilluminated areas of target <NUM> can be subsequently used to identify reflected light when illumination pattern B is active. Similarly, when illumination pattern B is active, reflected light arriving from outside of illumination pattern B can be subsequently used to identify reflected light during the next pulse of illumination pattern A. So in general, the reflected light or indirect light (IINDIRECT) detected originating from outside of the active illumination pattern can be recorded. When the next illumination pattern activates, the previously recorded indirect light (IINDIRECT) from the now active illumination pattern can be subtracted from all of the light (ITOTAL) received from the active illumination pattern in accordance with Eq(<NUM>) to identify the direct light.

It should be noted that in some embodiments, any ambient light reflecting off target <NUM> and back into sensor <NUM> can be filtered out by rejecting light not matching the modulation associated with the illumination patterns.

<FIG> shows exemplary illumination patterns A and B. The intensity of illumination patterns A and B can be distributed in a sinusoidal pattern as a function of vertical position. As depicted, illumination pattern <NUM> can be <NUM> degrees out of phase with illumination pattern <NUM>, resulting in illumination pattern <NUM> having a maximum intensity value when illumination pattern <NUM> is at a minimum intensity value. In this way, if the two illumination patterns were emitted simultaneously then a substantially uniform light pattern would be created. Graph <NUM> illustrates illumination pattern A while graph <NUM> illustrates illumination pattern B. Mathematically the intensity of the combined pattern would cause the intensity value to have a substantially constant value equal to <NUM>. More generally, the illumination intensity can be modeled using Eq(<NUM>).

In Eq(<NUM>), i indicates which illumination pattern of a total of N illumination patterns is being calculated. A<NUM> is the amplitude of the illumination pattern. f is the spatial frequency of the light bars. β is the angle of the vertical field of view of the sensor. φi represents the shift in phase for the illumination pattern whose value is determined by Eq(<NUM>).

As can be appreciated, Eq(<NUM>) makes clear that the phase shift can be <NUM> degrees for two patterns, <NUM> degrees for three patterns, <NUM> degrees for four patterns, etc. In general, more illumination patterns can be used to achieve more accurate results. Furthermore, in some embodiments, the shift in phase can also be varied in different manners.

<FIG> shows illumination patterns C and D. The intensity profiles of illumination patterns C and D are trapezoidal instead of sinusoidal. By having rapidly rising and falling intensities, a more abrupt transition between light bars of illumination patterns C and D can be achieved. A more crisp transition can be beneficial in minimizing ambiguity when filtering the indirect light from the direct light, as will be described in greater detail below.

<FIG> shows illumination patterns E, F and G. The intensity of illumination patterns E, F and G are distributed vertically so illumination pattern F is <NUM> degrees out of phase from illumination pattern E. In this way, successive light bars can be shifted vertically but without being complementary in nature. Graphs <NUM>, <NUM> and <NUM> quantitatively show how respective illumination patterns E, F and G vary in accordance with vertical position. The third illumination pattern can be generated by a third light source or by optics that can shift to create both the second and third patterns.

<FIG> shows how discrete pixels or sampling points can be distributed across multiple illumination patterns. Close up view <NUM> shows three different sampling points p1, p2 and p3 distributed within illumination patterns A and B. The indirect light at each of the sampling points can be identified by performing a number of calculations for each pixel / sampling point. In particular, Eq(<NUM>) can be used to sum up the light Si collected by the sensor during each sequential illumination pattern.

Eq(<NUM>) can then be used to calculate the amount of direct light when the intensity of each illumination pattern varies sinusoidally.

Eq(<NUM>) sums up the amplitude of each component of the light received, when each of the illumination patterns is active, in order to represent the total amount of light emitted over the span of one set of the illumination patterns. In a two illumination pattern projection system, the subtracted image represents reflected light detected from within illumination pattern A when illumination pattern B is active as well as reflected light detected from within illumination pattern B when illumination pattern A is active. By adding the two sets of reflected light together, the distribution of reflected light across the whole field of view can be determined. In general, this calculation assumes that the reflected light stays substantially the same regardless of which illumination pattern is active. Consequently, subtracting the subtracted image from the total light identifies the direct light within the field of view. Eq(<NUM>) shows how indirect light (I) can be calculated by subtracting the calculated direct light (D) from the total light (T).

In some embodiments, const[GL] can be subtracted from the total light. This constant can be optionally used to remove grey level bias when identifying the indirect light in the sensor field of view. In some embodiments, subtracting the grey level bias out can improve the accuracy of the depth data detected by the sensor. The grey level bias can be a factory setting or a value that can be periodically calibrated to keep the depth detection system working well.

<FIG> also demonstrates how depth detection at position p2 can be problematic for a system with only two illumination patterns. For p1 and p3 which are situated safely away from the boundary between illumination patterns, indirect light rejection can be straight forward since there are only two illumination patterns to consider. For p1, when illumination pattern A is active the received signal is equal to the direct light + any reflected light. When illumination pattern B is active, the received signal at p1 is equal to zero direct light + any reflected light. The direct light can be calculated by taking the difference between the two signals. This yields just the direct light since the reflected light cancels out and the direct light during illumination pattern B is equal to zero. For p3, the calculation works in a similar manner, yielding just the direct light. Unfortunately, at p2, which is located on the interface precisely between the illumination patterns, direct light from both patterns A and B will be detected at about the same intensity. This means that taking the difference in values results in a zero value. Furthermore, areas near the interface will also suffer from some inaccuracies any time direct light from both illumination patterns is present in substantial amounts. Consequently, illumination patterns with sharp boundaries between the illumination patterns will have fewer inaccuracies at the interfaces between the illumination patterns. However, direct light values for points near the interface can still be calculated by interpolation. The direct light value for p3 can be calculated by interpolation from direct light values for p4 and p5. Generally, p4 and p5 should be as close as possible to p2. For example, a processor can be configured to select an interpolation point p4 with an amount of direct light from illumination pattern B that falls below a predetermined threshold.

<FIG> shows a first optics assembly <NUM>, which is made up of a group of light shaping components positioned in front of a light source <NUM>. In some embodiments, light source <NUM> can be an infrared laser diode. Light source <NUM> emits light that passes through a first light shaping component, collimating lens <NUM>. Collimating lens <NUM> can be configured to focus light <NUM> emitted by light source <NUM> towards a second light shaping component, refractive optical element <NUM>. Refractive optical element <NUM> tilts focused light <NUM> by an angle θ and elongates the light vertically to generate a super gaussian beam <NUM> that is directed at a third light shaping component, diffractive optical element <NUM>. Diffractive optical element <NUM> then multiplies the super-gaussian beam <NUM>. While super gaussian beam <NUM> is depicted for illustrative purposes as being multiplied five times, this number can vary. For example, in some embodiments, diffractive optical element <NUM> can be configured to multiply the super-gaussian beam <NUM> times. The number and thickness of multiplied super gaussian beams <NUM> can be selected to match the vertical field of view of an associated imaging sensor. When super gaussian beams pass through micro-lens array <NUM>, micro-lens array <NUM> spreads each super gaussian beam horizontally to create an illumination pattern that illuminates regions <NUM>, as depicted. Micro-lens array <NUM> can by dual sided (as depicted), single sided or cylindrical. In some embodiments, regions <NUM> and regions <NUM> can be about the same size. Light from a second optics assembly can be configured to illuminate regions <NUM>. In some embodiments, the optics assemblies can emit light in complementary patterns so that one of horizontal regions <NUM> and <NUM> is illuminated at any given time.

<FIG> shows a second optics assembly <NUM>, which is made up of a group of light shaping components positioned in front of a light source <NUM>. In some embodiments, light source <NUM> can be an infrared laser diode. Light source <NUM> emits light that passes through a first light shaping component, collimating lens <NUM>. Collimating lens <NUM> can be configured to focus light <NUM> emitted by light source <NUM> towards a second light shaping component, refractive optical element <NUM>. Refractive optical element <NUM> tilts focused light <NUM> by an angle -θ and elongates the light vertically to generate a super gaussian beam <NUM> that is directed at a third light shaping component, diffractive optical element <NUM>. In some embodiments, orienting supper Gaussian beam <NUM> in a direction opposite from the direction of super Gaussian beam <NUM> a risk of cross-talk between the light sources can be reduced. Diffractive optical element <NUM> then multiplies the super-gaussian beam <NUM>. While super gaussian beam <NUM> is depicted for illustrative purposes as being multiplied five times, this number can vary. For example, in some embodiments, diffractive optical element <NUM> can be configured to multiply the super-gaussian beam <NUM> times. The number and thickness of multiplied super gaussian beams <NUM> can be selected to match the vertical field of view of an associated imaging sensor. When super gaussian beams pass through micro-lens array <NUM>, micro-lens array <NUM> spreads each super gaussian beam horizontally to create an illumination pattern that illuminates regions <NUM>, as depicted. In this way, light sources <NUM> and <NUM> can cooperatively illuminate regions <NUM> and <NUM>. Illumination of the regions can be staggered in different patterns. For example, regions <NUM> and <NUM> can be alternatively illuminated so that light shines in both regions for about the same amount of time.

<FIG> shows another optics assembly <NUM>, which is made up of three light shaping component positioned in front of a light source <NUM>. In some embodiments, light source <NUM> can be an infrared laser diode. Light source <NUM> emits light that passes through a first light shaping component, collimating lens <NUM>. Collimating lens <NUM> can be configured to focus light <NUM> emitted by light source <NUM> towards a second light shaping component, optical element <NUM>. Optical element <NUM> can include both a refractive surface <NUM> on a first side of optical element <NUM> and a diffractive surface <NUM> on a second side of optical element <NUM>. Refractive surfaces <NUM> and diffractive surfaces <NUM> can take the form of polymer material molded onto opposing sides of a glass or polycarbonate substrate. When collimated light <NUM> passes through refractive surface <NUM>, the light is tilted by an angle θ and elongated into a supergaussian beam <NUM> within optical element <NUM>. When the supergaussian beam passes through diffractive surface <NUM>, the super-gaussian beam <NUM> can be multiplied into multiple super-gaussian beams <NUM>. When super gaussian beams pass through micro-lens array <NUM>, micro-lens array <NUM> spreads each super gaussian beam <NUM> horizontally to create an illumination pattern that illuminates regions <NUM>, as depicted. In this way, light source <NUM> illuminates regions <NUM>.

<FIG> show a projector assembly <NUM> with two light sources that incorporates an optics assemblies for each light source similar to optics assembly <NUM>. <FIG> shows a top view of projection assembly <NUM>. Projection assembly <NUM> includes light sources <NUM> and <NUM>. Light sources <NUM> and <NUM> can both be mounted to rigid substrate <NUM>. In some embodiments, rigid substrate <NUM> can be formed from an alumina ceramic. Rigid substrate <NUM> keeps light sources <NUM> and <NUM> from shifting position relative to one another. Rigid substrate <NUM> can also have a low coefficient of thermal expansion that reduces shifting of light sources <NUM> and <NUM> with respect to the optics assemblies.

Light source <NUM> shines light through a first portion of dual collimating lens <NUM>, which focuses the light towards optics assembly <NUM>. A second portion of dual collimating lens <NUM> focuses light emitted by light source <NUM> towards optics assembly <NUM>. In some embodiments, dual collimating lens <NUM> can be replaced by two separate collimating lenses that accomplish the same function. Optics assembly <NUM> and <NUM> can each include a refractive optical element similar to <NUM>, a diffractive optical element similar to <NUM> and a micro-lens array similar to <NUM> for spreading the light from each light source in an illumination pattern. Optics assembly <NUM> can be slightly different from optics assembly <NUM>, making the illumination pattern generated by light source <NUM> vertically offset from the illumination pattern generated by light source <NUM> so that the illumination patterns are complementary. This allows the light bars from one illumination pattern to be positioned between the light bars of the other illumination pattern. In this way, the illumination patterns generated by light sources <NUM> and <NUM> cooperate to uniformly cover a surface. In some embodiments, the refractive optical element can shift light from light source <NUM> in an opposite direction from the light generated by light source <NUM>.

Projector assembly <NUM> can also include a processor <NUM> mounted on PCB <NUM> and configured to synchronize output from light sources <NUM> and <NUM>. For example, processor <NUM> can be mounted to PCB <NUM> and configured to direct light sources <NUM> and <NUM> to send out staggered pulses of light, so that neither illumination pattern is active at the same time. Processor <NUM> can also direct modulation of light sources <NUM> to help the depth sensor distinguish the pulses of light from other ambient light sources. In some embodiments, processor <NUM> can also be in communication with a sensor configured to receive the pulses of light after being reflected off objects within the sensor's field of view.

<FIG> shows a side view of projection assembly <NUM>. In particular, light source <NUM> is shown elevated by rigid substrate <NUM>. Rigid substrate can be inserted into a notch defined by PCB <NUM>. Rigid substrate <NUM> can also form a base for projector housing <NUM> of projector assembly <NUM>. Projector housing <NUM> can define a ledge <NUM> for supporting dual collimating lens <NUM>.

<FIG> show views of a multiple light source projector assembly <NUM> utilizing folded optics. <FIG> shows how projector assembly <NUM> includes two separate set of optics <NUM> and <NUM>, which emit light through folded optics <NUM> of collimating lens <NUM>. Folded optics <NUM> allows light sources <NUM> and <NUM> to be positioned closer to collimator <NUM> by shifting light path <NUM> laterally, thereby allowing an overall reduction in the height of projector assembly <NUM>.

<FIG> shows how by shifting the light path <NUM> laterally, the height of projector assembly <NUM> can be reduced, thereby allowing projector assembly <NUM> to be packaged within a smaller form-factor device. In particular, the laterally shifted light path <NUM> allows a length of the light path to be split into horizontal and vertical segments. The overall height of projector assembly <NUM> is reduced since the portion of the light path within the horizontal segment does not need to be incorporated within the overall height of projector assembly <NUM>. A direction of the light path through folded optics <NUM> is redirected by optically reflective surface <NUM>, which reorients the light from a horizontal orientation to a vertical orientation. In some embodiments, optically reflective surface <NUM> can be mirrored to.

<FIG> shows projector assembly <NUM>, which can have a shorter overall height than projector assembly <NUM>. Collimating lens <NUM> can include both folding optics <NUM> and cylindrical lens surface <NUM>. Cylindrical lens surface <NUM> can partially collimate the light emitted by light source <NUM> by narrowing the width of the light entering collimating lens <NUM>. Folded optics <NUM> can be shorter vertically due to the narrowed beam width of the light emitted by light source <NUM>. The light then becomes fully collimated upon exiting collimating lens <NUM>. In this way, a height of collimating lens <NUM> can be reduced.

<FIG> show side views of a projection assembly <NUM> using a single light source <NUM>. <FIG> shows projection assembly <NUM> in an inactive configuration. Because projection assembly <NUM> only includes a single light source <NUM>, in order to create two different illumination patterns projection assembly <NUM> includes a linearly actuated optic <NUM> configured to generate two complementary illumination patterns. Optic <NUM> can be linearly actuated by piezo-electric motor <NUM>, which actuates optic <NUM> between two or more positions by rotating linkage <NUM> two positions shown in <FIG>. Piezo-electric motor <NUM> can be configured to oscillate optic <NUM> back and forth at a rate allowing light source <NUM> to sequentially project complementary illumination patterns <NUM> and <NUM>. Light source <NUM> can be synchronized with the oscillation rate of optic <NUM> so that light source <NUM> emits light only when optic <NUM> is in a position corresponding to one of the complementary illumination patterns. It should be noted that while only two illumination patterns are shown that piezo-electric motor <NUM> can also be configured to define three or more different illumination patterns.

<FIG> shows a diagram depicting interaction between different components of the depth detection system described above. The top of the flow chart indicates the beginning of the interaction and progresses on moving down the flow chart. A projector of a depth detection system sends out alternating 1st and 2nd illumination patterns. Objects within a sensor field of view of the depth detection system reflect portions of the first and second illumination patterns back into the sensor of the depth detection system. The light travelling directly from the projector to the object and back (direct light) will arrive back at the sensor before light bouncing off another surface prior to returning to the sensor (indirect light) does. Consequently, a time of flight depth detection system will incorrectly increase the distance of an object from the sensor when indirect light is considered. The sensor then sends the light received from the first and second illumination patterns to the processor. The processor can then be configured to filter out indirect light from the total light received so that only direct light is considered when identifying the distance between the sensor and the objects within the sensor field of view. The processor can then assign the objects within the sensor field of view to appropriate depth planes of a display associated with the depth detection sensor. Finally, the processor can send imagery to depth planes corresponding to the various objects within the sensor field of view.

The various aspects, embodiments, implementations or features of the described embodiments can be used separately or in any combination. Various aspects of the described embodiments can be implemented by software, hardware or a combination of hardware and software. The described embodiments can also be embodied as computer readable code on a computer readable medium for controlling manufacturing operations or as computer readable code on a computer readable medium for controlling a manufacturing line. The computer readable medium is any data storage device that can store data, which can thereafter be read by a computer system. Examples of the computer readable medium include read-only memory, random-access memory, CD-ROMs, HDDs, DVDs, magnetic tape, and optical data storage devices. The computer readable medium can also be distributed over network-coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Claim 1:
A depth detection system (<NUM>), comprising:
a projection system (<NUM>), comprising:
a projector (<NUM>, <NUM>, <NUM>) housing having a rigid substrate (<NUM>),
a first light source (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) configured to emit light through a first plurality of light shaping components so as to produce a first illumination pattern (A, C, E, F), the first light source being mounted to the rigid substrate (<NUM>), and
a second light source (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) configured to emit light through a second plurality of light shaping components so as to produce a second illumination pattern (B, D, F, G) complementary to the first illumination pattern (A, C, E, F), the second light source (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) being mounted to the rigid substrate (<NUM>) adjacent to the first light source (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>), wherein the first and second illumination patterns (A-G) are configured to sequentially pulse such that only one of the first and second illumination patterns (A-G) is active at any given time;
an imaging sensor (<NUM>) proximate the projection system (<NUM>) and configured to receive light emitted by the first and second light sources (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) after being reflected off objects within a field of view of the imaging sensor (<NUM>); and
a processor (<NUM>) configured to:
cause the imaging sensor to record received light when the first illumination pattern (A, C, E, F) is active;
cause the imaging sensor to record received light when the second illumination pattern (B, D, F, G) is active;
to identify direct light, subtract a portion of the recorded light received by the imaging sensor (<NUM>) when the first illumination pattern (A, C, E, F) is active from light received by the imaging sensor (<NUM>) when the second illumination pattern (B, D, F, G) is active; and
calculate a distance based on the identified direct light between the depth detection system (<NUM>) and the objects within the field of view of the imaging sensor (<NUM>) by measuring an amount of time for light emitted by the second light source (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) to reflect off the objects within the field of view of the imaging sensor and return to the imaging sensor (<NUM>).