MULTIPATH DETECTION DEVICE AND MULTIPATH DETECTION METHOD

A multipath detection device includes: a signal controller; a light emitter; a light receiver; a data holder that holds reference data on a depth in a non-multipath environment; a signal processor that calculates a first depth and a second depth, the first depth being determined based on a ratio between (i) an amount of light received through light exposure during a first timing and (ii) an amount of light received through light exposure during a second timing, the second depth being determined based on a ratio between (iii) an amount of light received through light exposure during a third timing and (iv) an amount of light received through light exposure during a fourth timing; and a determiner that determines presence or absence of multipath using the reference data and a difference between the first depth and the second depth.

FIELD

The present disclosure relates to a multipath detection device and a multipath detection method for detecting the presence or absence of multipath when measuring a distance to a measurement target.

BACKGROUND

A conventional time-of-flight (TOF) camera system which measures a distance to a measurement target based on a time of flight taken by light to travel to and back from the measurement target is known. With this TOF camera system, measurement error increases and measurement precision deteriorates when there is a mixture of direct light that travels back directly from the measurement target and indirect light that travels back via an object different from the measurement target. To address this, Non-Patent Literature (NPL) 1, for example, proposes a TOF camera system which reduces the influence of multipath.

CITATION LIST

Non Patent Literature

SUMMARY

Technical Problem

To reduce the influence of multipath, it is necessary to determine whether the measurement environment is a multipath environment in which both direct light and indirect light are present, and then perform processing according to the presence or absence of multipath. The TOF camera system disclosed in NPL 1, however, has a problem that it involves a great amount of calculation for determining the presence or absence of multipath, that is, the processing load is heavy.

Solution to Problem

A multipath detection device according to the present disclosure includes: a signal controller that outputs a light emission control signal and a light exposure control signal; a light emitter that emits light in accordance with the light emission control signal; a light receiver that receives light through light exposure in accordance with the light exposure control signal; a data holder that holds reference data on a depth determined based on a ratio between: an amount of light received by the light receiver through light exposure in a non-multipath environment during a predetermined timing in accordance with a predetermined light emission control signal output from the signal controller; and an amount of light received by the light receiver through light exposure in the non-multipath environment during a timing different from the predetermined timing in accordance with the light emission control signal output from the signal controller in a time slot different from a time slot in which the predetermined light emission control signal is output; a signal processor that calculates a first depth and a second depth, the first depth being determined based on a ratio between (i) an amount of light received by the light receiver through light exposure during a first timing in accordance with a first light emission control signal output from the signal controller and (ii) an amount of light received by the light receiver through light exposure during a second timing different from the first timing in accordance with a second light emission control signal output from the signal controller in a time slot different from a time slot in which the first light emission control signal is output, the second depth being determined based on a ratio between (iii) an amount of light received by the light receiver through light exposure during a third timing in accordance with a third light emission control signal output from the signal controller and (iv) an amount of light received by the light receiver through light exposure during a fourth timing different from the third timing in accordance with a fourth light emission control signal output from the signal controller in a time slot different from a time slot in which the third light emission control signal is output; and a determiner that determines presence or absence of multipath using the reference data and a difference between the first depth and the second depth.

A multipath detection method according to the present disclosure includes: storing reference data on a depth determined based on a ratio between: an amount of light received through light exposure in a non-multipath environment during a predetermined timing in accordance with a predetermined light emission control signal; and an amount of light received through light exposure in the non-multipath environment during a timing different from the predetermined timing in accordance with a light emission control signal output in a time slot different from a time slot in which the predetermined light emission control signal is output; calculating a first depth based on a ratio between (i) an amount of light received through light exposure during a first timing in accordance with a first light emission control signal and (ii) an amount of light received through light exposure during a second timing different from the first timing in accordance with a second light emission control signal output in a time slot different from a time slot in which the first light emission control signal is output; calculating a second depth based on a ratio between (hi) an amount of light received through light exposure during a third timing in accordance with a third light emission control signal and (iv) an amount of light received through light exposure during a fourth timing different from the third timing in accordance with a fourth light emission control signal output in a time slot different from a time slot in which the third light emission control signal is output; and determining presence or absence of multipath using the reference data and a difference between the first depth and the second depth.

Advantageous Effects

With a multipath detection device and a multipath detection method according to the present disclosure, it is possible to reduce the processing load for determination of the presence or absence of multipath.

DESCRIPTION OF EMBODIMENT

Hereinafter, an embodiment will be specifically described with reference to the accompanying drawings. Note that the embodiment described below illustrates a specific example of the present disclosure. The numerical values, shapes, materials, constituent elements, the arrangement and connection of the constituent elements, steps, the processing order of the steps, etc. illustrated in the embodiment below are mere examples, and are therefore not intended to limit the present disclosure. Among the constituent elements indicated in the embodiment below, those not recited in any of the independent claims representing forms of realization according to an aspect of the present disclosure will be described as optional constituent elements. The forms of realization of the present disclosure are not limited to the current independent claims, and can be represented by other independent claims.

The drawings are represented schematically and are not necessarily precise illustrations. In the drawings, essentially the same constituent elements are given the same reference signs, and overlapping descriptions thereof may be omitted or simplified.

(Knowledge Forming the Basis of the Present Disclosure)

The following describes the knowledge forming the basis of the present disclosure with reference toFIG. 1toFIG. 13C. The knowledge forming the basis of the present disclosure includes new knowledge not found in the conventional technology as well as conventional knowledge forming the basis of the new knowledge.

1. Typical Distance Information Obtaining Device

First, a typical distance information obtaining device will be described with reference toFIG. 1.

FIG. 1is a block diagram illustrating an exemplary configuration of a typical distance information obtaining device. Note thatFIG. 1shows object OBJ which is a measurement target.

The distance information obtaining device is a time-of-flight (TOF) distance measurement device that measures the distance to a measurement target based on time of flight taken by light to travel to and back from the measurement target. The distance information obtaining device includes signal controller101, light emitter102, light receiver103, and signal processor104.

Signal controller101outputs, to light emitter102, a light emission control signal that controls light emission performed by light emitter102. Signal controller101also outputs, to light receiver103, a light exposure control signal that controls light exposure performed by light receiver103.

In accordance with an emission pulse of the light emission control signal, light emitter102emits light, that is, emits irradiation light. The irradiation light is near-infrared light, for example. The irradiation light reflects off object OBJ and travels back to the distance information obtaining device as reflected light.

Light receiver103is a solid-state imaging element that includes a plurality of pixels arranged in rows and columns. Light receiver103receives reflected light in accordance with an exposure pulse of the light exposure control signal, and outputs a light reception signal to signal processor104.

Signal processor104calculates depth D and luminance B for each pixel of light receiver103based on a light reception signal sequence obtained through three types of emission and exposure processing which will be described later. A distance can be calculated based on depth D. The method for calculating depth D and luminance B will be described later.

Next, operations of the distance information obtaining device in a non-multipath environment will be described. Note that the non-multipath environment refers to an environment in which no indirect light is present and only direct light is present,

FIG. 2is a timing diagram illustrating an example of operations of the distance information obtaining device in a non-multipath environment.

FIG. 2illustrates a waveform of light emission control signal (or irradiation light)1A having an emission puke, a waveform of incident light1C that enters light receiver103, and a waveform of light exposure control signal1D having an exposure puke. The waveform of incident light1C is represented by an amount of the light reception signal received by light receiver103. Note that this example assumes that the shapes of the waveforms of the light emission control signal and the irradiation light are substantially the same.

The emission pulse is the positive logic that represents active when it is high level, whereas the exposure pulse is the negative logic that represents active when it is low level. Incident light1C includes background light and reflected light that is a portion of the irradiation light that travels to object OBJ and travels back by being reflected by object OBJ. The reflected light enters light receiver103with a delay of a predetermined time period from the start of emission of the irradiation light. This delay time period depends on the distance from the distance information obtaining device to object OBJ. The (one) dot hatching region and the (two) diagonally shaded hatching regions of incident light1C illustrated inFIG. 2correspond to the amount of the light reception signal of each pixel.

The emission and exposure processing performed for distance measurement is implemented through, for example, S0exposure in which light emission and light exposure are performed, S1exposure in which light emission and light exposure are performed in a time slot different from the S0exposure, and BG exposure in which light emission and light exposure are performed in a time slot different from the S0exposure and the S1exposure. Note that in the S0exposure and the S1exposure, the start time t of light emission control signal1A is 0, whereas in the BG exposure, the start time t of light exposure control signal1D is 0.

In the S0exposure, the exposure pulse becomes active at the same time as the start of the emission pulse. In other words, the light exposure starts simultaneously with the light emission (t=0). Pulse width TS1of the exposure pulse is set to be twice or more than twice as large as pulse width TLof the emission pulse. In the S0exposure, the entirety of the reflected light can be received, for example.

In the S1exposure, the exposure pulse becomes active at the same time as the end of the emission pulse. In other words, the light exposure starts when the light emission ends (t=TL). Pulse width TS1of the exposure pulse is the same as the pulse width in the S0exposure. In the S1exposure, of the entire reflected light, reflected light that has entered after the end of the emission pulse can be received, for example.

In the BG exposure, the exposure pulse becomes active with no generation of the emission pulse. In other words, in the BG exposure, background light, which does not include the reflected light, is received. Pulse width TS1of the exposure pulse is the same as the pulse width in the S0exposure and the S1exposure.

Note that, in practice, each processing in the S0exposure, S1exposure, and BG exposure is performed using a plurality of emission pulses or a plurality of exposure pulses.FIG. 2illustrates a result of accumulation of each processing.

Signal processor104calculates distance L and luminance B for each pixel, using the amount of the light reception signal (the amount of signal charge generated by light reception) of the pixel in the S0exposure, S1exposure, and BG exposure.

Here, distance L of each pixel is calculated by Equation 1, where amount of light received S0, amount of light received S1, and amount of light received BG denote the amount of the light reception signal of each pixel in the S0exposure, the amount of the light reception signal of each pixel in the S1exposure, and the amount of the light reception signal of each pixel in the BG exposure, respectively. Depth D is the second term on the right side of Equation 1, and is calculated by dividing the amount of light received (S1−BG) by the amount of light received (S0−BG). Luminance B of each pixel is calculated by Equation 2. Note that c denotes the light speed (approximately 299,792,458 m/s) and TLdenotes the puke width of the emission pulse.

Next, another example of operations of the distance information obtaining device in a non-multipath environment will be described.

FIG. 3is a timing diagram illustrating another example of operations of the distance information obtaining device in a non-multipath environment.

FIG. 3illustrates a waveform of light emission control signal (or irradiation light)2A, a waveform of incident light2C, and a waveform of light exposure control signal2D. The example illustrated inFIG. 3is different from the example illustrated inFIG. 2in that the pulse width of the exposure pulse is the same as the pulse width of the emission pulse. In this case, distance L of each pixel is calculated by Equation 3, and luminance B by Equation 4.

2, Mechanism of Measurement Error

Next, the mechanism of how measurement errors occur in a multipath environment will be described with reference toFIG. 4andFIG. 5.

FIG. 4illustrates an example of a multipath environment in which both direct light and indirect light are present.

Object OBJ1illustrated inFIG. 4is the measurement target, and object OBJ2is the cause of indirect light. Distance measurement result OBJ1E is an image formed due to a measurement error caused by multipath when the distance information obtaining device carries out measurement on object OBJ1.

FIG. 4illustrates the paths of direct light and indirect light. The path of direct light is a path passing through object OBJ1, and is a path along which: direct irradiation light (D−Path1) becomes direct reflected light (D−Path2) by being reflected by object OBJ1; and the direct reflected light (D−Path2) reaches pixel103aof light receiver103.

The path of indirect light is a path passing through object OBJ2and object OBJ1, and is a path along which: indirect irradiation light (M−Path1) becomes indirect irradiation light (M−Path2) by being reflected by object OBJ2and further becomes indirect reflected light (M−Path3) by being reflected by object OEM; and the indirect reflected light (M−Path3) reaches pixel103aof light receiver103.

FIG. 5is a timing diagram illustrating an example of operations of the distance information obtaining device in a multipath environment.

FIG. 5illustrates a waveform of light emission control signal (or irradiation light)3A, a waveform of direct reflected light (D−Path2)3C1, a waveform of indirect reflected light (M−Path3)3C2, a waveform of mixed reflected light that is a sum of direct reflected light3C1and indirect reflected light3C2, and a waveform of light exposure control signal3D. Among these, the waveforms of light emission control signal3A, direct reflected light3C1, and light exposure control signal3D are the same as the waveforms of light emission control signal2A, incident light2C, and light exposure control signal2D illustrated inFIG. 2. That is to say,FIG. 5isFIG. 2plus the waveforms of indirect reflected light3C2and mixed reflected light.

Amount of light received S0in the S0exposure is a sum of amount of light received D0corresponding to direct reflected light3C1and amount of light received M0corresponding to indirect reflected light3C2. Likewise, amount of light received S1in the S1exposure is a sum of amount of light received D1acorresponding to direct reflected light3C1and amount of light received M1corresponding to indirect reflected light3C2. With these plugged into Equation 1, distance L of each pixel can be given by Equation 5.

The following describes the result of application of the above example to the other example inFIG. 3in which the exposure pulse and the emission pulse have the same pulse width. That is to say, amount of light received S0in the S0exposure is a sum of amount of light received D0corresponding to direct reflected light3C1and amount of light received M0corresponding to indirect reflected light3C2, whereas amount of light received S1in the S1exposure is a sum of amount of light received D1acorresponding to direct reflected light3C1and amount of light received M1corresponding to indirect reflected light3C2. With these plugged into Equation 3, distance L of each pixel in the example case illustrated inFIG. 3can be calculated by Equation 6.

Amounts of light received M0and M1corresponding to indirect reflected light3C2in Equation 5 are values dependent not only on the distance to object OBJ1but also on the location and reflectance of a peripheral object. Thus, distance L calculated by Equation 5 includes an unpredictable measurement error and causes deterioration of the measurement accuracy in small or large degrees. The same applies to Equation 6.

3. Actual Operations of Distance Information Obtaining Device

Next, actual operations of the distance information obtaining device will be described with reference toFIG. 6toFIG. 7C. Note that a non-multipath environment will be described here.

FIG. 6is a timing diagram illustrating an example of actual operations of the distance information obtaining device in a non-multipath environment.

FIG. 6illustrates a waveform of light emission control signal4A, a waveform of irradiation light4B, a waveform of incident light4C, and a waveform of light exposure control signal4D. Among these, the waveforms of light emission control signal4A and light exposure control signal4D are the same as the waveforms of light emission control signal1A and light exposure control signal1D illustrated inFIG. 2. That is to say,FIG. 6isFIG. 2plus the waveform of irradiation light4B, The waveforms of irradiation light4B and incident light4C illustrated inFIG. 6are waveforms resulting from distortion of pulse waveforms generated in actual operations.

Light emission control signal4A is a control signal that causes light emission to start at time t=0 and finish at time t=Tr. Correspondingly, the waveform of irradiation light4B actually emitted from light emitter102gradually rises from start time t=0, gradually falls from finish time t=Tr, and reaches the bottom at time t=Tr+Tf. Time Trcorresponds to the pulse rising period, whereas time Tfcorresponds to the puke falling period.

The rising period and the falling period are generated by a control circuit of the distance information obtaining device. The reason why the rising period and the falling period are taken into consideration is because the puke width of the emission pulse according to the present disclosure is of the order of nanoseconds (nsec). For example, when measuring a distance of from 0 m to 3 m, since the round-trip time of light is approximately 20 nsec, the puke width of the emission puke needs to be set to 20 nsec. When the puke width of the emission puke is short as in this case, the rising period and the falling period cannot be ignored. Thus, in practice, the waveform of irradiation light4B emitted by light emitter102becomes a distorted puke waveform that monotonically increases and then monotonically decreases as illustrated inFIG. 6. Note that the waveform of irradiation light4B may be shown with a triangular waveform or a sawtooth waveform. As described above, since the waveform of irradiation light4B is a pulse waveform having a rising period and a falling period, the waveform of incident light4C also has a rising period and a falling period. In the present disclosure, the shape of this waveform is used to detect the presence or absence of multipath.

FIG. 7Ais a graph illustrating a time waveform of emission intensity I of irradiation light4B.

As illustrated inFIG. 7A, emission intensity I at time (point in time) t is expressed by Equations 7 to 10, given that emission start time t of irradiation light4B is 0.

In Equation 8, Trdenotes time constant for the rising of the emission pulse. In Equation 9, Tfdenotes time constant for the falling of the emission pulse, Trdenotes a rising period, and Tfdenotes a falling period, and a denotes emission intensity at the start of falling. As for the waveform in the graph illustrated inFIG. 7A, time constant Tr=3.0 nsec, time constant Tf=1.4 nsec, rising period Tr=12.7 nsec, and falling period Tf=7.3 nsec.

FIG. 7Bis a graph illustrating a time waveform of the amount of reflected light received in a non-multipath environment. Note that the amount of reflected light received is an amount of light obtained by subtracting background light from incident light4C inFIG. 6.

As illustrated inFIG. 7B, amount of light received Rdirectat time (point in time) t is expressed by Equation 11, given that emission start time t of irradiation light4B is 0 and the light round-trip time of direct light taken by irradiation light4B to travel to the measurement target and travel back as direct reflected light is t1.

In Equation 11, at1denotes received-light intensity at the start of falling. Since light attenuates by the square of distance and is dependent on the reflectance of the measurement target, received-light intensity at1is an unknown. As indicated by Equation 11, the waveform of amount of light received Rdirectis a waveform obtained by attenuating, by received-light intensity at1which is an unknown, a waveform obtained by shifting waveform f(t) in Equations 8 to 10 by light round-trip time t1to and from the measurement target.

Here, amount of light received (S0−BG) which is obtained through the S0exposure inFIG. 6is expressed by Equation 12, where TS1=2×(Tr+Tf) denotes the pulse width of the exposure pulse inFIG. 6, and S denotes the area size of waveform f(t) when emission intensity I and emission intensity a are 1 in Equation 7.

Amount of light received (S1−BG) which is obtained through the S1exposure inFIG. 6is expressed by Equation 13.

As described above, depth D is the second term on the right side of Equation 1, and is calculated by dividing amount of light received (S1−BG) by amount of light received (S0−BG). Thus, when calculating depth D, unknown received-light intensity at1is offset, enabling calculation of light round-trip time t1. Once light round-trip time t1is calculated, distance L can be calculated by Equation 14.

FIG. 7Cis a graph illustrating a relationship between depth D and light round-trip time t from the emission of irradiation light4B to the reception of reflected light in a non-multipath environment,FIG. 7Dis a graph illustrating a relationship between depth D and depth slope α shown inFIG. 7C.FIG. 7Dwill be described in detail later.

InFIG. 7C, a curve passing through coordinates (t1, D(t1)) is drawn with a solid line, given that D(0) denotes the depth when light is received with light round-trip time t=0, and D(t1) denotes the depth when light is received with light round-trip time t=Depth D and light round-trip time t1have a one-to-one relationship, and light round-trip time t1and distance L also have a one-to-one relationship. Thus, when depth D(t1) is known, light round-trip time t1is also known, and distance L can be calculated by Equation 14. Also, luminance B can be calculated by Equation 12.

When the above example is applied to the other example inFIG. 3in which the exposure pulse and the emission pulse have the same pulse width, amount of light received (S0−BG) which is obtained through the S0exposure is calculated by Equation 15 shown below.

Meanwhile, amount of light received S1which is obtained through the S1exposure in the other example inFIG. 3is the same as that in Equation 13 because the amount of reflected light received is the same.

Depth D in the other example inFIG. 3is the second term on the right side of Equation 3, and is calculated by dividing amount of light received (S1−BG) by a sum of amount of light received (S0−BG) and amount of light received (S1−BG). Equation 16 is given by adding Equation 15 and Equation 13 that form the denominator of the second term on the right side of Equation 3.

Because Equation 16 is equal to Equation 12, distance L can be calculated by Equation 14 in the same manner. Note that luminance B can be calculated by Equation 16.

4. Actual Operations of Distance Information Obtaining Device in Multipath Environment

Next, actual operations of the distance information obtaining device in a multipath environment will be described with reference toFIG. 8toFIG. 13C.

Examples of multipath include a first example in which indirect reflected light reaches light receiver103later than direct reflected light, and a second example in which indirect reflected light reaches light receiver103earlier than direct reflected light.

First, the first example of multipath will be described.

As illustrated inFIG. 4described above, multipath occurs when object OBJ2different from object OBJ1is located ahead of object OBJ1, and a measurement error is thereby generated.

FIG. 8is a timing diagram illustrating an example of operations of the distance information obtaining device in a multipath environment in which indirect reflected light travels back later than direct reflected light.

FIG. 8illustrates a waveform of light emission control signal5A, a waveform of irradiation light5B, a waveform of direct reflected light (D−Path2)5C1, a waveform of indirect reflected light (M−Path3)5C2, a waveform of incident light5C, and a waveform of light exposure control signal5D.

Incident light5C includes mixed reflected light and background light. The mixed reflected light is a sum of (i) direct reflected light that is a portion of irradiation light5B that travels to object OBJ1and travels back through reflection by object OBJ1and (ii) indirect reflected light that is a portion of irradiation light53that travels to object OBJ1via object OBJ2and travels back through reflection by object OBJ1. Among these, the waveforms of light emission control signal5A, irradiation light53, direct reflected light5C1, and light exposure control signal5D are the same as the waveforms of light emission control signal4A, irradiation light4B, reflected light of incident light4C, and light exposure control signal4D illustrated inFIG. 6. That is to say,FIG. 8isFIG. 6plus the waveforms of indirect reflected light5C2and incident light5C,

FIG. 9Ais a graph illustrating time waveforms of received-light intensities of direct reflected light5C1and indirect reflected light5C2in the multipath environment illustrated inFIG. 8.

As illustrated inFIG. 9A, indirect reflected light5C2is temporally delayed because it travels along a path longer than the path of direct reflected light5C1. Amount of light received Rindirectof indirect reflected light5C2at time (point in time) t is expressed by Equation 17, where t1denotes the light round-trip time of direct reflected light5C1as illustrated inFIG. 7Band Equation 11, and multipath delay time tbdenotes delay time of indirect reflected light5C2with respect to light round-trip time t1.

Here, bt1is the received-light intensity of indirect reflected light5C2at the start of falling. Since light attenuates by the square of distance and is dependent on the reflectance of the measurement target and the reflectance of an object located on the path that the light passes through, received-light intensity bt1is an unknown,

FIG. 98is a graph illustrating a time waveform of the amount of light received of the mixed reflected light that is a sum of direct reflected light5C1and indirect reflected light5C2. In practice, direct reflected light5C1and indirect reflected light5C2cannot be received separately as illustrated inFIG. 9A, and thus, the waveform illustrated inFIG. 98is observed. Amount of light received Rmixof the mixed reflected light at time (point in time) t is expressed by Equation 18.

Here, amount of light received (S0−BG) which is obtained through the S0exposure inFIG. 8is expressed by Equation 19, where TS1=2×(Tr+Tf) denotes the pulse width of the exposure pulse inFIG. 8, and S denotes the area size of waveform f(t) when emission intensity I and emission intensity a are 1.

Amount of light received (S1−BG) which is obtained through the S1exposure inFIG. 8is expressed by Equations 20 and 21.

Depth D is the second term on the right side of Equation 1, and is calculated by dividing amount of light received (S1−BG) in Equation 20 by amount of light received (S0−BG) in Equation 19; however, in Equation 21 which is a part of Equations 18 and 20, unknown received-light intensity bt1is greater than 0, and multipath delay time tbis also greater than 0, which means that the effects of unknown received-light intensity bt1and multipath delay time tbcannot be offset. As a result, depth D becomes a large value as compared to the value of depth D when only the direct light is received.

FIG. 9Cis a graph illustrating a relationship between depth D and light round-trip time t from the emission of irradiation light to the reception of mixed reflected light in the multipath environment illustrated inFIG. 8.FIG. 9Dis a graph illustrating a relationship between depth D and depth slope α shown inFIG. 9C.

InFIG. 9C, a curve passing through coordinates (t1, D0(t1)) is drawn with a solid line, given that the depth is 0 when the mixed reflected light is received with light round-trip time t=0, and the depth is D0(t1) when the mixed reflected light is received with light round-trip time t=t1. Also, inFIG. 9C, a curve passing through coordinates (t1, Dref(t1)) is drawn with a dotted line, given that the depth is 0 when only direct reflected light5C1is received with light round-trip time t=0, and the depth is Dref(t1) when only direct reflected light5C1is received with light round-trip time t=t1. Note that this dotted line is the same as the curve illustrated inFIG. 7C. With use of these relationships, multipath detection device100according to the present disclosure is capable of detecting the presence or absence of multipath by determining whether or not the depth of reflected light is the same as the depth when only the direct light is present, that is, by determining whether or not depth D0(t1) is unequal to depth Dref(t1), as illustrated inFIG. 9D.FIG. 9Dwill be described in detail later.

Next, the following describes the second example of multipath in which indirect reflected light reaches light receiver103earlier than direct reflected light. The second example is also called a flare.

FIG. 10illustrates an example of a multipath environment in which indirect reflected light travels back earlier than direct reflected light.

InFIG. 10, lens109is disposed in front of light receiver103. Object OBJ1is the measurement target, and object OBJ2is the cause of indirect light. Distance measurement result OBJ1E is an image formed due to a measurement error caused by multipath when the distance information obtaining device carries out measurement on object OBJ1.

FIG. 10illustrates paths of two rays of direct light and a path of a single ray of indirect light.

The path of the first ray of direct light is a path passing through object OBJ1, and is a path along which: direct irradiation light (D1−Path1) becomes direct reflected light (D1−Path2) by being reflected by object OBJ1; and the direct reflected light (D1−Path2) reaches pixel103aof light receiver103via lens109.

The path of the second ray of direct light is a path passing through object OBJ2, and is a path along which: irradiation light (D2−Path1) becomes reflected light (D2−Path2) by being reflected by object OBJ2; and the reflected light (D2−Path2) reaches pixel103bof light receiver103via lens109.

The path of the indirect light is a path along which light is reflected by lens109, that is, a path along which indirect reflected light (M−Path1) reflected by pixel103bscatters at lens109and reaches pixel103aas indirect reflected light (M−Path2),

FIG. 11is a timing diagram illustrating an example of operations of the distance information obtaining device in a multipath environment in which indirect reflected light6C2travels back earlier than direct reflected light6C1,

FIG. 11illustrates a waveform of light emission control signal6A, a waveform of irradiation light6B, a waveform of direct reflected light (D1−Path2)6C1, a waveform of indirect reflected light (M−Path2)6C2, a waveform of incident light6C, and a waveform of light exposure control signal6D.

Incident light6C includes mixed reflected light and background light. The mixed reflected light is a sum of direct reflected light6C1and indirect reflected light6C2. Among these, the waveforms of light emission control signal6A, irradiation light6B, direct reflected light6C1, and light exposure control signal6D are the same as the waveforms of light emission control signal4A, irradiation light43, reflected light of incident light4C, and light exposure control signal4D illustrated inFIG. 6. That is to say,FIG. 11isFIG. 6plus the waveforms of indirect reflected light6C2and incident light6C,

FIG. 12Ais a graph illustrating time waveforms of received-light intensities of direct reflected light6C1and indirect reflected light6C2in the multipath environment illustrated inFIG. 11,FIG. 12Bis a graph illustrating a time waveform of mixed reflected light that is a sum of direct reflected light6C1and indirect reflected light6C2.

As illustrated inFIG. 12A, indirect reflected light6C2is temporally earlier because it travels along a path shorter than the path of direct reflected light6C1. Direct reflected light6C1is as indicated in Equation 11, and indirect reflected light6C2, as opposite to the first example inFIG. 7B, travels back earlier by time tb, and thus can be calculated in the same manner by setting unknown time tbto be less than 0 in Equations 17 to 21.

Depth D is the second term on the right side of Equation 1, and is calculated by dividing amount of light received (S1−BG) in Equation 20 by amount of light received (S0−BG) in Equation 19; however, in Equation 21 which is a part of Equations 18 and 20, unknown received-light intensity bt1is greater than 0, and multipath advance time tbis less than 0, which means that the effects of unknown received-light intensity bt1and multipath advance time tbcannot be offset. As a result, depth D becomes a small value as compared to the value of depth D when only the direct light is received.

FIG. 12Cis a graph illustrating a relationship between depth D and light round-trip time t from the emission of irradiation light6B to the reception of the mixed reflected light in a multipath environment illustrated inFIG. 11.FIG. 12Dis a graph illustrating a relationship between depth D and depth slope α shown inFIG. 12C.

InFIG. 12C, a curve passing through coordinates (t1, D0(t1)) is drawn with a solid line, given that the depth is 0 when the mixed reflected light is received with light round-trip time t=0, and the depth is D0(t1) when the mixed reflected light is received with light round-trip time t=t1.FIG. 12Calso illustrates, with a dotted line, the curve shown inFIG. 7C, With use of these relationships, it is possible, also in the second example, to detect the presence or absence of multipath by determining whether or not the depth of reflected light is the same as the depth when only the direct light is present, as illustrated inFIG. 12D.FIG. 12Dwill be described in detail later.

Note that although the multipath described thus far is the case of including indirect reflected light with a single path, the present disclosure is not limited to this example; the present disclosure also encompasses the case of including indirect reflected light with a plurality of paths as illustrated below.

FIG. 13Ais an explanatory diagram illustrating a situation in which a plurality of rays of indirect light are generated.

Object OBJ1illustrated inFIG. 13Ais the measurement target, and object OBJ2and object OBJ3are the causes of indirect light. Distance measurement result OBJ1E is an image formed due to a measurement error caused by multipath when the distance information obtaining device carries out measurement on object OBJ1. In this diagram, the path of direct light is drawn with a solid line, whereas the paths of three rays of indirect light passing through object OBJ2and the paths of three rays of indirect light passing through object OBJ3are drawn with dashed lines.

FIG. 13Bis a graph illustrating time waveforms of the amounts of the plurality of rays of indirect light received.

FIG. 13Bis a graph illustrating time waveforms of the amounts of light received when the six rays of indirect light inFIG. 13Aare assumed to be separately received by light receiver103. This graph shows that the longer the path of the indirect light is, the later the amount of light received rises and the lower the height of the amount of light received becomes.

FIG. 13Cis a graph illustrating a time waveform of a total amount of the plurality of rays of indirect light received.

As illustrated inFIG. 13C, even the waveform representing the total amount of the six rays of indirect light received shows the same waveform tendency as that shown inFIG. 73. Thus, even when a plurality of rays of indirect light are generated, the plurality of rays of indirect light can approximate indirect light of a single path described above. That is to say, the present knowledge can be applied even when a plurality of rays of indirect light are generated.

5. Method of Determining the Presence or Absence of Multipath

Next, a method of determining the presence or absence of multipath will be described with reference toFIG. 7D,FIG. 9D, andFIG. 12D. This method of determining the presence or absence of multipath focuses on depth slope α or a difference between two depths.

Slope α(t1) of depth D is a value obtained by differentiating depth D with respect to predetermined light round-trip time t1, and is expressed by Equation 22.

As can be understood from Equation 22, slope α(t1) is calculated by inverting the waveform in a direction opposite to time t and shifting the waveform by time Tr, and performing normalization to make the waveform area size 1 by the term 1/S.

FIG. 7Dis a graph illustrating the relationship between depth D and depth slope α shown inFIG. 7C, with the vertical axis and the horizontal axis reversed. Depth slope α obtained by differentiation of depth D is expressed by Equation 23.

In Equation 23, depth slope α can be calculated by assigning 0 to Δt, but in reality, it is difficult to assign 0 to Δt, and the influence of noise increases as Δt approaches 0. Thus, Δt is set to a relatively large value, for example. Specifically, Δt is set to, for example, a half or a third of rising period Trof the emission pulse. Note that instead of using depth slope α, it is possible to use the difference between two depths, with a fixed value given to Δt. This reduces the load of calculation caused by division, thus further enabling reduction in the processing load.

FIG. 9Dis a graph illustrating the relationship between depth D and depth slope α shown inFIG. 9C, with the vertical axis and the horizontal axis reversed.FIG. 9Dillustrates depth slope α0and depth D0(α0) when mixed reflected light is received with light round-trip time t=t1. The dotted line inFIG. 9Dshows, as reference data Dref, a relationship between the depth and the depth slope when only direct reflected light shown inFIG. 7Dis received.

As illustrated inFIG. 9D, depth D0(α0) corresponding to depth slope α0calculated with mixed reflected light is different from depth Dref(α0) corresponding to the same slope α0. As described above, it is possible to determine the presence or absence of multipath by, for example, holding in advance, as reference data Dref, depth D and its slope α when only direct reflected light is received, and determining whether or not the depth obtained by actual measurement matches reference data Dref.

FIG. 12Dis a graph illustrating the relationship between depth D and depth slope α shown inFIG. 12C, with the vertical axis and the horizontal axis reversed.FIG. 12Dillustrates depth slope α0and depth D0(α0) when mixed reflected light is received with light round-trip time t=t1. The dotted line inFIG. 12Dshows, as reference data Dref, a relationship between the depth and the depth slope when only direct reflected light shown inFIG. 7Dis received.

As illustrated inFIG. 12D, depth D0(α0) corresponding to depth slope α0calculated with mixed reflected light is different from depth Dref(α0) corresponding to the same slope α0. In the example case ofFIG. 12D, too, it is possible to determine the presence or absence of multipath by, for example, holding in advance, as reference data Dref, depth D and its slope α when only direct reflected light is received, and determining whether or not the depth obtained by actual measurement matches reference data Dref.

Working Example 1

1-1. Configuration of Multipath Detection Device

Based on the knowledge forming the basis of the present disclosure, the configuration of multipath detection device100according to Working Example 1 will be described with reference toFIG. 14andFIG. 15.

FIG. 14is a block diagram illustrating an exemplary configuration of multipath detection device100according to Working Example 1. Note thatFIG. 14schematically illustrates object OBJ, irradiation light, and reflected light as well.

Multipath detection device100is a TOF distance measurement device. Multipath detection device100includes signal controller101, light emitter102, light receiver103, signal processor104, pulse setter111, determiner112, and data holder113, Note that these functions of multipath detection device100are implemented by a microcomputer, a microcontroller, or a digital signal processor (DSP). The microcomputer, microcontroller, or DSP includes memory that stores a program for multipath detection and a central processing unit (CPU) that runs the program.

Pulse setter111outputs, to signal controller101, a pulse setting signal for setting the emission pulse and the exposure pulse.

Signal controller101outputs, to light emitter102, a light emission control signal that controls light emission performed by light emitter102. Signal controller101also outputs, to light receiver103, a light exposure control signal that controls light exposure performed by light receiver103.

In accordance with the emission pulse of the light emission control signal, light emitter102emits light, that is, emits irradiation light. The irradiation light is near-infrared light, for example. The irradiation light reflects off object OBJ and travels back to multipath detection device100as reflected light.

Light receiver103is a solid-state imaging element that includes a plurality of pixels arranged in rows and columns. Light receiver103receives reflected light in accordance with the exposure pulse of the light exposure control signal, and outputs a light reception signal to signal processor104.

Signal processor104calculates first depth D1, second depth D2, first luminance31, and second luminance32for each pixel of light receiver103based on a light reception signal sequence obtained through three types of emission and exposure processing.

FIG. 15is a timing diagram illustrating operations of multipath detection device100according to Working Example 1.FIG. 15illustrates a waveform of light emission control signal7A, a waveform of irradiation light7B, a waveform of incident light7C, and a waveform of light exposure control signal7D.

Multipath detection device100performs the following exposures to calculate depth slope α: S0exposure, S1exposure, and BG exposure that are performed in a first period, and S0exposure, S1exposure, and BG exposure that are performed in a second period different from the first period. Note that the settings for the S0exposure, S1exposure, and BG exposure in the first period are the same as the settings for the exposures inFIG. 6.

Each of first light emission control signal Es1, second light emission control signal Es2, third light emission control signal Es3, and fourth light emission control signal Es4illustrated inFIG. 15is a light emission control signal that is output from signal controller101. Signal controller101outputs first light emission control signal Es1and second light emission control signal Es2in different time slots in the first period, and outputs third light emission control signal Es3and fourth light emission control signal Es4in different time slots in the second period.

Each of first timing Tm1, second timing Tm2, third timing Tm3, and fourth timing Tm4is an exposure timing controlled by light exposure control signal7D. Signal controller101outputs light exposure control signal7D corresponding to first timing Tm1and second timing Tm2in the first period, and outputs light exposure control signal7D corresponding to third timing Tm3and fourth timing Tm4in the second period.FIG. 15illustrates amount of light received R1that is an amount of light received during first timing Tm1, amount of light received R2that is an amount of light received during second timing Tm2, amount of light received R3that is an amount of light received during third timing Tm3, and amount of light received R4that is an amount of light received during fourth timing Tm4.

As illustrated inFIG. 15, the start time of third timing Tm3which is based on third light emission control signal Es3is different from the start time of first timing Tm1which is based on first light emission control signal Es1, and is later than the start time of first timing Tm1by time Δt. Also, the start time of fourth timing Tm4which is based on fourth light emission control signal Es4is different from the start time of second timing Tm2which is based on second light emission control signal Es2, and is later than the start time of second timing Tm2by time Δt. In other words, the difference between the start time of fourth timing Tm4which is based on fourth light emission control signal Es4and the start time of second timing Tm2which is based on second light emission control signal Es2is the same as the difference between the start time of third timing Tm3which is based on third light emission control signal Es3and the start time of first timing Tm1which is based on first light emission control signal Es1.

First depth D1and second depth D2are calculated by signal processor104in the manner described below.

For example, first depth D1is calculated based on a ratio between (i) amount of light received R1that is an amount of light received by light receiver103through light exposure during first timing Tm1in response to first light emission control signal Es1output from signal controller101and (ii) amount of light received R2that is an amount of light received by light receiver103through light exposure during second timing Tm2in response to second light emission control signal Es2output from signal controller101. Note that second timing Tm2is different from first timing Tm1and starts later than first timing Tm1by time Tr.

Second depth D2is calculated based on a ratio between amount of light received R3that is an amount of light received by light receiver103through light exposure during third timing Tm3in response to third light emission control signal Es3output from signal controller101; and amount of light received R4that is an amount of light received by light receiver103through light exposure during fourth timing Tm4in response to fourth light emission control signal Es4output from signal controller101. Note that fourth timing Tm4is different from third timing Tm3and starts later than third timing Tm3by time Tr.

Data holder113holds in advance reference data Drefin a non-multipath environment. Reference data Drefis data on depth. This data on depth is calculated based on a ratio between: an amount of light received by light receiver103through light exposure during a predetermined timing in response to a predetermined light emission control signal output from signal controller101; and an amount of light received by light receiver103through light exposure during a timing different from the predetermined timing in response to a light emission control signal output from signal controller101in a time slot different from the time slot in which the predetermined light emission control signal is output.

Determiner112determines the presence or absence of multipath based on reference data Drefand the difference between first depth D1and second depth D2output from signal processor104. Specifically, determiner112calculates reference depth Dref(α0) in a non-multipath environment based on: depth slope α calculated based on the difference between first depth D1and second depth D2; and reference data Drefheld by data holder113. Determiner112then determines the presence or absence of multipath based on the magnitude of the difference between first depth D1and reference depth Dref(α0).

When doing so, a difference between two depths can be used instead of depth slope α. Specifically, determiner112may calculate a reference depth in a non-multipath environment based on: the difference between first depth D1and second depth D2; and reference data Drefheld by data holder113, and determine the presence or absence of multipath based on the magnitude of the difference between first depth D1and reference depth Dref(α0).

Note that, in the above example, the presence or absence of multipath is determined based on the magnitude of the difference between first depth D1and reference depth Dref(a0); however, the present disclosure is not limited to this example, and the presence or absence of multipath may be determined based on the magnitude of the difference between second depth D2and reference depth Dref(α0). Furthermore, reference data Drefmay be generated by an equation that uses Equations 12 and 13, or may be generated by actual measurement using different measurement target distances for the S0exposure, S1exposure, and BG exposure of the first period and for the S0exposure, S1exposure, and BG exposure of the second period described above.

1-2. Multipath Detection Method

FIG. 16is a flowchart illustrating a multipath detection method according to Working Example 1.

First, as preparation for multipath detection, multipath detection device100stores, in data holder113, reference data Drefin a non-multipath environment (Step S10). Reference data Drefin a non-multipath environment is light reception signal sequence data obtained when only direct reflected light is received, and is shown by a graph of depth D and slope α generated with direct reflected light only, as exemplified byFIG. 7D.

Subsequently, multipath detection device100performs the S0exposure, S1exposure, and BG exposure in the first period (Step S11). Specifically, light receiver103obtains amount of light received R1through light exposure during first timing Tm1, and obtains amount of light received R2through light exposure during second timing Tm2. Amount of light received R1and amount of light received R2are output to signal processor104.

Next, signal processor104calculates first depth D1based on the ratio between amount of light received R1and amount of light received R2(Step S12). First depth D1is calculated by determining the second term on the right side of Equation 1.

Subsequently, multipath detection device100performs the S0exposure, S1exposure, and BG exposure in the second period (Step S13). Specifically, light receiver103obtains amount of light received R3through light exposure during third timing Tm3, and obtains amount of light received R4through light exposure during fourth timing Tm4. Amount of light received R3and amount of light received R4are output to signal processor104.

Next, signal processor104calculates second depth D2using the ratio between amount of light received R3and amount of light received R4(Step S14), Second depth D2is calculated by determining the second term on the right side of Equation 1, Note that Steps S13and S14may be performed prior to Steps S11and S12.

Next, determiner112calculates depth slope α0based on first depth D1and second depth D2(Step S15). For example, determiner112calculates depth slope α0based on the difference between first depth D1and second depth D2.

Subsequently, determiner112obtains reference depth Dref(α0) that matches depth slope α0(Step S16). Specifically, determiner112calculates reference depth Dref(α0) in a non-multipath environment based on depth slope α0described above and reference data Drefheld by data holder113.

Subsequently, determiner112determines whether the magnitude of the difference between first depth D1and reference depth Dref(α0) is greater than threshold TH (Step S17). Threshold TH is a standard for determining the margin of measurement error, and is freely set according to the allowable error required by a downstream system. For example, when the depth is in a range of from 0.0 to 1.0, threshold TH is set to 0.1 if the system allows a 10%-margin of error.

When the magnitude of the difference between first depth D1and reference depth Dref(α0) is greater than threshold TH (Yes in S17), determiner112determines that multipath is present (Step S18). On the other hand, when the magnitude of the difference between first depth D1and reference depth Dref(α0) is less than or equal to threshold TH (No in S17), determiner112determines that multipath is absent (Step S19). This way, whether or not multipath is present at the time of measuring the distance to the measurement target is determined.

When the multipath is determined to be absent, depth D and distance L can be calculated using Equations 12 to 14.

When the multipath is determined to be present, four unknown parameters shown below are calculated to correct depth D.

Of the four unknown parameters: the first one is light round-trip time t1that irradiation light takes to travel to the measurement target and travel back from the measurement target as direct reflected light; the second one is received-light intensity an of the direct reflected light at the start of falling; the third one is multipath delay time tbof indirect reflected light with respect to light round-trip time t1of direct light; and the fourth one is received-light intensity bt1of indirect reflected light at the start of falling. Here, multipath delay time tbis delay time in the case where the indirect reflected light travels back later than the direct reflected light, and thus tb>0. In contrast, multipath advance time tbillustrated inFIG. 12Ais advance time in the case where the indirect reflected light travels back earlier than the direct reflected light, and thus tb<0.

Here, four measurement values are used to calculate the four unknown parameters. The first value is first depth D1, the second value is second depth D2, and the third value is depth slope a calculated based on first depth D1and second depth D2. The fourth value is luminance information that is output from signal processor104, The luminance information may be first luminance B1calculated from the first period, second luminance B2calculated from the second period, or an average of first luminance B1and second luminance B2.

Subsequently, four equations shown in Equations 24 to 27 are generated, and unknown parameters t1, at1, bt1, and tbwhich make the solution of each equation zero are calculated. To solve the four equations, general non-linear estimation may be used, or other high-speed estimation methods may be used.

Once unknown parameters t1, at1, bt1, and tbare calculated, depth D in the case of a non-multipath environment can be calculated.

In such a manner, signal processor104calculates: first luminance B1that is determined based on amount of light received R1which is an amount of light received by light receiver103through light exposure during first timing Tm1; and second luminance B2that is determined based on amount of light received R3which is an amount of light received by light receiver103through light exposure during third timing Tm3. When multipath is determined to be present, determiner112corrects first depth D1or second depth D2using: at least one of first luminance B1or second luminance B2; the difference between first depth D1and second depth D2; and reference data Dref, This makes it possible to calculate distance L in a multipath environment.

1-3. Variation of Working Example 1

Next, multipath detection device100according to a variation of Working Example 1 will be described with reference toFIG. 17.

FIG. 17is a timing diagram illustrating operations of multipath detection device100according to the variation of Working Example 1.FIG. 17illustrates light emission control signal8A, irradiation light8B, incident light8C, and light exposure control signal8D.

The settings for the S0exposure, S1exposure, and BG exposure in the first period are the same as the settings for the exposures inFIG. 6.

The S0exposure, S1exposure, and BG exposure in the second period are the same as those in the first period in terms of the timing of light exposure control signal8D, and are different from those in the first period in that light emission control signal8A in the second period is earlier than light emission control signal8A in the first period by Δt. As a result, the timings of irradiation light8B and incident light8C in the second period are also earlier than those in the first period by ΔT.

In the variation, too, first depth D1is calculated based on the ratio between amount of light received R1and amount of light received R2illustrated inFIG. 17, and second depth D2is calculated based on the ratio between amount of light received R3and amount of light received R4. Then, the presence or absence of multipath can be determined based on reference data Drefand the difference between first depth D1and second depth D2.

Multipath detection device100according to the present embodiment includes: signal controller101that outputs light emission control signal7A and light exposure control signal7D; light emitter102that emits light in accordance with light emission control signal7A; light receiver103that receives light through light exposure in accordance with light exposure control signal7D; data holder113that holds reference data Drefon a depth determined based on a ratio between: an amount of light received by light receiver103through light exposure in a non-multipath environment during a predetermined timing in accordance with a predetermined light emission control signal output from signal controller101; and an amount of light received by light receiver103through light exposure in the non-multipath environment during a timing different from the predetermined timing in accordance with a light emission control signal output from signal controller101in a time slot different from a time slot in which the predetermined light emission control signal is output; signal processor104that calculates first depth D1and second depth D2, first depth D1being determined based on a ratio between (i) amount of light received R1that is an amount of light received by light receiver103through light exposure during first timing Trail in accordance with first light emission control signal Es1output from signal controller101and (ii) amount of light received R2that is an amount of light received by light receiver103through light exposure during second timing Tm2different from first timing Tm1in accordance with second light emission control signal Es1output from signal controller101in a time slot different from a time slot in which first light emission control signal Es1is output, second depth D2being determined based on a ratio between (iii) amount of light received R3that is an amount of light received by light receiver103through light exposure during third timing Tm3in accordance with third light emission control signal Es3output from signal controller101and (iv) amount of light received R4that is an amount of light received by light receiver103through light exposure during fourth timing Tm4different from third timing Tm3in accordance with fourth light emission control signal Es4output from signal controller101in a time slot different from a time slot in which third light emission control signal Es3is output; and determiner112that determines the presence or absence of multipath using reference data Drefand a difference between first depth D1and second depth D2.

As described above, since signal processor104calculates first depth D1based on the ratio between amount of light received R1and amount of light received R2, and second depth D2based on the ratio between amount of light received R3and amount of light received R4, and determiner112determines the presence or absence of multipath based on reference data Drefand the difference between first depth D1and second depth D2, it is possible to reduce the processing load for the determination of the presence or absence of multipath.

Also, a waveform of irradiation light emitted by light emitter102may be a distorted pulse waveform that monotonically increases and then monotonically decreases.

This makes the relationship between depth and depth slope one-to-one, and the processing load for the multipath detection can be reduced.

Also, a start time of fourth timing Tm4which is based on fourth light emission control signal Es4may be different from a start time of second timing Tm2which is based on second light emission control signal Es1.

With this, a TOF multipath detection device that adjusts the settings of the emission puke and the exposure pulse becomes capable of detecting multipath through extension of the standard functions, thus enabling cost reduction of multipath detection device100.

Also, a start time of third timing Tm3which is based on third light emission control signal Es3may be different from a start time of first timing Trail which is based on first light emission control signal Es1.

With this, a TOF multipath detection device that adjusts the settings of the emission puke and the exposure pulse becomes capable of detecting multipath through extension of the standard functions, thus enabling cost reduction of multipath detection device100.

Also, a difference between the start time of fourth timing Tm4which is based on fourth light emission control signal Es4and the start time of second timing Tm2which is based on second light emission control signal Es1may be identical to a difference between the start time of third timing Tm3which is based on third light emission control signal Es3and the start time of first timing Tm1which is based on first light emission control signal Es1.

This makes it possible to accurately determine depths each calculated based on a ratio between two amounts of light received. Accordingly, the presence or absence of multipath can be accurately determined based on reference data Drefand the difference between two depths.

Also, signal controller101may output first light emission control signal Es1and third light emission control signal Es3in different time slots.

This makes it possible to easily calculate first depth D1and second depth D2, thus enabling reduction in the processing load for the determination of the presence or absence of multipath.

Also, data holder113may hold, as reference data Dref, a relationship between the depth and a depth slope, and determiner112may calculate reference depth Dref(α0) in the non-multipath environment based on: depth slope α0calculated based on a difference between first depth D1and second depth D2; and reference data Drefheld by data holder113, and determine the presence or absence of the multipath based on a magnitude of a difference between reference depth Dref(α0) and one of first depth D1and second depth D2.

In such a manner, by calculating reference depth Dref(α0) in a non-multipath environment based on depth slope α0and reference data Dref, and determining the presence or absence of multipath based on the magnitude of the difference between, for example, first depth D1and reference depth Dref(α0), it is possible to reduce the processing load for the determination of the presence or absence of multipath.

Also, determiner112may calculate reference depth Dref(α0) in the non-multipath environment based on: a difference between first depth D1and second depth D2; and reference data Drefheld by data holder113, and determine the presence or absence of the multipath based on a magnitude of a difference between reference depth Dref(α0) and one of first depth D1and second depth D2.

In such a manner, by calculating reference depth Dref(α0) in a non-multipath environment based on reference data Drefand the difference between two depths, and determining the presence or absence of multipath based on the magnitude of the difference between, for example, first depth D1and reference depth Dref(α0), it is possible to reduce the processing load for the determination of the presence or absence of multipath.

Also, signal processor104may calculate first luminance B1based on amount of light received R1that is the amount of light received by light receiver103through light exposure during first timing Tm1, and second luminance B2based on amount of light received R3that is the amount of light received by light receiver103through light exposure during third timing Tm3, and when the multipath is determined to be present, determiner112may correct first depth D1using: at least one of first luminance B1or second luminance B2; the difference between first depth D1and second depth D2; and reference data Dref.

This makes it possible to calculate the depth with no measurement error caused by multipath, and measure the correct distance based on the calculated depth.

The multipath detection method according to the present embodiment includes: storing reference data Drefon a depth determined based on a ratio between: an amount of light received through light exposure in a non-multipath environment during a predetermined timing in accordance with a predetermined light emission control signal; and an amount of light received through light exposure in the non-multipath environment during a timing different from the predetermined timing in accordance with a light emission control signal output in a time slot different from a time slot in which the predetermined light emission control signal is output; calculating first depth D1based on a ratio between (i) amount of light received R1that is an amount of light received through light exposure during first timing Tm1in accordance with first light emission control signal Es1and (ii) amount of light received R2that is an amount of light received through light exposure during second timing Tm2different from first timing Tm1in accordance with second light emission control signal Es1output in a time slot different from a time slot in which first light emission control signal Es1is output; calculating second depth D2based on a ratio between (iii) amount of light received R3that is an amount of light received through light exposure during third timing Tm3in accordance with third light emission control signal Es3and (iv) amount of light received R4that is an amount of light received through light exposure during fourth timing Tm4different from third timing Tm3in accordance with fourth light emission control signal Es4output in a time slot different from a time slot in which third light emission control signal Es3is output; and determining the presence or absence of multipath using reference data Drefand a difference between first depth D1and second depth D2.

As described above, by calculating first depth D1based on the ratio between amount of light received R1and amount of light received R2, and second depth D2based on the ratio between amount of light received R3and amount of light received R4, and determining the presence or absence of multipath based on reference data Drefand the difference between first depth D1and second depth D2, it is possible to reduce the processing load for the determination of the presence or absence of multipath.

Working Example 2

Next, multipath detection device100according to Working Example 2 will be described with reference toFIG. 18toFIG. 20. While Working Example 1 has described the example in which the light exposures in the first period and the light exposures in the second period are performed in time sequence, Working Example 2 will describe an example in which the S0exposures, S1exposures, and BG exposures in the first and second periods are associated with two pixels and performed collectively.

FIG. 18is a block diagram illustrating an exemplary configuration of multipath detection device100according to Working Example 2.

Signal controller101simultaneously generates and outputs timing signals corresponding to the first period and the second period inFIG. 17. For example, signal controller101outputs light exposure control signal9D to first pixel103a1of light receiver103, and outputs light exposure control signal9F to second pixel103a2of light receiver103.

Light receiver103includes first pixel103a1and second pixel103a2. First pixel103a1activates upon reception of light exposure control signal9D, and second pixel103a2activates upon reception of light exposure control signal9F.

Signal processor104receives, from pixel103a1, light reception signal1regarding an amount of light received, and calculates first depth D1and first luminance B1. Signal processor104also receives, from pixel103a2, light reception signal2regarding an amount of light received, and calculates second depth D2and second luminance B2. Signal processor104then outputs information on first depth D1, second depth D2, first luminance B1, and second luminance B2to determiner112,

FIG. 19is a timing diagram illustrating operations of multipath detection device100according to Working Example 2.FIG. 19illustrates light emission control signal9A, irradiation light9B, incident light9C, light exposure control signal9D, incident light9E, and light exposure control signal9F. The settings for the timings indicated by light emission control signal9A, irradiation light9B, incident light9C, and light exposure control signal9D inFIG. 19are the same as the settings for the timings inFIG. 6.

As illustrated inFIG. 19, first light emission control signal Es1and third light emission control signal Es3are simultaneously output, and second light emission control signal Es2and fourth light emission control signal Es4are simultaneously output.

Light exposure control signal9D is output during first timing Tm1which is based on first light emission control signal Es1, and light exposure control signal9D is output during second timing Tm2which is based on second light emission control signal Es2. Light exposure control signal9F is output during third timing Tm3which is based on third light emission control signal Es3, and light exposure control signal9F is output during fourth timing Tm4which is based on fourth light emission control signal Es4. The start time of third timing Tm3is later than that of first timing Tm1by time Δt, and the start time of fourth timing Tm4is later than that of second timing Tm2by time Δt. Furthermore, second timing Tm2starts later than first timing Tm1by time Trbased on light emission control signal9A, and fourth timing Tm4starts later than third timing Tm3by time Trbased on light emission control signal9A.

FIG. 20is a flowchart illustrating a multipath detection method according to Working Example 2.

First, as preparation for multipath detection, multipath detection device100stores, in data holder113, reference data Drefin a non-multipath environment (Step S20).

Subsequently, multipath detection device100performs the S0exposures, S1exposures, and BG exposures corresponding to the first period and the second period (Step S21). Specifically, light receiver103obtains amount of light received R1through light exposure during first timing Tm1, obtains amount of light received R2through light exposure during second timing Tm2, obtains amount of light received R3through light exposure during third timing Tm3, and obtains amount of light received R4through light exposure during fourth timing Tm4. Amounts of light received R1to R4are output to signal processor104.

Next, signal processor104calculates first depth D1based on the ratio between amount of light received R1and amount of light received R2, and calculates second depth D2based on the ratio between amount of light received R3and amount of light received R4(Step S22).

Next, determiner112calculates depth slope α0based on first depth D1and second depth D2(Step S25). For example, determiner112calculates depth slope α0based on the difference between first depth D1and second depth D2.

Subsequently, determiner112obtains reference depth Dref(α0) that matches depth slope α0(Step S26), Specifically, determiner112calculates reference depth Dref(α0) in a non-multipath environment based on depth slope α0described above and reference data Drefheld by data holder113.

Subsequently, determiner112determines whether the magnitude of the difference between first depth D1and reference depth Dref(α0) is greater than threshold TH (Step S27).

When the magnitude of the difference between first depth D1and reference depth Dref(α0) is greater than threshold TH (Yes in S27), determiner112determines that multipath is present (Step S28). On the other hand, when the magnitude of the difference between first depth D1and reference depth Dref(α0) is less than or equal to threshold TH (No in S27), determiner112determines that multipath is absent (Step S29). This way, whether or not multipath is present at the time of measuring the distance to the measurement target is determined.

When the multipath is determined to be absent, depth D and distance L can be calculated using Equations 12 to 14.

When the multipath is determined to be present, correction can be carried out based on the above-described depths, luminance, and slope in the same manner as in Working Example 1. In addition, it is possible to obtain a necessary light reception signal with less frames than in Working Example 1, and perform multipath detection and correction at high speed.

In such a manner as described above, multipath detection device100according to Working Example 2 calculates first depth D1based on the ratio between amount of light received R1and amount of light received R2illustrated inFIG. 19, and calculates second depth D2based on the ratio between amount of light received R3and amount of light received R4. Since the presence or absence of multipath is determined based on reference data Drefand the difference between first depth D1and second depth D2, it is possible to reduce the processing load for the determination of the presence or absence of multipath.

Signal controller101: simultaneously outputs first light emission control signal Es1and third light emission control signal Es3; simultaneously outputs second light emission control signal Es1and fourth light emission control signal Es4; outputs, to first pixel103a1of light receiver103, light exposure control signal9D for performing light exposure in response to first light emission control signal Es1and light exposure in response to second light emission control signal Es1; and outputs, to second pixel103a2of light receiver103, light exposure control signal9F for performing light exposure in response to third light emission control signal Es3and light exposure in response to fourth light emission control signal Es4.

This makes it possible to simultaneously perform the processing in the first period and the processing in the second period using two pixels, thus simplifying the real-time processing for multipath detection.

Other Embodiments

Although an embodiment has been described above, the present disclosure is not limited to the above embodiment. The present disclosure also encompasses embodiments achieved by making various modifications to the above embodiment that are conceivable to a person of skill in the art, as well as embodiments realized by arbitrarily combining constituent elements and functions of the above embodiment within the scope of the essence of the present disclosure.

Although only an exemplary embodiment of the present disclosure has been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiment without materially departing from the novel teachings and advantages of the present disclosure. Accordingly, all such modifications are intended to be included within the scope of the present disclosure.

INDUSTRIAL APPLICABILITY

The multipath detection device and multipath detection method according to the present disclosure are widely applicable to a TOF camera system, for example.