Patent Description:
An object detection device may measure a distance to an object by measuring a time of flight (ToF) of light to the object. More specifically, the object detection device may calculate the distance to the object by measuring a time until an optical signal emitted from a light source is reflected by the object and then returns to the object detection device, and generate a depth image of the object based on the calculated distance.

Such an object detection device may convert the optical signal into a digital signal to calculate the ToF of the light, but the ToF of the light may vary depending on a sampling rate of an analog-to-digital converter (ADC).

European Patent <CIT> discloses a device comprising an optoelectronic sensor with a light emitter which sends single light pulses in which light received signals are supplied to the delay path, with a varying temporal misalignment, in which the device compensates for temporal misalignment over the single light pulses to adjust fluctuations in the delay paths.

United States patent application <CIT> discloses an apparatus of a laser range detector with a delay circuit for generating a plurality of delay signals.

German Patent application <CIT> discloses a method of distance measurement having a delay circuit for generating a plurality of delay signals, which are supplied in parallel for further processing.

United States patent application <CIT> discloses a LIDAR device to calculate distances to objects by classifying a signal received from a sensor as being a particular type of signal and selecting, based on the type of signal, a detector for processing the received signal from among multiple detectors.

German Patent application <CIT> discloses a LiDAR system for the optical detection of a field of view, having a transmission unit for controlling the generation and transmission of light pulses as primary light into the field of view, and having a receiving unit for controlling the reception and evaluation of secondary light received from the field of view.

One or more example embodiments provide an object detection device capable of improving the accuracy of a distance to an object and an operating method of the object detection device.

According to the present invention there is provided an object detection device according to claim <NUM>.

When the time-delayed sub received signals are time-delayed by a same amount of time, the time-delayed sub received signals may be time-delayed with respect to the first sub received signal by <NUM>/<NUM> of a sampling period of the converter.

When the time-delayed sub received signals are time-delayed by (n-<NUM>) different times in which n is a natural number greater than <NUM>, a time difference between neighboring time-delayed sub received signals among the time-delayed sub-received signals that are time-delayed by (n-<NUM>) different times may be equal to <NUM>/n of a sampling period of the converter.

A sampling period of the converter may also be less than <NUM>/<NUM> of a pulse width of the received signal.

The splitter may be further configured to split the received signal into a plurality of sub received signals having a same intensity.

The number of the plurality of sub received signals may be equal to a number of the plurality of signal lines.

The processor may be further configured to determine a plurality of estimated distances to the object based on the plurality of maximum sampling points and determine an average of the plurality of estimated distances as the distance to the object.

The processor may be further configured to determine an average sampling point from the plurality of maximum sampling points and determine the distance to the object based on the average sampling point.

The processor may be further configured to, when determining a maximum sampling point from at least one cross-correlation signal among the plurality of cross-correlation signals, select a plurality of sampling points from the at least one cross-correlation signal and apply a quadratic function to the plurality of sampling points to determine the maximum sampling point.

The number of the plurality of sampling points may be greater than or equal to <NUM>.

The plurality of sampling points may include a first sampling point having a maximum absolute value in the at least one cross-correlation signal, a second sampling point at a time before m sampling periods from a time corresponding to the first sampling point, and a third sampling point at a time after the m sampling periods from the time corresponding to the first sampling point, in which m is a natural number greater than or equal to <NUM>.

The processor may be further configured to generate a point cloud based on the distance to the object, and obtain a three-dimensional (3D) image regarding the object based on the generated point cloud.

According to an aspect of the present invention there is provided a method according to claim <NUM>.

A time delay may be less than a pulse width of the received signal.

Although terms used in the present disclosure are selected with general terms popularly used at present under the consideration of functions in the present disclosure, the terms may vary according to the intention of those of ordinary skill in the art, judicial precedents, or introduction of new technology. In addition, in a specific case, the applicant voluntarily may select terms, and in this case, the meaning of the terms is disclosed in a corresponding description part of the present disclosure. Thus, the terms used in the present disclosure should be defined not by the simple names of the terms but by the meaning of the terms and the contents throughout the present disclosure.

In the specification, when a region is "connected" to another region, the regions may not only be "directly connected", but may also be "electrically connected" via another device therebetween. As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise. When it is assumed that a certain part includes a certain component, the term "including" means that a corresponding component may further include other components unless a specific meaning opposed to the corresponding component is written.

The term such as "comprise" or "include" used in the embodiments should not be interpreted as including all of elements or operations described herein, and should be interpreted as excluding some of the elements or operations or as further including additional elements or operations.

Terms such as first, second, and the like may be used to describe various elements, but may be used only for the purpose of distinguishing one element from another element. These terms are not intended to limit that substances or structures of elements are different.

The use of "the" and other demonstratives similar thereto may correspond to both a singular form and a plural form.

<FIG> is a diagram for describing an exemplary operation of an object detection device <NUM>, according to an example embodiment of the disclosure.

An object detection device <NUM> may be used as a sensor (e.g., a light detection and ranging (LiDAR) sensor, a radio detection and ranging (RADAR) sensor, a three-dimensional (3D) camera, a stereo camera, a depth sensor, etc.) for obtaining 3D information, such as distance information regarding an object <NUM>, etc., in real time. The object detection device <NUM> may be applied to an electronic device such as an unmanned vehicle, an autonomous vehicle, a robot, a drone, a portable terminal, etc..

Referring to <FIG>, the object detection device <NUM> may include a transmitter <NUM> that emits light L toward an object <NUM> according to a (electrical) transmission signal, a receiver <NUM> that detects the light L and outputs a (electrical) received signal based on the detected light L, and a processor <NUM> that controls the transmitter <NUM> and the receiver <NUM> to measure a distance to the object <NUM>, based on a transmission time of the emitted light L and a reception time of the detected light L.

The transmitter <NUM> may output the light L to be used for analysis of a position, a shape, etc., of the object <NUM>. However, the disclosure is not limited thereto. The transmitter <NUM> may output a radio frequency (RF) signal, a laser signal, or a light emitting diode (LED) signal, but a type and a frequency range of a signal output from the transmitter <NUM> are not limited thereto. Although distance measurement using light will be described below, it is needless to say that an RF signal may be equally applied.

For example, the transmitter <NUM> may output light of an infrared band wavelength. When the light in an infrared band is used, mixing with natural light in a visible light zone including sunlight may be prevented. However, it is not necessarily limited to the infrared band and light of various wavelength bands may be emitted.

The transmitter <NUM> may include at least one light source. For example, the transmitter <NUM> may include a light source such as a laser diode (LD), an edge emitting laser, a vertical-cavity surface emitting laser (VCSEL), a distributed feedback laser, a light emitting diode (LED), a super luminescent diode (SLD), etc..

The transmitter <NUM> may generate and output light in a plurality of different wavelength bands. The transmitter <NUM> may generate and output pulse light or continuous light.

According to an example embodiment of the present disclosure, the transmitter <NUM> may further include a beam steering element for changing the radiation angle of light. For example, the beam steering element may be a scanning mirror or an optical phased array.

According to an example embodiment of the present disclosure, the transmitter <NUM> may emit light whose frequency or phase is modulated over time. For example, the transmitter <NUM> may emit light using a frequency modulated continuous-wave (FMCW) method or a phase modulation continuous wave (PMCW) method.

The receiver <NUM> may include at least one detector <NUM>, and the detector <NUM> identifies and detects the light L reflected from the object <NUM>. According to an example embodiment of the present disclosure, the receiver <NUM> may further include an optical element for collecting the received signal to the detector <NUM>.

The transmitter <NUM> and the receiver <NUM> may be implemented as separate devices or may be implemented as a single device (e.g., a transceiver). For example, when the object detection device <NUM> is a radar device, a radar sensor may emit a radar signal to the outside and receive a radar signal reflected from the object <NUM>. The radar sensor may be both the transmitter <NUM> and the receiver <NUM>.

The processor <NUM> may control the transmitter <NUM> and the receiver <NUM> to control an overall operation of the object detection device <NUM>. For example, the processor <NUM> may perform power supply control, on/off control, pulse wave (PW) or continuous wave (CW) generation control, etc., with respect to the transmitter <NUM>.

The processor <NUM> may perform signal processing for obtaining information about the object <NUM>, by using the received signal output from the receiver <NUM>. The processor <NUM> may determine a distance to the object <NUM> based on a time of flight (ToF) of light output by the transmitter <NUM>, and perform data processing for analyzing the position and shape of the object <NUM>. For example, the processor <NUM> may generate a point cloud based on distance information about the object <NUM> and obtain a 3D image of the object <NUM> based on the point cloud.

The 3D image obtained by the processor <NUM> may be transmitted to another unit and utilized. For example, such information may be transmitted to the processor <NUM> of an autonomous driving device, such as an unmanned vehicle, a drone, etc., in which the object detection device <NUM> is employed. In addition, such information may be utilized by smartphones, cell phones, personal digital assistants (PDAs), laptops, personal computers (PCs), wearable devices, and other mobile or non-mobile computing devices.

Meanwhile, the object detection device <NUM> of the present disclosure may further include other general-purpose components in addition to the components of <FIG>.

For example, the object detection device <NUM> may further include a memory that stores various data. The memory may store data processed or to be processed in the object detection device <NUM>. Also, the memory may store applications, drivers, etc., to be driven by the object detection device <NUM>.

The memory may include random access memory (RAM) such as dynamic random-access memory (DRAM), static random-access memory (SRAM), etc., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk (CD)-ROM, Blu-ray or other optical disk storages, hard disk drive (HDD), solid state drive (SSD), or flash memory, and may also include other external storage devices that are accessible by the object detection device <NUM>.

The object detection device <NUM> may be implemented with a housing or may be implemented as a plurality of housings. When the object detection device <NUM> is implemented with a plurality of housings, a plurality of components may be connected wiredly or wirelessly. For example, the object detection device <NUM> may be divided into a first device including the transmitter <NUM> and the receiver <NUM> and a second device including the processor <NUM>. The object detection device <NUM> may also be implemented as a part of a device that performs other functions, for example, an autonomous driving device.

<FIG> is a graph showing a relationship between an actual distance and a distance measurement result in an analog-to-digital converter (ADC)-based object detection device. An X-axis represents an actual distance to an object, and a Y-axis represents a result of measuring the distance to the object using an ADC.

Since the ADC quantizes a received signal at a specific sampling rate, the accuracy of the distance to the object may depend on the sampling rate of the ADC. That is, a measurement result may change in a stepwise manner according to the sampling rate of the ADC. Thus, the measured distance may be the same even though the actual distance to the object is different. Also, even when an object exists at the same distance as a specific distance, for example, about <NUM> or <NUM>, the measurement result may vary greatly depending on a digital conversion time of the received signal.

For example, when the sampling rate is <NUM> GS/s, a sampling period is <NUM> ns, such that within a distance of <NUM> that light travels within <NUM> sampling period, the same measurement value is obtained, and thus a distance error may be repeated in a stepwise manner up to ±<NUM>. Therefore, the easiest way to increase distance accuracy is to increase the sampling rate of the ADC. However, this may increase the cost of the ADC.

The object detection device <NUM> according to an example embodiment of the present disclosure may increase the accuracy of the distance to the object even in a case in which a converter <NUM> is implemented to have a low sampling rate. <FIG> is a block diagram illustrating in more detail an object detection device with increased distance accuracy, according to an example embodiment of the present disclosure, and <FIG> is a flowchart illustrating a method of measuring a distance to an object, according to an example embodiment of the present disclosure.

Referring to <FIG> and <FIG>, the transmitter <NUM> may emit light toward the object <NUM> based on a transmission signal, in operation S410. More specifically, the transmitter <NUM> may generate the transmission signal under the control of the processor <NUM> and emit light according to the transmission signal. The transmission signal and the light may be of a pulse type. A part of the transmission signal of the transmitter <NUM> may be output to the processor <NUM>. The part of the transmission signal may be used to calculate the ToF of light. The transmitter <NUM> of <FIG> may correspond to the transmitter <NUM> of <FIG>.

The receiver <NUM> may include a detector <NUM> that detects light and outputs a received signal, a time-delaying unit <NUM> that time-delays a part of the received signal with respect to the rest of the received signal, and the converter <NUM> that converts the received signal in an analog domain into a digital signal. The converter <NUM> may be also referred to as an analog-to-digital converter.

The detector <NUM> of the receiver <NUM> detects the light and outputs a received signal corresponding to the light in operation S420. The light includes light reflected from the object <NUM>. The reflected light may be of the pulse type, such that the received signal corresponding to the light may also be of the pulse type. The detector <NUM> may be an optical sensor or an RF sensor.

The detector <NUM>, which is a sensor capable of sensing light, may be, for example, a light-receiving element that generates an electrical signal by light energy. For example, the detector <NUM> may include an avalanche photo diode (APD) or a single photon avalanche diode (SPAD). The type of the light-receiving element may not be particularly limited.

The time-delaying unit <NUM> time-delays a part of the received signal with respect to the rest of the received signal in operation S430. The time-delaying unit <NUM> includes a splitter (e.g., an optical splitter, a planar light wave circuit (PLC) splitter, a fiber optic splitter (FBT) splitter, etc.) <NUM> that splits the received signal into a plurality of sub received signals, a signal line <NUM> that applies a first sub received signal among the plurality of sub received signals to the converter <NUM>, and one or more time-delay lines <NUM> and <NUM> that time-delay the time-delayed sub received than the first sub received signal among the plurality of sub received signals from the first sub received signal.

The time-delay may be smaller than a pulse width of the received signal. Generally, a pulse width of a transmission signal, a pulse width of the light, and the pulse width of the received signal may be the same as one another. This is because light is emitted by the transmission signal and the received signal corresponds to the light reflected from the object out of the above-mentioned light. Noise may be generated in a process of transmitting light and a signal, but this may correspond to an error range. Hereinafter, for convenience, the pulse width may refer to the pulse width of the received signal.

Moreover, the time-delay may vary depending on the number of time-delay lines <NUM> and <NUM>. When a part of the received signal is time-delayed by one type (e.g., by the same amount of time), that is, when there is one time-delay line, a part of the received signal may be delayed by <NUM>/<NUM> of the sampling period of the converter <NUM> than the rest of the received signal. Alternatively, when a part of the received signal is delayed by n-<NUM> different times (n is a natural number greater than or equal to <NUM>), that is, when there are (n-<NUM>) time-delay lines, a time difference between neighboring time-delayed received signals among received signals delayed by (n-<NUM>) different times may be <NUM>/n of the sampling period of the converter <NUM>.

For example, the time-delaying unit <NUM> may include a splitter <NUM> that splits the received signal into first through third sub received signals, a signal line <NUM> that sends the first sub received signal output from the splitter <NUM> to the converter <NUM>, a first time-delay line <NUM> that first time-delays the second sub received signal, output from the splitter <NUM>, and applies the delayed second sub received signal to the converter <NUM>, and a second time-delay line <NUM> that second time-delays the third sub received signal, output from the splitter <NUM>, by a second time and applies the delayed third sub received signal to the converter <NUM>. Herein, the first time-delaying may mean time-delaying by <NUM>/<NUM> of a pulse width, and the second time-delaying may mean time-delaying by <NUM>/<NUM> of the pulse width.

While one signal line <NUM> and two time-delay lines <NUM> and <NUM> are shown in <FIG>, the present disclosure is not limited thereto. There may be one time-delay line or three or more time-delay lines, as the time-delay lines <NUM> and <NUM>. The object detection device <NUM> may be designed differently according to an application field thereof. For example, the object detection device <NUM> is applied to an autonomous vehicle for use in detection of a long-distance object, a time-delay line may be small. Meanwhile, when the object detection device <NUM> is used to detect a short-range object such as in face recognition, the more the time-delay lines are, the higher the accuracy of a 3D image is.

The converter <NUM> converts the received signal into a digital signal of range diversity according to a preset sampling rate, in operation S440. The converter <NUM> may include an ADC. For example, the sampling rate may be set in a range of <NUM> to <NUM>. The converter <NUM> converts a part of the time-delayed received signal into one or more time-delayed digital signals, and converts the rest of the received signal into a digital signal. Herein, the digital signal and the one or more time-delayed digital signals may be referred to as a digital signal of range diversity.

For example, as shown in <FIG>, the converter <NUM> may convert the first sub received signal input through the signal line <NUM> into a digital signal, convert the second sub received signal input through the first time-delay line <NUM> into a first time-delayed digital signal, and convert the third sub received signal input through the second time-delay line <NUM> into a second time-delayed digital signal.

The converter <NUM> may output each of the digital signal and the first and second time-delayed digital signals as vector data in the form of a column vector or a row vector. The vector data may mean an array in the form of a column vector or a row vector including a set of elements. A quantized value of the received signal may be stored in each element. The converter <NUM> may be an ADC that converts an analog-type received signal into a digital type.

Alternatively, the transmitter <NUM> and the receiver <NUM> may further include a high-pass filter that removes an offset from the transmission signal and the received signal, and an amplifier (AMP) that amplifies the magnitudes of the transmission signal and the received signal.

The transmitting unit <NUM> may further include an ADC that converts the transmission signal in an analog domain into a digital signal. In another example, the converter <NUM> may be provided outside the receiver <NUM>, and may be configured to receive analog signals that are output from both the transmitter <NUM> and the receiver <NUM>, and convert the analog signals to digital signals.

The processor <NUM> determines the distance to the object <NUM> based on the digital signal of range diversity, in operation S450.

The processor <NUM> determines the distance to the object by using a cross-correlation signal between the digital signal of range diversity, that is, a digitized received signal, and the transmitted signal.

The cross-correlation signal is a result of quantitatively calculating a similarity between the received signal and the transmission signal, such that the transmission signal may have the most similar shape as the received signal when the transmission signal has a time-delay equal to a ToF of light. Thus, when the time equal to the ToF is delayed, the cross-correlation signal may have a maximum value, and a time when the cross-correlation signal has the maximum value may be a basis for the ToF of light.

<FIG> is a flowchart illustrating a method of determining a distance to an object by using a digital signal of a range diversity, according to an example embodiment of the present disclosure.

Referring to <FIG> and <FIG>, the processor <NUM> may include a cross-correlating unit <NUM> that generates a cross-correlation signal between the digital signal of range diversity and the transmission signal, a distance determining unit <NUM> that determines a distance to the object <NUM> from the cross-correlation signal, and a point cloud generating unit <NUM> that generates a point cloud based on distance information.

The cross-correlating unit <NUM> may generate a plurality of cross-correlation signals between each of the digital signal and the one or more time-delayed digital signals and the transmission signal, in operation S510. To this end, the cross-correlating unit <NUM> may further include a correlator.

The cross-correlating unit <NUM> may receive a quantized (i.e., digitized) transmission signal xk from the transmitter <NUM> and a quantized (i.e., digitized) received signal (yi+k) from the receiver <NUM>, and generate a cross-correlation signal Rxyi between the transmission signal and the received signal, as in Equation <NUM> below.

Herein, the quantized received signal (yi+k) may be a digital signal or one or more time-delayed digital signals.

For example, the cross-correlating unit <NUM> may generate a first cross-correlation signal between a digital signal as the received signal and a digitized transmission signal, generate a second cross-correlation signal between the first time-delayed digital signal and the digitized transmission signal, and a third cross-correlation signal between the second time-delayed digital signal and the digitized transmission signal.

The distance determining unit <NUM> determines a plurality of maximum sampling points from the plurality of cross-correlation signals, in operation S520. The sampling point may be an element of the cross-correlation signal, and each sampling point may include time information and intensity information. That is, the cross-correlation signal is formed by the digitized received signal and the digitized transmission signal, such that the cross-correlation signal may be a combination of sampling points. A period of the sampling point may be the same as the sampling period of the converter <NUM>.

In performing operation S520, the distance determining unit <NUM> may determine a first maximum sampling point from sampling points included in the first cross-correlation signal, determine a second maximum sampling point from sampling points included in the second cross-correlation signal, and determine a third maximum sampling point from sampling points included in the third cross-correlation signal. The distance determining unit <NUM> may select a maximum sampling point from among sampling points included in the cross-correlation signal, and estimate the maximum sampling point by using the sampling points included in the cross-correlation signal. A method of estimating a maximum sampling point will be described later.

The distance determining unit <NUM> determines the distance to the object based on the plurality of maximum sampling points, in operation S530.

<FIG> is a flowchart illustrating a method of determining a distance via a plurality of maximum sampling points, according to an example embodiment of the present disclosure.

The distance determining unit <NUM> may determine a plurality of distances to an object based on a plurality of maximum sampling points, in operation S610. For example, the distance determining unit <NUM> may determine a first distance to the object based on the first maximum sampling point, determine a second distance to the object based on the second maximum sampling point, and determine a third distance to the object based on the third maximum sampling point. The distance determining unit <NUM> may determine the distance to the object from a ToF of light that may be calculated using a sampling rate S of the converter <NUM> and a time value imax of each maximum sampling point. For example, the distance determining unit <NUM> may determine 2imax/S as a ToF of light and determine 2cimax/S as the distance to the object <NUM> (c is the speed of light).

The distance determining unit <NUM> may determine an average of the plurality of distances as a final distance to the object, in operation S620.

In another unclaimed example embodiment of the present disclosure, when signal values indicating a received signal or a transmission signal include negative values due to noise, oscillation, etc., an amplification effect based on calculation of a cross-correlation function may be reduced. The processor <NUM> may convert each of the received signal and the transmission signal into a unipolar signal to prevent the amplification effect based on calculation of the cross-correlation function from being reduced due to noise, oscillation, etc. The unipolar signal, which is the opposite of a bipolar signal, may mean a signal having signal values of either a negative polarity or a positive polarity.

The processor <NUM> may convert the received signal into a unipolar transmission signal and a unipolar received signal by taking absolute values of at least some of the transmission signal and the received signal. Alternatively, the processor <NUM> may convert the received signal and the transmission signal into a unipolar signal using a method other than the method of taking the absolute value. For example, the processor <NUM> may convert the received signal and the transmission signal into a unipolar signal by using a scheme to replace signal values less than a specific value (greater than or equal to <NUM>) out of signal values indicating the received signal or the transmission signal with the specific value, and convert the received signal and the transmission signal into a unipolar signal by using a method of squaring the signal values indicating the received signal or the transmission signal.

The processor <NUM> may generate a plurality of cross-correlation signals between a unipolar received signal, i.e., a unipolar digital signal of range diversity and a unipolar digitized transmission signal, determine a sampling point having a maximum magnitude from each of the plurality of cross-correlation signals, determine a plurality of distances to an object by using each sampling point, and determine a final distance by averaging the plurality of distances.

As described above, when the distance to the object is determined using the digital signal of range diversity, a distance error to the object may be reduced.

<FIG> is a reference diagram showing a result of measuring a distance by using a received signal having no range diversity, as a comparative example. (i) of <FIG> shows a transmission signal output from the transmitter <NUM>, and (ii) of <FIG> shows a received signal output from the detector <NUM>. The transmission signal and the received signal may be of a pulse type. The transmitter <NUM> may output light according to the transmission signal, and a part of the light may be reflected by the object and detected by the detector <NUM>. The detector <NUM> may output a received signal corresponding to the detected light. Generally, the received signal corresponds to a part of light, such that the magnitude of the received signal may be smaller than that of the transmission signal. To facilitate distance measurement, the receiver <NUM> may further include an amplifier for amplifying the received signal.

The received signal may be converted into a digital signal by the converter <NUM>, and the cross-correlating unit <NUM> may generate a cross-correlation signal between the transmission signal and the digital signal. (iii) of <FIG> shows a cross-correlation signal between the transmission signal and the digital signal. (iv) of <FIG> shows the distance to the object, measured from the cross-correlation signal. In (iv) of <FIG>, the x axis indicates an actual distance of the object and the y axis indicates the distance measured by the object detection device <NUM>. The measured distance may be proportional to the actual distance in a stepwise fashion because the received signal and the transmission signal are quantized, i.e., digitized.

<FIG> is a reference diagram showing a method of measuring a distance by using a received signal of a range diversity, according to an example embodiment of the present disclosure. (i) of <FIG> shows a transmission signal output from the transmitter <NUM>, and (ii) of <FIG> shows a plurality of sub received signals output from the splitter <NUM>. For example, (ii)(a) of <FIG> shows a first sub received signal applied to the converter <NUM> through the signal line <NUM> out of the received signal, (ii)(b) of <FIG> shows a second sub received signal applied to the converter <NUM> after being delayed by a first time through the first time-delay line <NUM> out of the received signal, and (ii)(c) of <FIG> shows a third sub received signal applied to the converter <NUM> after being delayed by a second time through the second time-delay line <NUM> out of the received signal.

(iii) of <FIG> shows cross-correlation signals among the transmission signal and the plurality of sub received signals. For example, (iii)(a) of <FIG> shows a first cross-correlation signal based on the first sub received signal, (iii)(b) of <FIG> shows a second cross-correlation signal based on the second sub received signal, and (iii)(c) of <FIG> shows a third cross-correlation signal based on the third sub received signal.

(iv) of <FIG> shows distance information regarding the object, obtained from each cross-correlation signal. In (iv) of <FIG>, the x axis indicates the actual distance of the object and the y axis indicates the distance measured by the object detection device <NUM>. For example, (iv)(a) of <FIG> shows a distance based on the first cross-correlation signal, (iv)(b) of <FIG> shows a distance based on the second cross-correlation signal, and (iv)(c) of <FIG> shows a distance based on the third cross-correlation signal. A distance measured from each of the first through third cross-correlation signals may be changed in a stepwise fashion similarly with a comparative example.

(v) of <FIG> shows a distance to an object based on the first through third cross-correlation signals. The distance determining unit <NUM> may determine a plurality of distances to the first through third cross-correlation signals, respectively, and determine final distance information by averaging a plurality of pieces of distance information. A final distance may be expected to have a significantly reduced error range when compared to the distance information based on each of the first to third cross-correlation signals.

While it is described in <FIG> that a plurality of distances are determined based on a plurality of maximum sampling points and an average of the plurality of distances is determined as a final distance, the present disclosure is not limited thereto.

<FIG> is a flowchart illustrating a method of determining a distance to an object, according to another example embodiment of the present disclosure.

The distance determining unit <NUM> may determine an average sampling point from a plurality of maximum sampling points, in operation S910. For example, the distance determining unit <NUM> may determine the first maximum sampling point from the first cross-correlation signal, determine the second maximum sampling point from the second cross-correlation signal, and determine the third maximum sampling point from the third cross-correlation signal. The distance determining unit <NUM> may determine an average of the first through third maximum sampling points as the average sampling point.

The distance determining unit <NUM> may determine the distance to the object based on the average sampling point, in operation S920. Determination of the distance to the object may be made based on the ToF of the light as described above. That is, the distance to the object may be determined using the speed of light, the sampling rate, and the time of the sampling point.

The processor <NUM> may further include the point cloud generating unit <NUM> that generates a point cloud based on distance about the object. The point cloud generating unit <NUM> may obtain a 3D image of the object <NUM> based on the point cloud.

<FIG> shows a result of simulating a 3D image based on a digital signal of a range diversity, according to an example embodiment of the present disclosure. (a) of <FIG> shows a 3D image of a human face in which a facial shape corresponding to the 3D image of the human face is arranged at a distance of <NUM> from the receiver <NUM> and then simulated.

(b) of <FIG> shows a result of simulating a digital signal without range diversity. There is no large depth difference in the contour of the face, such that a result measured with a digital signal without range diversity is almost like a two-dimensional (2D) image. (b) of <FIG> shows a result of simulation with a digital signal with <NUM> range diversities. Here, the <NUM>-range diversity digital signal may mean a result of splitting the received signal into <NUM> sub received signals, digitally converting <NUM> sub received signal without a time-delay, and digitally converting <NUM> sub received signals after respectively delaying the <NUM> sub received signals by <NUM>/<NUM>, <NUM>/<NUM>, and <NUM>/<NUM> times of the sampling period.

(c) of <FIG> shows a result of simulation with a digital signal with <NUM> range diversities. The <NUM>-range diversity digital signal may mean a result of splitting the received signal into <NUM> sub received signals, digitally converting <NUM> sub received signal without a time-delay, and digitally converting <NUM> sub received signals after respectively delaying the <NUM> sub received signals such that a time difference between neighboring sub received signals is <NUM>/<NUM> of the sampling period of the converter <NUM>.

As shown of <FIG>, it may be seen that the higher the range diversity of the digital signal is, the clearer the 3D image is. It may be seen that by classifying the received signal into the plurality of sub received signals with different time-delays and digitally converting the same before converting the received signal into the digital signal, a distance resolution, i.e., the accuracy of the distance may be increased.

When the distance determining unit <NUM> determines a maximum sampling point from each of the plurality of cross-correlation signals, a sampling point with the maximum magnitude among sampling points in the cross-correlation signal may be determined as a maximum sampling point, and the maximum sampling point may be estimated using the sampling point in the cross-correlation signal.

The cross-correlation signal is generated using the digitized transmission signal and the digital signal of range diversity, and thus may also be the digitized signal. Thus, the sampling points included in the digitized cross-correlation signal may not correspond to the actual distance of the object. Thus, the distance determining unit <NUM> may estimate the maximum sampling point approximate to the actual distance of the object. Maximum sampling may be estimated from one cross-correlation signal. In an example embodiment of the present disclosure, the plurality of cross-correlation signals are generated, such that the distance determining unit <NUM> may apply maximum sampling estimation in at least one cross-correlation signal among the plurality of cross-correlation signals.

<FIG> is a flowchart illustrating a method, performed by the distance determining unit <NUM>, of estimating a maximum sampling point, according to an example embodiment of the present disclosure, and <FIG> is a reference diagram for describing a method of estimating a maximum sampling point in a cross-correlation signal.

The distance determining unit <NUM> may select a plurality of sampling points from the cross-correlation signal, in operation S1110. Each of the sampling points may include time and intensity information.

For example, the distance determining unit <NUM> may select three sampling points. The distance determining unit <NUM> may select three sampling points having different sampling point times in which the intensities of the three sampling points may be equal. Generally, the cross-correlation signal is in the form of a 2D function as shown in <FIG>, such that the distance determining unit <NUM> may select three sampling points. However, the present disclosure is not limited thereto. According to a type of the cross-correlation signal, four or more sampling points may be selected.

The distance determining unit <NUM> may, among the sampling points of the cross-correlation signal, select a sampling point having the maximum absolute value as a first sampling point S<NUM>, select a sampling point at a time (t<NUM>-m) before m sampling periods (m is a natural number greater than or equal to <NUM>) from a time t<NUM> corresponding to the first sampling point S<NUM>, as a second sampling point S<NUM>, and select a sampling point at a time (t<NUM>+m) after the m sampling periods (m is a natural number greater than or equal to <NUM>) from the time t<NUM> corresponding to the first sampling point S<NUM>, as a third sampling point S<NUM>. Here, the sampling point having the maximum intensity among the sampling points is selected as the first sampling point S<NUM>, but the present disclosure is not limited thereto. The distance determining unit <NUM> may select three sampling points with different times.

The distance determining unit <NUM> may estimate a maximum sampling point SM based on the plurality of sampling points. For example, the distance determining unit <NUM> may apply a quadric function as shown in Equation <NUM> below to the first to third sampling points S<NUM>, S<NUM>, and S<NUM>.

Here, y indicates an intensity, u indicates a constant, t indicates a time, T indicates a time of the maximum sampling point SM, and P indicates the intensity of the maximum sampling point SM.

The distance determining unit <NUM> may estimate the maximum sampling point SM having a time T and an intensity P as in Equation <NUM> below.

Here, to indicate a time of the first sampling point S<NUM>, M indicates an intensity of the first sampling point S<NUM>, A indicates an intensity of the second sampling point S<NUM>, and B indicates an intensity of the third sampling point S<NUM>.

The distance determining unit <NUM> may obtain the distance to the object based on the estimated maximum sampling point SM. For example, the distance determining unit <NUM> may calculate the ToF of light by using the sampling rate S of the converter <NUM> and a time imax corresponding to the maximum sampling point. For example, the distance determining unit <NUM> may determine 2imax/S as a ToF of light and determine 2cimax/S as the distance to the object <NUM> (c is the speed of light).

<FIG> is a diagram showing a range diversity and an error when an object is detected by estimating a maximum sampling point, according to an example embodiment of the present disclosure. The pulse width of the received signal may be <NUM> ns, and the sampling rate of the received signal may be <NUM>. (i) of <FIG> shows a result in which the transmission signal is quantized with <NUM> range diversities, but a maximum sampling point is not estimated, and (ii) of <FIG> shows a result in which the transmission signal is quantized with <NUM> range diversities, but a maximum sampling point is not estimated. (iii) of <FIG> shows a result in which the transmission signal is quantized with <NUM> range diversities and a maximum sampling point is estimated, and (iv) of <FIG> shows a result in which the transmission signal is quantized with <NUM> range diversities, but a maximum sampling point is estimated. As shown in the drawings, an error range is about ±<NUM> when the transmission signal is quantized without range diversity and the distance to the object is measured using the maximum sampling point of the cross-correlation signal, but as a result of application of <NUM> range diversities, the error range is reduced to about ±<NUM>. As the number of range diversities is increased to <NUM>, the error range is significantly reduced to about ±<NUM>. In addition, it may be seen that he error range is further reduced when the maximum sampling point is estimated, in comparison to when only range diversity is applied.

<FIG> illustrates a block diagram of an object detection device, according to another unclaimed example embodiment of the present disclosure. Comparing <FIG> with <FIG>, the detector <NUM> of the object detection device <NUM> of <FIG> may include a plurality of sub detectors 210a. For example, the detector 210a may include first through third sub detectors <NUM>, <NUM>, and <NUM>. Each of the first through third sub detectors <NUM>, <NUM>, and <NUM> may detect light reflected from the object and output a received signal. The transmitter <NUM> emits light corresponding to the transmission signal, and each received signal corresponds to a part of light reflected from the object and includes the same distance information with respect to the object.

The time-delaying unit <NUM> may include the signal line <NUM> that applies the first sub received signal output from one sub detector <NUM> among the plurality of sub detectors 210a, and the one or more time-delay lines <NUM> and <NUM> that time-delay the time-delayed sub received signals than the first sub received signal among the plurality of sub received signals with respect to the first sub received signal. The object detection device <NUM> of <FIG> may be different from that of <FIG> in that the object detection device <NUM> does not include a splitter for splitting the received signal and each sub detector <NUM> outputs the sub received signal. Noise may be added as the received signal is split by a splitter. The object detection device <NUM> of <FIG> may prevent noise addition by the splitter.

So far, it has been described that the received signal is converted into a digital signal having range diversity, but the present disclosure is not limited thereto. When the received signal is simply converted into the digital signal and the plurality of sampling points included in the cross-correlation signal are selected, the maximum sampling point may be estimated based on the plurality of sampling points and the distance to the object may be obtained using a time of a sampling point having the intensity of the maximum sampling point.

<FIG> illustrates a block diagram of an object detection device without a time-delaying unit, according to another unclaimed example embodiment of the present disclosure. As shown in <FIG>, the object detection device may not include a time-delaying unit. When the received signal is simply converted into the digital signal and the plurality of sampling points included in the cross-correlation signal are selected, the maximum sampling point may be estimated based on the plurality of sampling points and the distance to the object may be obtained using a time of a sampling point having the intensity of the maximum sampling point. When the distance is determined using the maximum sampling point without using the digital signal of range diversity, the distance accuracy may be improved.

The detection object device according to an example embodiment of the present disclosure may be applied to various electronic devices for detecting the distance to the object or obtaining a 3D image.

<FIG> is a block diagram illustrating a schematic structure of an electronic device, according to an unclaimed example embodiment of the present disclosure.

Referring to <FIG>, the electronic device <NUM> in a network environment <NUM> may communicate with another electronic device <NUM> via a first network <NUM> (e.g., a short-range wireless communication network, etc.), or another electronic device <NUM> or a server <NUM> via a second network <NUM> (e.g., a long-range wireless communication network, etc.). The electronic device <NUM> may communicate with the electronic device <NUM> via the server <NUM>. The electronic device <NUM> may include a processor <NUM>, memory <NUM>, an input device <NUM>, a sound output device <NUM>, a display device <NUM>, an audio module <NUM>, a sensor module <NUM>, an interface <NUM>, a haptic module <NUM>, a camera module <NUM>, a power management module <NUM>, a battery <NUM>, a communication module <NUM>, a subscriber identification module <NUM>, and/or an antenna module <NUM>. Some (e.g., the display device <NUM>, etc.) of the components may be omitted from the electronic device <NUM>, or other components may be added to the electronic device <NUM>. Some of the components may be implemented as a single integrated circuitry. For example, a fingerprint sensor <NUM>, an iris sensor, an illuminance sensor, etc., of the sensor module <NUM> may be implemented as embedded in the display device <NUM> (e.g., a display, etc.).

The processor <NUM> may execute software (e.g., a program <NUM>, etc.) to control one component or a plurality of different components (e.g., a hardware or software component, etc.) of the electronic device <NUM> coupled with the processor <NUM>, and may perform various data processing or computation. As a part of the data processing or computation, the processor <NUM> may load a command or data received from another component (e.g., the sensor module <NUM>, the communication module <NUM>, etc.) in volatile memory <NUM>, process the command and/or the data stored in the volatile memory <NUM>, and store resulting data in non-volatile memory <NUM>. The processor <NUM> may include a main processor <NUM> (e.g., a central processing unit, an application processor, etc.), and an auxiliary processor <NUM> (e.g., a graphics processing unit, an image signal processor, a sensor hub processor, a communication processor, etc.) that is operable independently from, or in conjunction with, the main processor <NUM>. The auxiliary processor <NUM> may use less power than the main processor <NUM> and perform a specialized function.

The auxiliary processor <NUM> may control functions and/or states related to some components (e.g., the display device <NUM>, the sensor module <NUM>, the communication module <NUM>, etc.) among the components of the electronic device <NUM>, instead of the main processor <NUM> while the main processor <NUM> is in an inactive (e.g., sleep) state, or together with the main processor <NUM> while the main processor <NUM> is in an active (e.g., application execution) state. The auxiliary processor <NUM> (e.g., an image signal processor, a communication processor, etc.) may be implemented as part of another component (e.g., the camera module <NUM>, the communication module <NUM>, etc.) functionally related thereto.

The memory <NUM> may store various data needed by a component (e.g., the processor <NUM>, the sensor module <NUM>, etc.) of the electronic device <NUM>. The various data may include, for example, software (e.g., the program <NUM>, etc.) and input data and/or output data for a command related thereto. The memory <NUM> may include the volatile memory <NUM> and/or the non-volatile memory <NUM>.

The program <NUM> may be stored in the memory <NUM> as software, and may include, for example, an operating system <NUM>, middleware <NUM>, and/or an application <NUM>.

The input device <NUM> may receive a command and/or data to be used by other component (e.g., the processor <NUM>, etc.) of the electronic device <NUM>, from the outside (e.g., a user, etc.) of the electronic device <NUM>. The input device <NUM> may include a microphone, a mouse, a keyboard, and/or a digital pen (e.g., a stylus pen, etc.).

The sound output device <NUM> may include a speaker and/or a receiver. The receiver may be coupled as a part of the speaker or may be implemented as an independent separate device.

The display device <NUM> may visually provide information to the outside of the electronic device <NUM>. The display device <NUM> may include a display, a hologram device, or a projector and control circuitry to control a corresponding one of the display, hologram device, and projector. The display device <NUM> may include touch circuitry adapted to detect a touch, and/or sensor circuitry (e.g., a pressure sensor, etc.) adapted to measure the intensity of force incurred by the touch.

The audio module <NUM> may obtain the sound via the input device <NUM>, or output the sound via the sound output device <NUM> and/or a speaker and/or a headphone of another electronic device (e.g., the electronic device <NUM>, etc.) directly (e.g., wiredly) or wirelessly coupled with the electronic device <NUM>.

The sensor module <NUM> may detect an operational state (e.g., power, temperature, etc.) of the electronic device <NUM> or an environmental state (e.g., a state of a user, etc.) external to the electronic device <NUM>, and then generate an electrical signal and/or data value corresponding to the detected state. The sensor module <NUM> may include the fingerprint sensor <NUM>, an acceleration sensor <NUM>, a position sensor <NUM>, a 3D sensor <NUM>, etc., and also include an iris sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a bio sensor, a temperature sensor, a humidity sensor, and/or an illuminance sensor.

The 3D sensor <NUM> may sense shape, movement, etc. of the object by radiating light to the object and analyzing the light reflected from the object, and may include the object detection device <NUM> according to the above-described embodiment of the present disclosure.

The interface <NUM> may support one or more specified protocols to be used for the electronic device <NUM> to be coupled with another electronic device (e.g., the electronic device <NUM>, etc.) directly or wirelessly. The interface <NUM> may include a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, and/or an audio interface.

A connecting terminal <NUM> may include a connector via which the electronic device <NUM> may be physically connected with another electronic device (e.g., the electronic device <NUM>, etc.). The connecting terminal <NUM> may include, for example, a HDMI connector, a USB connector, a SD card connector, and/or an audio connector (e.g., a headphone connector, etc.).

The haptic module <NUM> may convert an electrical signal into a mechanical stimulus (e.g., a vibration, motion, etc.) or electrical stimulus which may be recognized by a user via his tactile sensation or kinesthetic sensation. The haptic module <NUM> may include a motor, a piezoelectric element, and/or an electric stimulator.

The camera module <NUM> may include a lens assembly including one or more lenses, image sensors, image signal processors, and/or flashes. The lens assembly included in the camera module <NUM> may collect light emitted from an object whose image is to be taken.

The power management module <NUM> may be implemented as a part of a power management integrated circuit (PMIC).

The battery <NUM> may supply power to a component of the electronic device <NUM>. The battery <NUM> may include a primary cell which is not rechargeable, a secondary cell which is rechargeable, and/or a fuel cell.

The communication module <NUM> may support establishing a direct (e.g., wired) communication channel and/or a wireless communication channel between the electronic device <NUM> and another electronic device (e.g., the electronic device <NUM>, the electronic device <NUM>, the server <NUM>, etc.) and performing communication via the established communication channel. The communication module <NUM> may include one or more communication processors that are operable independently from the processor <NUM> (e.g., the application processor, etc.) and supports a direct communication and/or a wireless communication. The communication module <NUM> may include a wireless communication module <NUM> (e.g., a cellular communication module, a short-range wireless communication module, a global navigation satellite system (GNSS) communication module, etc.) and/or a wired communication module <NUM> (e.g., a local area network (LAN) communication module, a power line communication module, etc.). A corresponding one of these communication modules may communicate with the external electronic device via the first network <NUM> (e.g., a short-range communication network, such as Bluetooth™, Wireless-Fidelity (Wi-Fi) direct, or infrared data association (IrDA)) or the second network <NUM> (e.g., a long-range communication network, such as a cellular network, the Internet, or a computer network (e.g., LAN, wide area network (WAN), etc.). These various types of communication modules may be implemented as a single component (e.g., a single chip, etc.), or may be implemented as multi components (e.g., multi chips) separate from each other. The wireless communication module <NUM> may identify and authenticate the electronic device <NUM> in a communication network, such as the first network <NUM> and/or the second network <NUM>, using subscriber information (e.g., international mobile subscriber identity (IMSI), etc.) stored in the subscriber identification module <NUM>.

The antenna module <NUM> may transmit or receive a signal and/or power to or from the outside (e.g., another electronic device, etc.). The antenna may include a radiator including a conductive pattern formed on a substrate (e.g., a printed circuit board (PCB), etc.). The antenna module <NUM> may include one antenna or a plurality of antennas. When the plurality of antennas are included, an antenna that is appropriate for a communication scheme used in a communication network such as the first network <NUM> and/or the second network <NUM> may be selected by the communication module <NUM> from among the plurality of antennas. The signal and/or the power may then be transmitted or received between the communication module <NUM> and another electronic device via the selected antenna. A part (e.g., a radio frequency integrated circuit (RFIC), etc. other than an antenna may be included as a part of the antenna module <NUM>.

Some of the above-described components may be coupled mutually and communicate signals (e.g., commands, data, etc.) therebetween via an inter-peripheral communication scheme (e.g., a bus, general purpose input and output (GPIO), a serial peripheral interface (SPI), mobile industry processor interface (MIPI), etc.).

The other electronic devices <NUM> and <NUM> may be a device of a same type as, or a different type, from the electronic device <NUM>. All or some of operations to be executed at the electronic device <NUM> may be executed at one or more of the other electronic devices <NUM>, <NUM>, and <NUM>. For example, when the electronic device <NUM> performs a function or a service, the electronic device <NUM>, instead of executing the function or the service, may request the one or more other electronic devices to perform the entire function or service or a part thereof. One or more other electronic devices receiving a request may perform an additional function or an additional service related to the request, and transfer an outcome of the performing to the electronic device <NUM>. To that end, a cloud computing, distributed computing, and/or client-server computing technology may be used, for example.

The object detection device <NUM> according to the embodiments of the present disclosure may be applied to a mobile phone or smart phone <NUM> shown in <FIG>, a tablet or smart tablet <NUM> shown in <FIG>, a laptop <NUM> shown in <FIG> , etc. For example, the smartphone <NUM> or the smart tablet <NUM> may extract depth information of subjects in an image by using the object detection device <NUM> that is an object 3D sensor, adjust out-focusing of the image, or automatically identify the subjects in the image.

The object detection device <NUM> may also be applied to a smart refrigerator <NUM> shown in <FIG>, a security camera <NUM> shown in <FIG>, a robot <NUM> shown in <FIG>, etc. For example, the smart refrigerator <NUM> may automatically recognize food in the refrigerator using an image sensor, and inform a user of the presence of specific food, the type of incoming or outgoing food, etc., through the smartphone. A security camera <NUM> may make it possible to recognize an object or people in the image even in a dark environment. A robot <NUM> may provide a 3D image by being put into a disaster or industrial site that is not directly accessible by humans.

In addition, the object detection device <NUM>, which is a 3D sensor, may be applied to a vehicle <NUM> as illustrated in <FIG>. The vehicle <NUM> may include a plurality of object detection devices (<NUM>, <NUM>, <NUM>, and <NUM>) disposed at various locations. The vehicle <NUM> may use the plurality of object detection devices (<NUM>, <NUM>, <NUM>, and <NUM>) to provide a driver with various information about the interior or surroundings of the vehicle <NUM>, and automatically recognize an object or people in the image and provide information necessary for autonomous driving.

Meanwhile, an operating method of the above-described object detecting device <NUM> may be recorded in a computer-readable recording medium in which one or more programs including instructions for executing the operating method are recorded. Examples of the computer-readable recording medium may include magnetic media such as hard disk, floppy disk, and magnetic tape, optical media such as compact disk read only memory (CD-ROM) and digital versatile disk (DVD), magneto-optical media such as floptical disk, and a hardware device especially configured to store and execute a program command, such as read only memory (ROM), random access memory (RAM), flash memory, etc. Examples of the program instructions include a machine language code created by a complier and a high-level language code executable by a computer using an interpreter.

The disclosed object detection device and the operating method thereof may increase distance accuracy with respect to an object by using a digital signal of range diversity.

Claim 1:
An object detection device (<NUM>) comprising:
a detector (<NUM>) configured to detect light reflected from an object (<NUM>) and output an electrical signal in an analog domain corresponding to the light, as a received signal;
a converter (<NUM>) configured to perform analog-to-digital conversion on the received signal;
a splitter (<NUM>) provided between the detector and the converter, and configured to split the received signal into a plurality of sub received signals;
a plurality of signal lines provided between the splitter and the converter, the plurality of signal lines comprising:
a non-delay line (<NUM>) configured to send a first sub received signal from among the plurality of sub received signals to the converter; and
one or more time-delay lines (<NUM>, <NUM>) configured to time-delay sub received signals other than the first sub received signal, from among the plurality of sub received signals, and send the time-delayed sub received signals to the converter; and
a processor (<NUM>) configured to determine a distance to the object based on a signal output from the converter,
wherein the converter is further configured to convert the first sub received signal into a digital signal and convert the time-delayed sub received signals into one or more time-delayed digital signals,
wherein a time delay of the time-delayed sub received signals is less than a sampling period of the converter,
characterized in that the processor is further configured to generate a plurality of cross-correlation signals between each of the digital signal and the one or more time-delayed digital signals and a transmission signal corresponding to the light, determine a plurality of maximum sampling points from each of the plurality of cross-correlation signals, and determine the distance to the object based on the plurality of maximum sampling points.