Method and device for tracking object using LiDAR sensor, vehicle including the device, and recording medium storing program to execute the method

A method of tracking an object using a LiDAR sensor includes clustering LiDAR data that includes a plurality of points for an object detected by the LiDAR sensor, generating information on a plurality of segment boxes for each channel using a result of the clustering, and selecting, among the segment boxes, an associated segment box at a current time for a target object that is being tracked. The selecting includes calculating a correlation index between a current representative point and each of a tracking representative point and a previous representative point of each of the segment boxes at the current time, selecting candidate segment boxes for the associated segment box using the correlation index, and selecting the associated segment box at the current time.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority Korean Patent Application No. 10-2020-0124752, filed on Sep. 25, 2020, which application is hereby incorporated herein by reference.

TECHNICAL FIELD

Embodiments relate to a method and device for tracking an object using a LiDAR sensor, a vehicle including the device, and a recording medium storing a program to execute the method.

BACKGROUND

A highway driving pilot (HDP) system of a vehicle is a system that maintains the speed of the vehicle according to conditions set by a driver based on information set by the driver about the speed of the vehicle and the distance to a preceding vehicle traveling in the lane of the vehicle without operation of an accelerator pedal or a brake pedal by the driver.

For example, information on a target vehicle may be obtained using a light detection and ranging (LiDAR) sensor, and an HDP function may be performed using the obtained information. However, if the information on the target vehicle obtained using the LiDAR sensor is incorrect, the HDP function may be erroneously performed, leading to deterioration in the reliability of the vehicle.

SUMMARY

Accordingly, embodiments are directed to a method and device for tracking an object using a LiDAR sensor, a vehicle including the device, and a recording medium storing a program to execute the method that substantially obviate one or more problems due to limitations and disadvantages of the related art.

Embodiments may provide a method and device for tracking an object using a LiDAR sensor having stable tracking performance, a vehicle including the device, and a recording medium storing a program to execute the method.

However, the objects to be accomplished by the embodiments are not limited to the above-mentioned object, and other objects not mentioned herein will be clearly understood by those skilled in the art from the following description.

A method of tracking an object using a LiDAR sensor according to an embodiment may include clustering LiDAR data composed of a plurality of points for an object obtained by the LiDAR sensor, generating information on a plurality of segment boxes for each channel using the result of the clustering, and selecting, among the plurality of segment boxes, an associated segment box at the current time t for a target object that is being tracked. The selecting may include obtaining a correlation index between a current representative point and each of a tracking representative point and a previous representative point of each of the plurality of segment boxes at the current time t, selecting, among the plurality of segment boxes, candidate segment boxes for the associated segment box using the correlation index, and selecting, among the selected candidate segment boxes, the associated segment box at the current time t. The tracking representative point may correspond to a representative point of a tracking box of the target object at the current time t, estimated using history information. The previous representative point may correspond to a representative point of a segment box selected as the associated segment box at a previous time t−1.

For example, the current representative point may include a first peripheral representative point located at a corner of the segment box and a first central representative point located at the center of the segment box. The tracking representative point may include a second peripheral representative point located at a corner of the tracking box and a second central representative point located at the center of the tracking box. The previous representative point may include a third peripheral representative point located at a periphery of the associated segment box selected at the previous time.

For example, the correlation index may include a first correlation index between the first peripheral representative point and the second peripheral representative point, a second correlation index between the first central representative point and the second central representative point, and a third correlation index between the first peripheral representative point and the third peripheral representative point.

For example, the first, second and third correlation indices may be obtained as follows.

Here, γ1represents the first correlation index, γ2represents the second correlation index, γ3represents the third correlation index, each of xm0and xtm0represents the horizontal-axis coordinate of the first peripheral representative point, each of ym0and ytm0represents the vertical-axis coordinate of the first peripheral representative point, x0represents the horizontal-axis coordinate of the second peripheral representative point, y0represents the vertical-axis coordinate of the second peripheral representative point, σ2xmrepresents the horizontal-axis variance value of the first peripheral representative point, σ2ymrepresents the vertical-axis variance value of the first peripheral representative point, σ2xrepresents the horizontal-axis variance value of the second peripheral representative point, σ2yrepresents the vertical-axis variance value of the second peripheral representative point, xmcrepresents the horizontal-axis coordinate of the first central representative point, ymcrepresents the vertical-axis coordinate of the first central representative point, σ2xmcrepresents the horizontal-axis variance value of the first central representative point, σ2ymcrepresents the vertical-axis variance value of the first central representative point, xcrepresents the horizontal-axis coordinate of the second central representative point, ycrepresents the vertical-axis coordinate of the second central representative point, xt-1m0represents the horizontal-axis coordinate of the third peripheral representative point, and yt-1m0represents the vertical-axis coordinate of the third peripheral representative point.

For example, in the selecting the candidate segment boxes, one of the plurality of segment boxes that satisfies at least one of three conditions below may be selected as a candidate segment box for the associated segment box.
γ1C1
γ2C2
γ3C3

For example, one of the plurality of segment boxes that does not satisfy the three conditions but overlaps the tracking box may be selected as a candidate segment box.

For example, the selecting the associated segment box may include assigning a first score to distance suitability of each of the candidate segment boxes, assigning a second score to reliability suitability of each of the candidate segment boxes, assigning a third score to correlation between each of the candidate segment boxes and the associated segment box selected at the previous time, summing the first to third scores assigned to each of the candidate segment boxes to calculate a final score, and selecting, among the candidate segment boxes, a candidate segment box that has the highest final score as the associated segment box at the current time t.

For example, in the assigning the first score, the first score may be assigned to one of the candidate segment boxes in which the smallest one of the first to third correlation indices is equal to or less than a first threshold value.

For example, the assigning the second score may include obtaining a reliability level of each of the candidate segment boxes and assigning the second score to one of the candidate segment boxes that has a reliability level greater than a second threshold value.

For example, the obtaining the reliability level may include a shape reliability level determination step of obtaining a preset 2-1streliability level corresponding to the shape of the candidate segment box, an attribute reliability level determination step of obtaining a preset 2-2ndreliability level corresponding to at least one of the ratio of the number of points included in the candidate segment box to the size of the candidate segment box or the degree of dispersion of points included in the candidate segment box, a geometric reliability level determination step of obtaining a preset 2-3rdreliability level corresponding to the position of the candidate segment box and the distance from a reference point to the candidate segment box, and summing the preset 2-1st, 2-2nd, and 2-3rdreliability levels to determine the reliability level to be compared with the second threshold value.

For example, in the assigning the third score, the third score may be assigned in proportion to the ratio of an area of each candidate segment box that overlaps the associated segment box selected at the previous time to the entire area of each candidate segment box.

For example, the method may further include determining whether the associated segment box selected at the current time t is present, upon determining that the associated segment box is present, updating the history information of a channel in which the associated segment box is included, and upon determining that the associated segment box is not present, deleting the history information of a channel in which the associated segment box is not present.

A device for tracking an object using a LiDAR sensor according to another embodiment may include a clustering unit configured to group LiDAR data composed of a plurality of points for an object obtained by the LiDAR sensor, a shape analysis unit configured to generate information on a plurality of segment boxes for each channel using the result of clustering, and an object-tracking unit configured to select, among the plurality of segment boxes, an associated segment box at the current time t for a target object that is being tracked. The object-tracking unit may include a storage unit configured to store history information for each channel, a correlation index calculation unit configured to calculate a correlation index between a current representative point and each of a tracking representative point and a previous representative point of each of the plurality of segment boxes at the current time t, a candidate selection unit configured to select, among the plurality of segment boxes, candidate segment boxes for the associated segment box using the correlation index, and a final selection unit configured to select, among the selected candidate segment boxes, the associated segment box at the current time t. The tracking representative point may correspond to a representative point of a tracking box of the target object at the current time t, estimated using the history information. The previous representative point may correspond to a representative point of a segment box selected as the associated segment box at a previous time t−1.

For example, the correlation index calculation unit may include a first index calculation unit configured to calculate a first correlation index between a first peripheral representative point and a second peripheral representative point, a second index calculation unit configured to calculate a second correlation index between a first central representative point and a second central representative point, and a third index calculation unit configured to calculate a third correlation index between the first peripheral representative point and a third peripheral representative point. The current representative point may include the first peripheral representative point located at a corner of the segment box and the first central representative point located at the center of the segment box. The tracking representative point may include the second peripheral representative point located at a corner of the tracking box and the second central representative point located at the center of the tracking box. The previous representative point may include the third peripheral representative point located at a periphery of the associated segment box selected at the previous time.

For example, the candidate selection unit may include a first comparison unit configured to compare the first correlation index for each of the plurality of segment boxes with a first critical index, a second comparison unit configured to compare the second correlation index for each of the plurality of segment boxes with a second critical index, a third comparison unit configured to compare the third correlation index for each of the plurality of segment boxes with a third critical index, and a box selection unit configured to select a candidate segment box for the associated segment box in response to the results of the comparison by the first to third comparison units.

For example, the candidate selection unit may further include an overlap determination unit configured to determine whether one of the plurality of segment boxes that has not been selected as the candidate segment box overlaps the tracking box using the correlation index in response to a control signal. The box selection unit may generate the control signal in response to the results of the comparison by the first to third comparison units, and may select the candidate segment box in response to the result of the determination by the overlap determination unit.

For example, the final selection unit may include a score assignment unit configured to assign a first score to distance suitability of each of the candidate segment boxes, to assign a second score to reliability suitability of each of the candidate segment boxes, and to assign a third score to correlation between each of the candidate segment boxes and the associated segment box selected at the previous time, a score calculation unit configured to sum the first to third scores assigned to each of the candidate segment boxes to calculate a final score, and a score comparison unit configured to select one of the candidate segment boxes that has the highest final score as the associated segment box at the current time t.

A vehicle according to still another embodiment may include a LiDAR sensor and a device for tracking an object using the LiDAR sensor.

According to still another embodiment, a recording medium in which a program for executing a method of tracking an object using a LiDAR sensor is recorded may store a program to implement a clustering function of grouping LiDAR data composed of a plurality of points for an object obtained by the LiDAR sensor, a function of generating information on a plurality of segment boxes for each channel using the result of clustering, and a function of selecting, among the plurality of segment boxes, an associated segment box at the current time t for a target object that is being tracked. The function of selecting may include a function of calculating a correlation index between a current representative point and each of a tracking representative point and a previous representative point of each of the plurality of segment boxes at the current time t, a function of selecting, among the plurality of segment boxes, candidates for the associated segment box using the correlation index, and a function of selecting, among the selected candidates, the associated segment box at the current time t. The tracking representative point may correspond to a representative point of a tracking box of the target object at the current time t, estimated using history information. The previous representative point may correspond to a representative point of a segment box selected as the associated segment box at a previous time t−1.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The present disclosure will now be described more fully hereinafter with reference to the accompanying drawings, in which various embodiments are shown. The examples, however, may be embodied in many different forms, and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be more thorough and complete, and will more fully convey the scope of the disclosure to those skilled in the art.

It will be understood that when an element is referred to as being “on” or “under” another element, it may be directly on/under the element, or one or more intervening elements may also be present.

When an element is referred to as being “on” or “under”, “under the element” as well as “on the element” may be included based on the element.

In addition, relational terms, such as “first”, “second”, “on/upper part/above” and “under/lower part/below”, are used only to distinguish between one subject or element and another subject or element, without necessarily requiring or involving any physical or logical relationship or sequence between the subjects or elements.

Hereinafter, a method and device600for tracking an object using a light detection and ranging (LiDAR) sensor500and a vehicle1000using the same according to embodiments will be described with reference to the accompanying drawings. For convenience of description, the method and device600for tracking an object using the LiDAR sensor500and the vehicle1000using the same will be described using the Cartesian coordinate system (x-axis, y-axis, z-axis). However, the embodiments are not limited thereto. In the Cartesian coordinate system, the x-axis, the y-axis, and the z-axis are perpendicular to each other, but the embodiments are not limited thereto. That is, the x-axis, the y-axis, and the z-axis may intersect each other obliquely.

FIG.1is a flowchart for explaining an object-tracking method using the LiDAR sensor500according to an embodiment, andFIG.2is a block diagram for explaining an object-tracking device600using the LiDAR sensor500according to an embodiment.

For convenience of description, the object-tracking method shown inFIG.1will be described as being performed by the object-tracking device600shown inFIG.2, but the embodiments are not limited thereto. That is, according to another embodiment, the object-tracking method shown inFIG.1may be performed by an object-tracking device having a configuration different from that of the object-tracking device600shown inFIG.2. In addition, the object-tracking device600shown inFIG.2will be described as performing the object-tracking method shown inFIG.1, but the embodiments are not limited thereto. That is, according to another embodiment, the object-tracking device600shown inFIG.2may perform an object-tracking method having processes different from those of the object-tracking method shown inFIG.1.

The object-tracking device600using the LiDAR sensor500shown inFIG.2may include a clustering unit620, a shape analysis unit (or a segment unit)630, and an object-tracking unit (a tracking unit, a tracking and classification unit, or an object detection unit)640. In addition, the object-tracking device600may further include a preprocessing unit610. In addition, the vehicle1000according to an embodiment may include the LiDAR sensor500, the object-tracking device600, and a vehicle device700.

The LiDAR sensor500may radiate a single circular laser pulse having a wavelength of, for example, 905 nm to 1550 nm to an object present within a measurement range, and may measure the time taken for the laser pulse reflected from the object to return, thereby detecting information on the object, for example, the distance from the LiDAR sensor500to the object, the orientation of the object, the speed of the object, the temperature of the object, the material distribution of the object, and the concentration characteristics of the object. Here, the object may be, for example, another vehicle, a person, or an object present outside the vehicle1000in which the LiDAR sensor500is mounted (hereinafter referred to as the “host vehicle”). However, the embodiments are not limited to any specific type of object.

The LiDAR sensor500may include a transmitter (not shown), which transmits a laser pulse, and a receiver (not shown), which receives the laser reflected from the surface of an object present within a detection range. The receiver has a predetermined field of view (FOV), which is a range that the LiDAR sensor500is capable of observing simultaneously without moving or rotating.

Since the LiDAR sensor500exhibits higher detection accuracy in vertical/horizontal directions than a radio detecting and ranging (RaDAR) sensor, the LiDAR sensor500is capable of providing accurate vertical/horizontal-directional position information, and is thus advantageously used for obstacle detection and vehicle position recognition. As examples of the LiDAR sensor500, there are a two-dimensional (2D) LiDAR sensor and a three-dimensional (3D) LiDAR sensor. The 2D LiDAR sensor is configured to be tiltable or rotatable, and is used to obtain LiDAR data including 3D information through tilting or rotation. The 3D LiDAR sensor is capable of obtaining a plurality of 3D points and thus of predicting the height information of an obstacle, thus helping in accurate and precise detection and tracking of an object. The 3D LiDAR sensor may be composed of multiple 2D LiDAR sensor layers, and may generate LiDAR data including 3D information.

The LiDAR sensor500outputs point cloud data (hereinafter referred to as “LiDAR data”) composed of a plurality of points for a single object.

The method and device600for tracking an object according to the embodiments are not limited to any specific shape, position, or type of LiDAR sensor500.

The object-tracking device600may receive LiDAR data, and may use the same to determine the presence or absence of an object, to start, continue, or stop tracking an object, to update, store, or delete information on an object, and to classify the type of object.

The preprocessing unit610may preprocess LiDAR data (step100). To this end, the preprocessing unit610may perform calibration to match the coordinates between the LiDAR sensor500and the vehicle1000. That is, the preprocessing unit610may convert LiDAR data into data suitable for the reference coordinate system according to the positional angle at which the LiDAR sensor500is mounted to the vehicle1000. In addition, the preprocessing unit610may perform filtering to remove points having low intensity or reflectance using intensity or confidence information of the LiDAR data.

In addition, the preprocessing unit610may remove data reflected by the body of the host vehicle1000. That is, since there is a region that is shielded by the body of the host vehicle1000according to the mounting position and the field of view of the LiDAR sensor500, the preprocessing unit610may remove data reflected by the body of the host vehicle1000using the reference coordinate system.

In the object-tracking method according to the embodiment, step100may be omitted. In this case, the preprocessing unit610may be omitted from the object-tracking device600according to the embodiment.

After step100, the clustering unit620groups the point cloud data, which is the LiDAR data consisting of a plurality of points for the object obtained through the LiDAR sensor500, into meaningful units according to a predetermined criterion (step200). In the case in which step100, which is the preprocessing step, and the preprocessing unit610are not omitted, the clustering unit620may group the LiDAR data preprocessed by the preprocessing unit610. For example, the clustering unit620may group the point cloud data by applying vehicle modeling or guardrail modeling thereto to perform clustering to determine the external appearance of the object. The result detected by the LiDAR sensor500shows a plurality of points, each of which has only position information. Accordingly, the clustering unit620serves to group the plurality of points detected by the LiDAR sensor500into meaningful shape units.

As examples of the clustering unit620, there are a 2D clustering unit and a 3D clustering unit. The 2D clustering unit is a unit that performs clustering in units of points or a specific structure by projecting data onto the X-Y plane without considering height information. The 3D clustering unit is a unit that performs clustering in the X-Y-Z plane in consideration of height information Z.

After step200, the shape analysis unit630generates information on a plurality of segment boxes for each channel using the result of clustering from the clustering unit620(step300). Here, the segment box may be the result of converting the result of clustering into a geometric box shape. In addition, the information on the segment box may be at least one of the width, length, position, or direction (or heading) of the segment box. The channel will be described later.

The following description of step400according to the embodiment is not limited to the presence or absence of step100or to any specific method of performing the preprocessing process in step100, the clustering process in step200, or the process of generating segment box information in step300. Similarly, the following description of the object-tracking unit640according to the embodiment is not limited to the presence or absence of the preprocessing unit610or to any specific type of operation performed by the preprocessing unit610, the clustering unit620, or the shape analysis unit630. That is, step400and the object-tracking unit640according to the embodiments may also be applied when the preprocessing unit610is omitted (i.e. when step100is omitted), when the preprocessing unit610performing step100processes LiDAR data in a manner different from that described above, when the clustering unit620performing step200clusters LiDAR data in a manner different from that described above, or when the shape analysis unit630performing step300generates segment box information in a manner different from that described above.

After step300, the object-tracking unit640selects a segment box in association with the object that is being tracked (hereinafter referred to as a “target object”) at the current time t (a final segment box or an associated segment box), among a plurality of segment boxes for each channel (step400). A plurality of segment boxes may be obtained with respect to the same object depending on the visibility of the LiDAR sensor500and the shape of the object. Here, the term “association” is a process of selecting a segment box that is to be used to maintain tracking of a target object that is being currently tracked, among a plurality of pieces of segment box information. This association may be performed at a predetermined period.

In order to select an associated segment box from the respective plurality of segment boxes provided by each channel from the shape analysis unit630, the object-tracking unit640may convert the information on each of the plurality of segment boxes into a predetermined format, and may select an associated segment box among the plurality of segment boxes having the converted format (or segment boxes of a meta object).

FIGS.3A to3Care diagrams for explaining the format of data (i.e. segment box information) processed by the object-tracking unit640.

The method and device600for tracking an object according to the embodiments may track “M” target objects. Here, “M” is a positive integer of 1 or more. That is, the number M of target objects that may be tracked is the number M of tracks Trk shown inFIG.3A. In addition, the unit in which history information on a unit target object is stored is referred to as a “channel”, and the number of channels is the same as the number of tracks Trk. In this case, the history information may be information accumulated in each channel prior to the current time t with respect to the target object that is being tracked. The history information may include, for example, position information and speed information of the target object, measured periodically.

In addition, “N” segment boxes Seg #1to Seg #N may be generated at the current time t with respect to the unit target object by the shape analysis unit630, and may be provided to the object-tracking unit640. Here, “N” is a positive integer of 1 or more, and may be the same as or different from “M”. Hereinafter, “N” will be described as being a positive integer of 2 or more, but the following description may also apply to the configuration in which “N” is 1. That is, as shown inFIG.3C, “N” segment boxes Seg #1to Seg #N may be included in each Trk #m (1≤m≤M) of the first to Mthchannels Trk #1to Trk #M.

The object-tracking device600selects an associated segment box in each channel at the current time t with respect to the target object that is being currently tracked, among the “N” segment boxes Seg #1to Seg #N included in each of the first to Mthchannels (step400).

Hereinafter, for convenience of description, the process of selecting an associated segment box in the mthchannel Trk #m at the current time t with respect to the target object that is being currently tracked, among the “N” segment boxes Seg #1to Seg #N included in the mthchannel Trk #m shown inFIG.3A, will be described. However, the following description may also apply to a process of selecting an associated segment box at the current time t with respect to the target object that is being currently tracked, among the “N” segment boxes Seg #1to Seg #N included in each of the other channels.

FIG.4is a flowchart for explaining an embodiment400A of step400shown inFIG.1, andFIG.5is a block diagram for explaining an embodiment640A of the object-tracking unit640shown inFIG.2.

For convenience of description, step400A shown inFIG.4will be described as being performed by the object-tracking unit640A shown inFIG.5, but the embodiments are not limited thereto. That is, according to another embodiment, step400A shown inFIG.4may be performed by an object-tracking unit having a configuration different from that of the object-tracking unit640A shown inFIG.5. In addition, the object-tracking unit640A shown inFIG.5will be described as performing step400A shown inFIG.4, but the embodiments are not limited thereto. That is, according to another embodiment, the object-tracking unit640A shown inFIG.5may perform a method having processes different from those of step400A shown inFIG.4.

The object-tracking unit640A shown inFIG.5may include a storage unit642, a correlation index calculation unit644, a candidate selection unit646, and a final selection unit648.

The correlation index calculation unit644calculates the correlation index between the current representative point and each of the tracking representative point and the previous representative point of each of the multiple (i.e. “N”) segment boxes Seg #1to Seg #N included in the mthchannel Trk #m at the current time t, and outputs the calculated correlation index to the candidate selection unit646(step410). That is, the correlation index calculation unit644may calculate the correlation index between the current representative point and the tracking representative point and the correlation index between the current representative point and the previous representative point.

FIGS.6A and6Bare diagrams for explaining the concepts of the current representative point, the tracking representative point, and the previous representative point.

FIG.6Ashows each Btof a plurality of segment boxes at the current time t, andFIG.6Bshows an associated segment box Bt-1selected at a time t−1 prior to the current time t. In addition,FIG.6Bshows a tracking box TB of the target object estimated using history information at the current time t. For example, a tracking box TB may be generated by estimating tracking information, such as the current position, shape, and speed of the target object that is being tracked, using history information.

The storage unit642may store the history information for the respective channels Trk #1to Trk #M shown inFIG.3A.

In order to select an associated segment box for the target object at the current time t, the selecting of the representative point of each of the boxes Bt, Bt-1and TB is very important to accurately perform “association”. The reason for this is that the selecting of an associated segment box is accomplished through comparison between points.

The current representative point is a representative point of each Btof the plurality of segment boxes at the current time t, which is provided from the shape analysis unit630to the correlation index calculation unit644through an input terminal IN1. For example, the current representative point may include a representative point located at the periphery (or the edge) of the box Bt(hereinafter referred to as a “first peripheral representative point”) and a representative point located at the center of the box Bt(hereinafter referred to as a “first central representative point”). For example, as shown inFIG.6A, reference numerals Pm0, Pm1, Pm2and Pm3are assigned to the first peripheral representative points of each Btof the plurality of segment boxes at the current time t, in the clockwise direction from the lower left-hand corner thereof, and reference numeral Pmcis assigned to the first central representative point located at the center thereof.

The tracking box is a box at which the associated segment box of the target object, which is being tracked, is estimated to be located at the current time t by reading out history information on the target object that is being tracked from the storage unit642and using the read-out history information. To this end, information on the tracking box TB may be stored in the storage unit642, or may be generated in the correlation index calculation unit644using the history information stored in the storage unit642.

The tracking representative point is a representative point of the tracking box TB at the current time t. For example, the tracking representative point may include a representative point located at the periphery (or the edge) of the tracking box TB (hereinafter referred to as a “second peripheral representative point”) and a representative point located at the center of the tracking box TB (hereinafter referred to as a “second central representative point”). As shown inFIG.6B, reference numerals P0, P1, P2and P3are assigned to the second peripheral representative points of the tracking box TB at the current time t, in the clockwise direction from the lower left-hand corner thereof, and reference numeral Pcis assigned to the second central representative point located at the center thereof.

The previous representative point is a representative point of the segment box Bt-1selected as the associated segment box at a time t−1 prior to the current time t. For example, the previous representative point may include a representative point located at the periphery of the associated segment box Bt-1selected previously (hereinafter referred to as a “third peripheral representative point”). As shown inFIG.6B, reference numerals Pt-1m0, Pt-1m1, Pt-1m2and Pt-1m3are assigned to the third peripheral representative points of the segment box Bt-1selected as the associated segment box at a time t−1 prior to the current time t, in the clockwise direction from the lower left-hand corner thereof.

FIG.7is a block diagram of an embodiment644A of the correlation index calculation unit644and an embodiment646A of the candidate selection unit646shown inFIG.5.

The correlation index calculation unit644A may include first, second and third index calculation units720,730and740. In addition, the correlation index calculation unit644A may further include an overlap determination unit710.

The first index calculation unit720may calculate a first correlation index between the first peripheral representative point and the second peripheral representative point of the segment box Btat the current time t, which is output from the shape analysis unit630and is provided thereto through the input terminal IN1, for example, using Equation 1 below, and may output the calculated first correlation index to the candidate selection unit646or646A.

Here, γ1represents the first correlation index, xm0represents the horizontal-axis coordinate of the first peripheral representative point, ym0represents the vertical-axis coordinate of the first peripheral representative point, x0represents the horizontal-axis coordinate of the second peripheral representative point, y0represents the vertical-axis coordinate of the second peripheral representative point, σ2xmrepresents the horizontal-axis variance value of the first peripheral representative point, σ2ymrepresents the vertical-axis variance value of the first peripheral representative point, σ2xrepresents the horizontal-axis variance value of the second peripheral representative point, and σ2yrepresents the vertical-axis variance value of the second peripheral representative point.

To this end, the first index calculation unit720may receive the second peripheral representative point from the storage unit642through the input terminal IN2, or may generate the same using the history information received from the storage unit642through the input terminal IN2.

The second index calculation unit730may calculate a second correlation index between the first central representative point Pmcof the segment box Btat the current time t, which is output from the shape analysis unit630and is provided thereto through the input terminal IN1, and the second central representative point Pcof the tracking box TB, for example, using Equation 2 below, and may output the calculated second correlation index to the candidate selection unit646or646A.

Here, γ2represents the second correlation index, xmcrepresents the horizontal-axis coordinate of the first central representative point Pmc, ymcrepresents the vertical-axis coordinate of the first central representative point Pmc, σ2xmcrepresents the horizontal-axis variance value of the first central representative point Pmc, σ2ymcrepresents the vertical-axis variance value of the first central representative point Pmc, xcrepresents the horizontal-axis coordinate of the second central representative point, and ycrepresents the vertical-axis coordinate of the second central representative point.

To this end, the second index calculation unit730may receive the second central representative point from the storage unit642through the input terminal IN2, or may generate the same using the history information received from the storage unit642through the input terminal IN2.

The third index calculation unit740may calculate a third correlation index between the first peripheral representative point of the segment box Btat the current time t, which is output from the shape analysis unit630and is provided thereto through the input terminal IN1, and the third peripheral representative point, which is provided thereto from the storage unit642through the input terminal IN2, for example, using Equation 3 below, and may output the calculated third correlation index to the candidate selection unit646or646A.

Here, γ3represents the third correlation index, xtm0represents the horizontal-axis coordinate of the first peripheral representative point at the current time t, and ytm0represents the vertical-axis coordinate of the first peripheral representative point at the current time t. In Equations 1 and 2 set forth above, the coordinates xm0, ym0, x0and y0, to which “t” is not added as a superscript, are coordinates at the current time t. That is, xm0in Equation 1 is the same as xtm0in Equation 3, and ym0in Equation 1 is the same as ytm0in Equation 3. However, in Equation 3, xm0is expressed as xtm0, and ym0is expressed as ytm0in order to distinguish between the current time t and the previous time t−1. Furthermore, xt-1m0represents the horizontal-axis coordinate of the third peripheral representative point, and yt-1m0represents the vertical-axis coordinate of the third peripheral representative point.

After step410, the candidate selection unit646or646A may select candidates for the associated segment box among the plurality of segment boxes at the current time t using the correlation indices calculated in the correlation index calculation unit644, i.e. the first to third correlation indices, and may output information on the selected candidate segment boxes to the final selection unit648(step420).

FIG.8is a flowchart of an embodiment420A of step420shown inFIG.4.

For example, the candidate selection unit646or646A may include first to third comparison units810,820and830and a box selection unit840, as shown inFIG.7.

After step410, the first comparison unit810compares the first correlation index γ1for each of the plurality of segment boxes at the current time t with a first critical index C1. That is, the first comparison unit810determines whether the first correlation index γ1is less than the first critical index C1, as expressed using Equation 4 below, and outputs the result of the determination to the box selection unit840(step421).
γ1C1  [Equation 4]

When the first correlation index γ1is determined to be less than the first critical index C1 based on the result of comparison by the first comparison unit810, the box selection unit840selects, among the plurality of segment boxes at the current time t, a segment box, in which the first correlation index γ1is less than the first critical index C1, as a candidate segment box for the associated segment box, and outputs information on the selected candidate segment box to the final selection unit648(step428).

In addition, the second comparison unit820compares the second correlation index γ2for each of the plurality of segment boxes at the current time t with a second critical index C2. That is, the second comparison unit820determines whether the second correlation index γ2is less than the second critical index C2, as expressed using Equation 5 below, and outputs the result of the determination to the box selection unit840(step422).
γ2C2  [Equation 5]

When the second correlation index γ2is determined to be less than the second critical index C2 based on the result of the comparison by the second comparison unit820, the box selection unit840selects, among the plurality of segment boxes at the current time t, a segment box, in which the second correlation index γ2is less than the second critical index C2, as a candidate segment box for the associated segment box, and outputs information on the selected candidate segment box to the final selection unit648(step428).

In addition, the third comparison unit830compares the third correlation index γ3for each of the plurality of segment boxes at the current time t with a third critical index C3. That is, the third comparison unit830determines whether the third correlation index γ3is less than the third critical index C3, as expressed using Equation 6 below, and outputs the result of the determination to the box selection unit840(step423).
γ3C3  [Equation 6]

In Equations 4, 5 and 6, the first, second and third critical indices C1, C2 and C3 may be set in advance.

When the third correlation index γ3is determined to be less than the third critical index C3 based on the result of the comparison by the third comparison unit830, the box selection unit840selects, among the plurality of segment boxes at the current time t, a segment box, in which the third correlation index γ3is less than the third critical index C3, as a candidate segment box for the associated segment box, and outputs information on the selected candidate segment box to the final selection unit648(step428).

Although it is illustrated inFIG.8that, when step421is not satisfied, the process goes to step422, and when step422is not satisfied, the process goes to step423, the embodiments are not limited thereto. That is, according to another embodiment, step421, step422, and step423may be performed in any order. According to still another embodiment, step421, step422, and step423may be performed simultaneously. In the case in which step421, step422, and step423are performed simultaneously, the candidate selection unit646A shown inFIG.7corresponds to the embodiment of the candidate selection unit646.

The box selection unit840may select a candidate segment box for the associated segment box at the current time t in response to the results of comparison by the first to third comparison units810,820and830. As such, after step410, among a plurality of segment boxes, a segment box that satisfies at least one of the three conditions indicated in Equations 4, 5 and 6 may be selected as a candidate segment box for the associated segment box at the current time t.

In addition, the candidate selection unit646A may further include an overlap determination unit710, as shown inFIG.7.

After steps421,422and423, it is determined whether, among a plurality of segment boxes, a segment box that does not satisfy any of the three conditions indicated in Equations 4, 5 and 6 overlaps the tracking box TB (step424). If there is a segment box overlapping the tracking box TB, the segment box overlapping the tracking box TB may be selected as a candidate segment box (step428).

FIG.9is a diagram showing an example in which the segment box Btoverlaps the tracking box TB at the current time t.

Step424and step428may be performed by the box selection unit840and the overlap determination unit710.

The box selection unit840may generate a control signal CS in response to the results of the comparison by the first to third comparison units810,820and830. When it is determined that there is a segment box that is not selected as the candidate segment box as a result of detecting the plurality of segment boxes using the correlation indices in response to the control signal CS, the overlap determination unit710may determine whether this segment box Btoverlaps the tracking box TB, and may output the result of the determination to the box selection unit840(step424). For example, as shown inFIG.9, the segment box Btand the tracking box TB may overlap each other.

The box selection unit840may select a candidate segment box in response to the result of the determination by the overlap determination unit710. That is, when the segment box Btis determined to overlap the tracking box TB based on the result of the determination by the overlap determination unit710, the box selection unit840may select the segment box Btthat overlaps the tracking box TB as a candidate segment box (step428).

FIGS.10A to10Iare diagrams showing various examples in which the segment box Btand the tracking box TB overlap each other.

The segment box Btand the tracking box TB may overlap each other at one point, as illustrated inFIGS.10A and10B. The segment box Btand the tracking box TB may overlap each other at two points, as illustrated inFIG.10C. The segment box Btand the tracking box TB may overlap each other at three points, as illustrated inFIG.10D. The segment box Btand the tracking box TB may overlap each other at four points, as illustrated inFIG.10E. The segment box Btand the tracking box TB may overlap each other at five points, as illustrated inFIG.10F. The segment box Btand the tracking box TB may overlap each other at six points, as illustrated inFIG.10G. The segment box Btand the tracking box TB may overlap each other at seven points, as illustrated inFIG.10H. The segment box Btand the tracking box TB may overlap each other at eight points, as illustrated inFIG.10I. Irrespective of the number of overlapping points, the segment box Bthaving a point overlapping the tracking box TB may be selected as a candidate segment box. Alternatively, according to another embodiment, only a segment box Btthat overlaps the tracking box TB at a predetermined number of points or more may be selected as a candidate segment box.

The above-described first to third correlation indices are factors related to distance, and may be obtained through Euclidean distance calculation, or may be obtained through Mahalanobis distance calculation, as shown in Equations 1 to 3 above.

After steps421,422and423, when, among a plurality of segment boxes at the current time t, a segment box that does not satisfy any of the three conditions indicated in Equations 4, 5 and 6 does not overlap the tracking box, whether steps421to424have been performed on all of the first to Nthsegment boxes included in the mthchannel may be determined (step426).

If steps421to424have not been performed on all of the first to Nthsegment boxes included in the mthchannel, steps421to424and428may be performed on the segment boxes that have not undergone steps421to424, as described above.

For example, step426may be performed by the box selection unit840. That is, whenever steps421to424are performed, the box selection unit840may count to determine which one of the “N” segment boxes undergoes steps421to424.

Referring again toFIGS.4and5, after step420, the final selection unit648may select an associated segment box at the current time t, among the candidate segment boxes selected by the candidate selection unit646, and may output information on the associated segment box selected thereby through an output terminal OUT1(step430).

FIG.11is a block diagram of an embodiment648A of the final selection unit648shown inFIG.5.

The final selection unit648A shown inFIG.11may include a score assignment unit910, a score calculation unit920, and a score comparison unit930.

The score assignment unit910may assign a first score SCORE1to the distance suitability of each candidate segment box, may assign a second score SCORE2to the reliability suitability of each candidate segment box, and may assign a third score SCORE3to the correlation between each candidate segment box and the associated segment box Bt-1selected previously. To this end, the score assignment unit910receives information on the candidate segment boxes from the candidate selection unit646or646A through an input terminal IN4.

With regard to the distance suitability according to an embodiment, the smallest one of the first to third correlation indices of each candidate segment box may be compared with a first threshold value TV1in order to determine the distance suitability. That is, among the candidate segment boxes, the first score SCORE1may be assigned to a candidate segment box in which the smallest one of the first to third correlation indices is equal to or less than the first threshold value TV1.

With regard to the reliability suitability according to an embodiment, the reliability suitability may be determined using the reliability level of each candidate segment box. That is, among the candidate segment boxes, the second score SCORE2may be assigned to a candidate segment box having a reliability level greater than a second threshold value TV2.

For example, the step of obtaining the reliability level of each candidate segment box may include a step of determining a shape reliability level, a step of determining an attribute reliability level, and a step of determining a geometric reliability level.

In the step of determining a shape reliability level, a 2-1streliability level corresponding to the shape of the candidate segment box, e.g. the size thereof, may be obtained. In the step of determining an attribute reliability level, a 2-2ndreliability level corresponding to at least one of the ratio of the number of points included in the candidate segment box to the size of the candidate segment box or the degree of dispersion of points included in the candidate segment box may be obtained. In the step of determining a geometric reliability level, a 2-3rdreliability level corresponding to the position of the candidate segment box and the distance from a reference point to the candidate segment box may be obtained. For example, in the case in which the LiDAR sensor500and the object-tracking device600are mounted to the vehicle1000, the reference point may be the position of the vehicle1000. That is, the 2-3rdreliability level may be obtained based on the distance from the vehicle1000to the candidate segment box and on whether the candidate segment box is located at the boundary of the field of view of the LiDAR sensor500. The 2-1st, 2-2ndand 2-3rdreliability levels may be set in advance.

Thereafter, the 2-1st, 2-2ndand 2-3rdreliability levels may be summed to determine a reliability level to be compared with the second threshold value TV2.

With regard to the correlation according to an embodiment, the correlation may be determined using the ratio of the area of each candidate segment box that overlaps the associated segment box selected previously to the entire area of each candidate segment box. Specifically, the third score may be assigned to the correlation in proportion to the ratio of the area of each candidate segment box that overlaps the previously selected associated segment box to the entire area of each candidate segment box. That is, the higher the ratio, the higher the third score that may be assigned.

The score calculation unit920may sum the first to third scores SCORE1, SCORE2and SCORE3assigned to each candidate segment box to calculate a final score TSCORE.

The score comparison unit930may select, among the candidate segment boxes, the candidate segment box having the highest final score TSCORE as an associated segment box at the current time t, and may output the selected associated segment box through the output terminal OUT1.

FIG.12is a flowchart for explaining an object-tracking method440using the LiDAR sensor500according to another embodiment.

First, it is determined whether there is an associated segment box at the current time t (step442). Step442may be performed after step430, or may be performed at step420. If it is determined at step420that there is no candidate segment box, it may be determined that there is no associated segment box at the current time t. Alternatively, when an associated segment box at the current time t is selected at step430, it may be determined that there is an associated segment box at the current time t.

If there is an associated segment box, the history information of the mthchannel to which the associated segment box belongs may be updated (step444). For example, the final selection unit648may output information on the associated segment box at the current time t for the target object that is being tracked in the mthchannel to the storage unit642through the output terminal OUT1to update the history information of the mthchannel.

However, when there is no associated segment box, the history information of the mthchannel in which there is no associated segment box may be deleted (step446). For example, when it is determined that none of the “N” segment boxes which belong to the mthchannel is selected as a candidate segment box, the box selection unit840shown inFIG.7may output a deletion request signal to the storage unit642through an output terminal OUT3, and the storage unit642may delete the history information of the mthchannel in response to the deletion request signal.

In addition, when it is determined that at least one of the “N” segment boxes which belong to the mthchannel is a segment box associated with a new target object, the object-tracking unit640may store the information on the associated segment box for the new target object in an empty channel of the storage unit642, if any. In this way, the object-tracking unit640may distinguish between the target object that is currently being tracked and the newly recognized target object, and may assign a new identification (ID) number to the new target object. When the target object that is being tracked disappears, the object-tracking unit640may retrieve the identification (ID) number assigned to the disappeared target object.

In addition, the object-tracking unit640shown inFIG.2may determine whether the target object is an obstacle, a vehicle, or a person using the information on the associated segment box.

Referring toFIG.2, for example, when the object-tracking method and device600described above are used in the vehicle1000, the vehicle1000may further include a vehicle device700. The vehicle device700may control the vehicle1000based on the information on the target object tracked by the object-tracking unit640and the type of target object.

The vehicle device700may control the vehicle1000based on the determined information on an object, received from the object-tracking device600. For example, the vehicle device700may include a lane-keeping assist system for preventing a vehicle from deviating from a lane while maintaining the distance to a preceding vehicle, an obstacle detection system for detecting obstacles present around a vehicle, a collision prevention system for detecting the risk of a collision, an autonomous driving system for controlling a vehicle to travel autonomously while detecting obstacles present ahead of the vehicle, and a safe driving system for warning of the approach of another vehicle adjacent to the host vehicle and for controlling the host vehicle to assist in safe driving of the host vehicle.

The LiDAR sensor500may be located at at least one of the front side, lateral sides, or rear side of the vehicle1000. The object-tracking method and device600and the vehicle1000including the same according to the embodiments are not limited as to the specific position at which the LiDAR sensor500is mounted in the vehicle1000.

A recording medium in which a program for executing the method of tracking an object using the LiDAR sensor500is recorded may store a program for implementing a clustering function of grouping LiDAR data composed of a plurality of points for an object obtained by the LiDAR sensor500, a function of generating information on a plurality of segment boxes for each channel using the result of clustering, and a function of selecting, among the plurality of segment boxes, an associated segment box at the current time t for the target object that is being tracked. The function of selecting an associated segment box at the current time t may include a function of calculating a correlation index between the current representative point and each of the tracking representative point and the previous representative point of each of the plurality of segment boxes at the current time t, a function of selecting, among the plurality of segment boxes, candidates for an associated segment box using the correlation index, and a function of selecting, among the selected candidates, an associated segment box at the current time t. The recording medium may be read by a computer system.

The computer-readable recording medium includes all kinds of recording devices in which data that may be read by a computer system are stored. Examples of the computer-readable recording medium include a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disk ROM (CD-ROM), a magnetic tape, a floppy disc, and an optical data storage. The computer-readable recording medium can also be distributed over network-connected computer systems so that the computer-readable code is stored and executed in a distributed fashion. Also, functional programs, code, and code segments for accomplishing the object-tracking method can be easily construed by programmers skilled in the art to which the present disclosure pertains.

Hereinafter, an object-tacking method according to a comparative example and the object-tracking method according to the embodiment will be described with reference to the accompanying drawings.

In first and second comparative examples, in which “association” is performed, it may be determined whether there is a correlation between a predicted tracking box and a plurality of segment boxes of a target object that is being tracked, and information on the segment box having a correlation may be selected. For example, the presence or absence of the correlation is determined through comparison between the points of boxes. The correlation between the tracking box and the segment box may be determined using an Euclidean distance or a Mahalanobis distance. When the determined distance satisfies an allowable boundary value, it may be determined that there is a correlation.

In the case of the first comparative example, the center of the segment box is used as a representative point. In this case, since the information on the segment box does not accurately indicate the heading of the segment box, the first comparative example is robust to heading error. However, due to the characteristics of the LiDAR sensor500, when the size of the segment box changes due to determining the visibility and the shape of the object, the changed size of the segment box may be reflected in the error.

FIG.13Ais a diagram showing a target object10and a guardrail12that are being tracked using the LiDAR sensor500, andFIG.13Bis a diagram for explaining the process of tracking the target object10according to the second comparative example.

FIG.14Ais a diagram of a tracking box, in which an arrow indicates the heading direction thereof,FIG.14Bis a diagram showing two exemplary candidate segment boxes CB1and CB2selected by the second comparative example, andFIG.14Cis a diagram showing a segment box estimated using history information updated using an associated segment box incorrectly selected by the second comparative example.

In the case of the second comparative example, the center of the rear side of the segment box is used as a representative point. For example, according to the second comparative example, the centers of the rear sides of the candidate segment boxes CB1and CB2for association are used as representative points RP1and RP2. Since the density of the point cloud is high at the center of the rear side of the segment box with respect to the mounting position of the LiDAR sensor500, the second comparative example is robust to a change in the size of the segment box according to the shape of a target object, thereby stably providing the position of the measured value in the longitudinal direction. However, since the second comparative example is incapable of accurately recognizing the heading when generating information on the segment box, there is a problem in that the reference of the rear side is changed (e.g. a problem in that the width of the segment box and the length thereof are switched to each other), thus incurring a large error in the position of the measured value. In this way, according to the second comparative example, in which “association” is performed on the basis of the rear side of the segment box, when the correlation is determined using the distance, tracking loss may occur due to incorrect association.

In the curved section shown inFIG.13A, the guardrail12having a curved shape is recognized as a large segment box CB2. However, this segment box CB2is not a segment box which is substantially necessary in order to maintain tracking of the target object10that is being tracked. Nevertheless, if the unnecessary segment box CB2is selected as an associated segment box and history information is updated using the same, it can be seen fromFIG.14Cthat, when a tracking box is predicted based on the updated history information, the heading indicated by the arrow is not aligned with the actual heading of the target object10.

FIG.15is a diagram showing the result of tracking when “association” is performed by the object-tracking method and device according to the embodiment.

According to the embodiment, among the plurality of segment boxes CB1and CB2, the optimal associated segment box CB1, which matches the target object10that is being tracked, is selected, and the history information of the target object10is updated using the same. In this case, it can be seen fromFIG.15that, when the tracking box CSB of the target object10is generated using the updated history information, it is possible to minimize errors in the heading, position, and shape of the target object10. In particular, according to the embodiment, a candidate segment box is primarily selected using a factor related to distance. At this time, whether a segment box that has not been primarily selected as a candidate segment box overlaps the tracking box TB, i.e. is in surface contact with the tracking box TB, is determined in order to secondarily select a candidate segment box. Thus, it is possible to prevent an associated segment box from being incorrectly selected due to selection of a candidate segment box using only the distance factor.

As is apparent from the above description, a method and device for tracking an object using a LiDAR sensor, a vehicle including the device, and a recording medium storing a program to execute the method according to embodiments are capable of tracking a target object using a LiDAR sensor without errors or with minimized errors.

However, the effects achievable through the disclosure are not limited to the above-mentioned effect, and other effects not mentioned herein will be clearly understood by those skilled in the art from the above description.

The above-described various embodiments may be combined with each other without departing from the objects of the present disclosure unless they are incompatible with each other. In addition, for any element that is not described in detail in any of the various embodiments, reference may be made to the description of an element having the same reference numeral in another embodiment.

While the present disclosure has been particularly shown and described with reference to exemplary embodiments thereof, these embodiments are only proposed for illustrative purposes and do not restrict the present disclosure, and it will be apparent to those skilled in the art that various changes in form and detail may be made without departing from the essential characteristics of the embodiments set forth herein. For example, respective configurations set forth in the embodiments may be modified and applied. Further, differences in such modifications and applications should be construed as falling within the scope of the present disclosure as defined by the appended claims.