IDENTIFICATION OF EDGE POINTS AND PLANAR POINTS IN POINT CLOUD OBTAINED BY VEHICLE LIDAR SYSTEM

A system in a vehicle includes a lidar system to transmit incident light and receive reflections from one or more objects as a point cloud of points. The system also includes processing circuitry to identify feature points among the points of the point cloud using principal component analysis. The feature points are planar points that form one or more surfaces or edge points. A set of edge points forms a linear surface.

The subject disclosure relates to the identification of edge points and planar points in a point cloud obtained by a vehicle lidar system.

Vehicles (e.g., automobiles, trucks, construction equipment, farm equipment) increasingly include sensors that obtain information about the vehicle and its environment. The information facilitates semi-autonomous or autonomous operation of the vehicle. For example, sensors (e.g., camera, radar system, lidar system, inertial measurement unit (IMU), steering angle sensor) may facilitate semi-autonomous maneuvers such as automatic braking, collision avoidance, or adaptive cruise control. A lidar system obtains a point cloud that must be processed to obtain information that would facilitate control of vehicle operation. Accordingly, it is desirable to provide the identification of edge points and planar points in the point cloud obtained by a vehicle lidar system.

SUMMARY

In one exemplary embodiment, a system in a vehicle includes a lidar system to transmit incident light and receive reflections from one or more objects as a point cloud of points. The system also includes processing circuitry to identify feature points among the points of the point cloud using principal component analysis, the feature points being planar points that form one or more surfaces or edge points. A set of edge points forms a linear surface.

In addition to one or more of the features described herein, the lidar system is a beam-based lidar system that transmits each beam of incident light across a horizontal scan line.

In addition to one or more of the features described herein, the lidar system is a non-beam-based lidar system that transmits each beam of incident light over an area.

In addition to one or more of the features described herein, the processing circuitry identifies neighbor points of each point in the point cloud. The neighbor points are all points of the point cloud that are within a threshold distance of the point.

In addition to one or more of the features described herein, the processing circuitry uses principal component analysis by calculating eigenvalues [λi0, λi1, λi2] of a covariance matrix of the neighbor points of each point, where λi0>λi1>λi2.

In addition to one or more of the features described herein, eigenvectors corresponding to the eigenvalues λi0, λi1, and λi2are not limited to a particular orientation.

In addition to one or more of the features described herein, the processing circuitry identifies the point as an edge point, a planar point, or neither the edge point or the planar point alone based on comparing the eigenvalues to threshold values.

In addition to one or more of the features described herein, the processing circuitry identifies the point as the edge point based on the eigenvalue λi0exceeding a first threshold while the eigenvalues λi1and λi2are less than a second threshold.

In addition to one or more of the features described herein, the processing circuitry identifies the point as the planar point based on the eigenvalues λi0and λi1exceeding a third threshold while the eigenvalue λi2is less than a fourth threshold.

In addition to one or more of the features described herein, the processing circuitry identifies an object based on the feature points.

In another exemplary embodiment, a method in a vehicle includes obtaining, at processing circuitry from a lidar system configured to transmit incident light and receive reflections from one or more objects, a point cloud of points. The  method also includes identifying, by the processing circuitry, feature points among the points of the point cloud using principal component analysis, the feature points being planar points that form one or more surfaces or edge points. A set of edge points forms a linear surface.

In addition to one or more of the features described herein, the obtaining the point cloud is from a beam-based lidar system that transmits each beam of incident light across a horizontal scan line.

In addition to one or more of the features described herein, the obtaining the point cloud is from a non-beam-based lidar system that transmits each beam of incident light over an area.

In addition to one or more of the features described herein, the method also includes the processing circuitry identifying neighbor points of each point in the point cloud, wherein the neighbor points are all points of the point cloud that are within a threshold distance of the point.

In addition to one or more of the features described herein, the method also includes the processing circuitry using principal component analysis by calculating eigenvalues [λi0, λi1, λi2] of a covariance matrix of the neighbor points of each point, where λi0>λi1>λi2.

In addition to one or more of the features described herein, eigenvectors corresponding to the eigenvalues λi0, λi1, and λi2are not limited to a particular orientation.

In addition to one or more of the features described herein, the method also includes the processing circuitry comparing the eigenvalues to threshold values to identify the point as an edge point, a planar point, or neither the edge point or the planar point alone.

In addition to one or more of the features described herein, the method also includes the processing circuitry identifying the point as the edge point based on  the eigenvalue λi0exceeding a first threshold while the eigenvalues λi1, and λi2are less than a second threshold.

In addition to one or more of the features described herein, the method also includes the processing circuitry identifying the point as the planar point based on the eigenvalues λi0and λi1exceeding a third threshold while the eigenvalue λi2is less than a fourth threshold.

In addition to one or more of the features described herein, the method also includes the processing circuitry identifying an object based on the feature points.

DETAILED DESCRIPTION

As previously noted, a point cloud obtained with a lidar system must be processed in order to obtain information about detected objects. The process is referred to as feature extraction. More specifically, feature extraction refers to the identification of features such as edges and planes within the point cloud. The identification of these edges and planes facilitates the identification of objects in the scene. A beam-based point cloud refers to one that is made up of multiple horizontal scan lines corresponding to multiple beams of the light source (e.g., laser) that are transmitted to obtain the point cloud as reflections. That is, each scan line corresponds to a transmitted beam. The vertical resolution of a beam-based point cloud is limited by how close the transmitted beams and, consequently, how close the scan lines are to each other. Thus, another type of point cloud that may be obtained is a non-beam-based point cloud. A non-beam-based point cloud may refer, for example, to a point cloud formed as a patch (e.g., cube) per beam. Such a point cloud does not include the horizontal scan lines that define a beam-based point cloud.

Prior feature extraction techniques (e.g., laser odometry and mapping (LOAM)) are well-suited to beam-based point clouds but rely on the horizontal scan lines and, thus, are unsuited for non-beam-based point clouds. Embodiments of the systems and methods detailed herein relate to the identification of edge points and planar points in a point cloud obtained with a vehicle lidar system. Generally, a set of  edge points may indicate a linear surface (e.g., long and narrow) (e.g., of a tree, lamp post) while a set of planar points may indicate a surface area (e.g., of a road, broadside of a car). A grouping or set of neighbor points is identified. Principal component analysis (PCA) (i.e., an eigen-decomposition of the covariance matrix of the neighbor points) is used to identify edge points and planar points, as detailed.

In accordance with an exemplary embodiment,FIG.1is a block diagram of a vehicle100that implements identification of edge points E and planar points P in a lidar point cloud205(FIG.2) obtained with a lidar system110. The exemplary vehicle100shown inFIG.1is an automobile101. The lidar system110may be beam-based or non-beam-based, as illustrated inFIGS.3A and3B. The lidar system110includes a lidar controller115. The vehicle100includes additional sensors120(e.g., radar system, camera, IMU) and a controller130. The controller130may obtain information from the lidar system110and other sensors120and may control semi-autonomous or autonomous operation of the vehicle100. The numbers and locations of the lidar system110and other sensors120are not intended to be limited by the exemplary illustration inFIG.1.

The feature extraction processes (i.e., processes to identify edge points E and planar points P among the point cloud205) discussed for the lidar system110may be performed by the lidar controller115, controller130, or a combination of the two. The lidar controller115and controller130may include processing circuitry that may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and memory that executes one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that provide the described functionality.

The lidar system110transmits incident light and receives reflected light. The reflected light is a result of reflection of the incident light by different parts of the objects140in the field of view of the lidar system110. The reflected light is in the form of points pithat form a point cloud205(FIG.2). In order to identify and locate objects140within the point cloud205, the point cloud205must be processed. Specifically, feature extraction may be performed, as discussed with reference to FIG.2. InFIG.1, three exemplary objects140a,140b,140c(generally referred to as140) in the field of view of the lidar system110are shown. The object140ais a road surface and the object140bis a hedge row. Both of these objects140may reflect incident light from the lidar system110to generate planar points P in a point cloud205, as indicated. The object140cis a light pole and may reflect incident light from the lidar system110to generate a set of edge points E in the point cloud205, as indicated.

As noted, the feature extraction results in the identification of points pithat are edge points E or planar points P. The points pithat are planar points P may be part of a horizontal plane h or a vertical plane v. An exemplary horizontal plane h from which points piin the point cloud205may be obtained is illustrated by the road surface (object140a). Similarly, an exemplary vertical plane v from which points piin the point cloud205may be obtained is illustrated by the hedge row140b. As shown inFIG.2, planar points P need not be limited to only the vertical plane v or the horizontal plane h. When the point cloud205is obtained, performing feature extraction may help to identify the object140that reflects incident light at the points p, in the point cloud205.

FIG.2is a process flow of a method200of identifying edge points E and planar points P in a point cloud205obtained with a lidar system110of a vehicle100according to one or more embodiments. At block210, the processes include selecting a set of neighbor points Njfor each point piof the point cloud205, as indicated. The neighbor points Njmay be identified as those within a threshold distance of the pointp for example. An exemplary point pxis shown at the center of a circle206that represents the threshold distance such that the subset of all the points piof the point cloud205that are within that circle206are neighbor points Njof the exemplary point px.

At block220, the processes include performing principal component analysis (PCA). Specifically, the processes include calculating eigenvalues [λi0, λi1, λi2] of a covariance matrix of neighbor points Ni. In PCA, eigenvalues [λi0, λi1, λi2] represent the variance in magnitude in the direction of the largest spread of the  neighbor points N. The eigenvalues [λi0, λi1, λi2] are in order of decreasing magnitude (i.e., λi0>λi1>λi2) and may pertain to any orientation. That is, for example, based on the specific set of neighbor points Nj, the eigenvalue λi0(i.e., the one with the highest magnitude) may pertain to the x, y, or z axis, whichever represents the direction of largest spread. This is further discussed with reference toFIGS.4A,4B, and4C. At block230, a check is done of whether all of the following are true: λi0(the highest eigenvalue)>threshold1 and both λi1and λi2<threshold2. If the check at block230indicates that the conditions are met, then the point piis identified as an edge point E at block235. An exemplary set of edge points E is shown. If the check at block230indicates that at least one of the conditions is not met, then the check at block240is completed.

At block240, a check is done of whether all of the following are true: λi0and λi1(the highest and second highest eigenvalues)>threshold3 and λi2(the lowest eigenvalue)<threshold4. If the check at block240indicates that all the conditions are met, then the point piis identified as a planar point P at block245. An exemplary set of edge points P is shown. If the check at block240indicates that not only the conditions checked at block230but also the conditions checked at block240are not met, then the point piis designated, at block250, as neither an edge point E nor a planar point P alone. The point pimay be both an edge point E and a planar point P or neither. The point pimay be an edge between two planes formed in different directions, as another example of a point pithat may be designated at block250. As previously noted, identifying the planar points P and the edge points E may facilitate identifying the object140that reflected light from the lidar system110to provide the point cloud205.

FIG.3Aillustrates an exemplary beam-based lidar system110aandFIG.3Billustrates an exemplary non-beam-based lidar system110b. Each of the lidar systems110a,110bis shown with an object140(e.g., wall) in its field of view. As shown inFIG.3A, each beam310athrough310n(generally referred to as310) results in a horizontal scan line320athrough320n(generally referred to as320). Thus, the point cloud400formed from reflections of the scan lines320would also be in the form of lines with separation in the vertical dimension that corresponds with a  separation between adjacent beams310. As previously noted, this limits the vertical resolution of the beam-based lidar system110aaccording to how closely the beams310are spaced.

As shown inFIG.3B, each beam310results in an area330athrough330n(generally referred to as330) or a patch that is scanned by the beam. Thus, in the non-beam-based lidar system110b, a horizontal scan is not accomplished by each beam310individually, as it is in the beam-based lidar system110a. As previously noted, prior feature extraction techniques rely on the horizontal scan lines320of a beam-based lidar system110a. According to one or more embodiments, the processes discussed with reference toFIG.3are applicable to both beam-based and non-beam based lidar systems110.

FIGS.4A,4B, and4Cillustrate eigenvectors resulting from the PCA performed at block220.FIG.4Aillustrates eigenvectors corresponding with eigenvalues [λi0, λi1, λi2] obtained from neighbor points Njfor an exemplary point pithat is an edge point E. As shown, and consistent with the check at block230, the eigenvector corresponding with the highest eigenvalue λi0is longer than the eigenvectors corresponding with the other two eigenvalues λi1and λi2. As previously noted, the eigenvalues [λi0, λi1, λi2] are arranged in order of magnitude. In the exemplary case shown inFIG.4A, the highest eigenvalue λi0corresponds with the z axis according to the exemplary coordinate system shown.

FIG.4Billustrates eigenvectors corresponding with eigenvalues [λi0, λi1, λi2] obtained from neighbor points Njfor an exemplary point pithat is a planar point P. As shown, and consistent with the check at block240, the eigenvectors corresponding with the highest two eigenvalues λi0and λi1are longer than the eigenvector corresponding with the smallest eigenvalue λi2. In the exemplary case shown inFIG.4B, the highest eigenvalue λi0corresponds with the x axis according to the exemplary coordinate system shown.

FIG.4Cillustrates eigenvectors corresponding with eigenvalues [λi0, λi1, λi2] obtained from neighbor points Njfor an exemplary point pithat is neither an  edge point E nor a planar point P alone. As shown, and consistent with failing the checks at blocks230and240, the eigenvectors corresponding with all of the eigenvalues [λi0, λi1, λi2] are large (e.g., all the eigenvalues [λi0, λi1, λi2] exceed threshold2 and threshold4, as checked at blocks230and240). The exemplary point pimay be both an edge point E and a planar point P or neither.