Patent ID: 11915470
Assignee: QUANZHOU EQUIPMENT MANUFACTURING RESEARCH INSTITUTE
Field: Measurement (Instruments)
Classification: CPC G  Y | IPC G

Claim 0:
1. A target detection method based on fusion of vision, lidar, and millimeter wave radar, comprising:
(1) obtaining original data detected by each of a camera, a millimeter wave radar, and a lidar, and synchronizing the millimeter wave radar, the lidar, and the camera in time and space;
(2) performing a calculation on the original data detected by the millimeter wave radar according to a radar protocol;
(3) using a position based on the original data, which is detected by the millimeter wave radar and has been calculated, as a first anchor point, and generating a first region of interest, which is three-dimensional, according to a speed and a radar reflection area with the first anchor point as a center of the first region of interest;
(4) generating a second anchor point arranged according to a specified distance in a blind area in which radar points of the millimeter wave radar are not generated, and generating a second region of interest by traversing the second anchor point with the second anchor point as a center of the second region of interest;
(5) pre-processing the original data detected by the lidar to generate a point cloud bird's-eye view, and extracting a feature map of the point cloud bird's-eye view and a feature map of the original data detected by the camera;
(6) projecting the first region of interest and the second region of interest generated in step 3 and step 5, which are three-dimensional, onto the feature map of the point cloud bird's-eye view and the feature map of the original data detected by the camera; and
(7) fusing the first region of interest, the second region of interest, the feature map of the point cloud bird's-eye view, and the feature map of the original data detected by the camera, which have a same size, to form a fused image, processing the fused image through a fully connected layer, and outputting an image of a test result.