Patent ID: 11915470
Assignee: QUANZHOU EQUIPMENT MANUFACTURING RESEARCH INSTITUTE
Field: Measurement (Instruments)
Classification: CPC G  Y | IPC G

Claim 6:
7. The target detection method based on fusion of vision, lidar, and millimeter wave radar according to claim 1, wherein:
the step 6 comprises:
setting an index number for each of the first anchor point and the second anchor point;
projecting the first region of interest and the second region of interest onto the feature map of the point cloud bird's-eye view and the feature map of the original data detected by the camera;
determining a three-dimensional region of interest through a spatial synchronization of the original data detected by the millimeter wave radar and the lidar in step 1 since the original data detected by the millimeter wave radar and the lidar are both three-dimensional data; and
obtaining a vertex coordinate of the three-dimensional region of interest R=[x; y; z],

a conversion relationship of the three-dimensional region of interest is:, ⌈
    
     
      
       
        X
        p
       
      
     
     
      
       
        Y
        p
       
      
     
     
      
       Z
      
     
    
    ⌉
   
   =
   
    
     P
     [
     
      
       
        
         R
         c
        
       
       
        
         T
         c
        
       
      
      
       
        
         O
         T
        
       
       
        1
       
      
     
     ]
    
    [
    
     
      
       x
      
     
     
      
       y
      
     
     
      
       z
      
     
     
      
       1
      
     
    
    ]
   
  
  ,
 

(Xp, Yp) are coordinates of projection points in an image coordinate system,
P is a matrix of camera parameters,
Rc is a rotation matrix of the camera relative to an Inertial Measurement Unit (IMU),
Tc is a translation matrix of the camera relative to the IMU,
coordinate points of a 3D area in an image are obtained through the steps 1-7,
each of the vertex coordinates, which has been obtained, is adjusted according to a ground vector to obtained an adjusted coordinate point (X, Y, Z*),
the feature maps of the same size are fused to form the fused image,
the fused image is processed through the fully connected layer,
anchor points are filtered out, and
a size and direction of anchor point boxes are regressed.