Patent ID: 11875524
Assignee: BEIHANG UNIVERSITY
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. An unmanned aerial vehicle platform based vision measurement method for a static rigid object, comprising the following steps:
step 1: determining a target object, planning a flight path, and mounting a camera on the unmanned aerial vehicle platform, wherein an unmanned aerial vehicle (UAV) has a global navigation satellite system (GNSS) and an inertial measurement unit (IMU);
step 2: operating the UAV for aerial photography according to the planned flight path, after the UAV returns, reading images, position information, and IMU data, and determining a corresponding relationship therebetween according to a timestamp;
step 3: performing object detection on each image successively, and recording the position of the target object in the image, wherein an appropriate object detection method should be selected according to target characteristics and task requirements, the position of the target object in the image is indicated by a detection frame, instance segmentation is further performed on the image, and a mask obtained by the segmentation is used to represent the position of the target object;
step 4: converting the UAV position information and the IMU data corresponding to each image into camera extrinsics:, R
    =
    
      
        
          [
          
            
              
                
                  cos
                  ⁢
                  
                    
                  
                  ⁢
                  γ
                
              
              
                
                  sin
                  ⁢
                  
                    
                  
                  ⁢
                  γ
                
              
              
                0
              
            
            
              
                
                  
                    -
                    sin
                  
                  ⁢
                  
                    
                  
                  ⁢
                  γ
                
              
              
                
                  cos
                  ⁢
                  
                    
                  
                  ⁢
                  γ
                
              
              
                0
              
            
            
              
                0
              
              
                0
              
              
                1
              
            
          
          ]
        
        ⁡
        
          [
          
            
              
                1
              
              
                0
              
              
                0
              
            
            
              
                0
              
              
                
                  cos
                  ⁢
                  
                    
                  
                  ⁢
                  β
                
              
              
                
                  sin
                  ⁢
                  
                    
                  
                  ⁢
                  β
                
              
            
            
              
                0
              
              
                
                  
                    -
                    sin
                  
                  ⁢
                  
                    
                  
                  ⁢
                  β
                
              
              
                
                  cos
                  ⁢
                  
                    
                  
                  ⁢
                  β
                
              
            
          
          ]
        
      
      ⁡
      
        [
        
          
            
              
                cos
                ⁢
                
                  
                
                ⁢
                α
              
            
            
              
                sin
                ⁢
                
                  
                
                ⁢
                α
              
            
            
              0
            
          
          
            
              
                
                  -
                  sin
                
                ⁢
                
                  
                
                ⁢
                α
              
            
            
              
                cos
                ⁢
                
                  
                
                ⁢
                α
              
            
            
              0
            
          
          
            
              0
            
            
              0
            
            
              1
            
          
        
        ]
      
    
  

  
    t
    =
    
      
        P
        1
      
      -
      
        P
        2
      
    
  

where α, β, and γ are rotation transformation Euler angles between a UAV body coordinate system that is an IMU coordinate system corresponding to a current image and a local horizontal coordinate system that is a North-East-Down (NED) coordinate system, which are measured by the IMU; P1 is initial position coordinates of the UAV in the NED coordinate system, which is provided by the GNSS; P2 is UAV position coordinates corresponding to a current image in the NED coordinate system, which can be provided directly by the GNSS or can also be calculated from the IMU data in combination with P1, and combining P2 obtained by two different channels can improve the accuracy of the result; t calculated by P1−P2 is a translation corresponding to the current image of the UAV; and with R used as a rotation matrix of the camera, and t used as a translation vector of the camera, the camera extrinsics can be obtained;
step 5: performing three-dimensional reconstruction by combining the result of the object detection and the calculated camera extrinsics;
(1) extracting features of each image successively, and removing parts not belonging to the target object;
(2) screening the images to remove images with feature numbers less than a threshold T1, and generating feature descriptors for the remaining images;
(3) performing feature matching between the images, and removing image pairs with matching numbers less than a threshold T2;
(4) performing three-dimensional reconstruction with the remaining image pairs to obtain a three-dimensional point cloud, wherein in the reconstruction process, the camera extrinsics calculated in the step 4 are used;
step 6: performing three-dimensional point cloud processing on the reconstruction result;
(1) performing point cloud preprocessing to remove noise in the three-dimensional point cloud;
(2) performing point cloud segmentation to further remove noise, wherein a target object point cloud is obtained by the segmentation at the same time;
step 7: modeling and analyzing the target object point cloud, marking a size to be measured and performing measurement;
for an object with a simple shape, establishing multiple geometric body models, using these geometric bodies to fit the target object, fitting a size to be measured with side lengths and radius parameters of the geometric bodies, and automatically marking the position of the size to be measured through a computer, obtaining its specific values at the same time; and
for an object with a more complex shape, performing up-sampling and surface fitting on a target object point cloud to ensure the integrity of the target object, then checking a three-dimensional model of the target object, and manually marking a size to be measured on the computer and obtaining its specific values at the same time.