Patent ID: 11948368
Assignee: CHONGQING UNIVERSITY
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 1:
2. The real-time object detection and 3D localization method based on a single frame image according to claim 1, wherein in step S5, a regularization algorithm for maintaining horizontal geometric locality on the image is used to constrain prediction of 3D horizontal information and increase detection accuracy of an overall 3D-box, which comprises the following steps:
S51: designing assumption for maintaining horizontal geometric locality as a regularization term in a loss function of the center point of the 3D-box, and assuming that M target samples exist in the image; a matrix S={sij} is defined as an M×M adjacent matrix as formula (1):

sij=exp[−(ui(2d)−uj(2d))2]/exp[(zi(3d)−zj(3d))2/λ]  (1), wherein sij represents a horizontal adjacency measure between two targets with similar depths, ui(2d) and uj(2d) are horizontal pixel offsets of object i and object j, and zi(3d) is the ground-truth depth information of object i;
S52: applying the regularization term to a fully connected layer of a neural network predicted at the center point of the 3D-box; assuming the output of object i in this layer is yi=Wxi+b, wherein xi represents an input of the fully connected layer, W is a connection weight, and b is the deviation vector; assuming that training targets are adjacent in 3D depth and 2D horizontal directions, the whole network will try to estimate the best connection weight W so that target pairs are close to each other in 3D space; and the regularization term R(W) is defined as a feature difference of an adjacent target pair, and an expression is shown as formula (2):, R
      ⁡
      (
      W
      )
     
     =
     
      
       β
       2
      
      ⁢
      
       
        ∑
        ij
       
       
        
         
          
          
           
            Wx
            i
           
           -
           
            Wx
            j
           
          
          
         
         2
         2
        
        ⁢
        
         s
         ij
        
       
      
     
    
   
   
    
     (
     2
     )
    
   
  
 

wherein β is a balance coefficient; the greater the adjacent degree of an i,j sample pair is, the greater the adjacency measure sij will be, and the faster sij can reduce distance between Wxi and Wxj in the process of minimizing an overall loss function so as to maintain adjacency of target pairs from 2D space to 3D space; R(W) is added to an overall loss function of a single frame image 3D target detection network, and a final overall loss function L of the network is expressed as:

L=L2d+Ldepth+L3d+R(W)

a related loss function is defined by errors of an L1 or L2 loss function;
wherein L2d represents sum of a loss function of target confidence and a loss function of a 2D-box in a 2D object detection loss function;
Ldepth represents a final depth information loss function formed by that a depth information loss function uses the L1 loss function to calculate depth loss functions of a deep neural network and a shallow neural network respectively, and uses a weight W to link the two loss functions;
L3d represents a loss function dividing a 3D loss function into the 3D-box and the center point thereof, both of which are represented by the L1 loss function.