Patent ID: 11900619
Assignee: nan
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 0:
1. A method for intelligently measuring vehicle trajectory based on a binocular stereo vision system, comprising:
step 1, inputting a dataset of images containing license plates into a SSD neural network, and training the SSD neural network with license plate as a detection feature to obtain a license plate recognition model;
step 2, installing the binocular stereo vision system on the right side, middle or above a lane, calibrating the binocular stereo vision system to acquire internal and external parameters of two cameras; and recording videos of moving target vehicles by the calibrated binocular stereo vision system;
step 3, detecting license plates in video frames recorded in step 2 with the license plate recognition model trained in step 1, and locating license plate positions of target vehicles;
step 4, performing feature point extraction and stereo matching on license plates in subsequent frames of a same camera by a feature-based matching algorithm, and reserving correct matching points after filtering with a homography matrix; performing feature point extraction and stereo matching on license plates in corresponding left-view and right-view video frames of the binocular stereo vision system by a feature-based matching algorithm, and reserving correct matching points after filtering with a homography matrix;
step 5, screening the reserved matching points in step 4 by a distance measurement method of the binocular stereo vision system, and reserving one closest to a license plate center as the position of a target vehicle in a current frame;
step 6, performing stereo measurement on the screened matching points by the binocular stereo vision system to get spatial position coordinates of vehicle in the video frames; and generating a moving trajectory of the vehicle in time sequence, wherein a calibration method for the binocular stereo vision system in the step 2 comprises: calibrating the two cameras with Zhengyou Zhang's calibration method to acquire their respective parameters such as coordinates of optical center, focal length, scale factor and/or lens distortion; after acquiring the parameters of the two cameras, calibrating the binocular stereo vision system with Zhengyou Zhang's calibration method again to acquire a displacement and rotation angle of a right-view camera relative to a left-view camera, wherein the calibration method for the binocular stereo vision system with Zhengyou Zhang's calibration method comprises: taking the left-view camera as the reference, calibrating with Zhengyou Zhang's calibration method to acquire the external parameters (relative translation vector T1=(l,m,n)T and relative rotation vector V=(α,β,γ)T)) of the right-view camera and the internal parameters (focal length, optical-axis angle, and distortion) of the two cameras themselves, wherein, l,m,n refer to displacements of the right-view camera relative to the left-view camera in three directions of x, y and z respectively, α,β,γ refer to the rotation angles of the right-view camera relative to the left-view camera around three axes of x, y and z respectively, wherein a convergence point of the binocular stereo vision system is determined according to the external parameters:, {
   
    
     
      
       B
       =
       
        
         
          l
          2
         
         +
         
          m
          2
         
         +
         
          n
          2
         
        
       
      
     
    
    
     
      
       ε
       =
       β
      
     
    
   
  
  ,
 

wherein, B is a baseline length between the two cameras, and ε indicates the angle between the optical axes of the two cameras, wherein a transformation model is established for a target angle, and imaging points of the same space point in the two cameras are respectively called a left corresponding point and a right corresponding point, which are intersections of the left-view camera and the right-view camera's optical axes and their respective imaging planes, α′ represents a difference between the left corresponding point and the convergence point in the u direction of the image coordinate system, and b′ represents a difference between the right corresponding point and the convergence point in the u direction of the image coordinate system when the left corresponding point or the right corresponding point is on the left side of the convergence point, a difference value is less than 0, otherwise a difference is greater than 0, wherein the optical axes are perpendicular to their respective imaging planes, a line connecting the optical center to the target point is called corresponding axis, and angles α and b between the corresponding axes and the optical axes is calculated as:, {
   
    
     
      
       a
       =
       
        arctan
        ⁡
        (
        
         
          a
          ′
         
         /
         
          f
          l
         
        
        )
       
      
     
    
    
     
      
       
        b
        =
        
         arctan
         ⁡
         (
         
          
           b
           ′
          
          /
          
           f
           r
          
         
         )
        
       
        
      
     
    
   
  
  ,
 

wherein, fl and fr indicate a focal length of the left-view camera and the right-view camera respectively, wherein when the target point is in region I, a target angle c can be calculated as, {
   
    
     
      
       c
       =
       
        
         ε
         +
         
          
           ❘
           "\[LeftBracketingBar]"
          
          b
          
           ❘
           "\[RightBracketingBar]"
          
         
         -
         
          
           ❘
           "\[LeftBracketingBar]"
          
          a
          
           ❘
           "\[RightBracketingBar]"
          
         
        
        =
        
         ε
         -
         b
         +
         a
        
       
      
     
    
    
     
      
       
        a
        <
        0
       
       ,
       
        
         
          
           b
           <
           0
          
          &
         
         ⁢
           
         ε
        
         
        >
        0
       
      
     
    
   
  
  ;
 

the world coordinates of a target point P in region I are (x, y, z), and a depth calculation model for the target point P are being established as:, tan
     ⁢
     a
    
    =
    
     x
     z
    
   
   ;
  
  ⁢
  

  
   
    
     tan
     ⁡
     (
     
      c
      -
      a
     
     )
    
    =
    
     
      l
      +
      x
     
     
      z
      -
      n
     
    
   
   ,
   
    a
    <
    0
   
   ,
   

   then
  
  ⁢
  

  
   
    z
    =
    
     
      
       n
       ⁢
       
        tan
        ⁡
        (
        
         ε
         -
         b
        
        )
       
      
      +
      l
     
     
      
       tan
       ⁡
       (
       
        ε
        -
        b
       
       )
      
      +
      
       tan
       ⁢
       a
      
     
    
   
   ,
  
 

wherein a world coordinate x can be calculated with the depth calculation model for the target point P as:, x
   =
   
    
     
      
       n
       ⁢
       
        tan
        ⁡
        (
        
         ε
         -
         b
        
        )
       
      
      +
      l
     
     
      
       tan
       ⁡
       (
       
        ε
        -
        b
       
       )
      
      +
      
       tan
       ⁢
       a
      
     
    
    ⁢
    tan
    ⁢
    a
   
  
  ,
 

wherein a world coordinate y can be calculated according to a relation between the left corresponding point and the right corresponding point in the image coordinate system and a relation between the image coordinate system and a world coordinate system, wherein the world coordinates of the target point P are:, {
  
   
    
     
      
       x
       =
       
        
         
          
           n
           ⁢
           
            tan
            ⁡
            (
            
             ε
             -
             b
            
            )
           
          
          +
          l
         
         
          
           tan
           ⁡
           (
           
            ε
            -
            b
           
           )
          
          +
          
           tan
           ⁢
           a
          
         
        
        ⁢
        tan
        ⁢
        a
       
      
     
    
    
     
      
       y
       =
       
        
         
          
           n
           ⁢
           
            tan
            ⁡
            (
            
             ε
             -
             b
            
            )
           
          
          +
          l
         
         
          
           tan
           ⁡
           (
           
            ε
            -
            b
           
           )
          
          +
          
           tan
           ⁢
           a
          
         
        
        ·
        
         
          v
          ′
         
         
          f
          l
         
        
       
      
     
    
    
     
      
       z
       =
       
        
         
          n
          ⁢
          
           tan
           ⁡
           (
           
            ε
            -
            b
           
           )
          
         
         +
         l
        
        
         
          tan
          ⁡
          (
          
           ε
           -
           b
          
          )
         
         +
         
          tan
          ⁢
          a
         
        
       
      
     
    
   
   ;
  
 

wherein, v′ indicates the pixel difference between the target point and the image center in a longitudinal direction of the image coordinate system, and fl is the focal length of the left-view camera, similarly, world coordinates of the target point in region II, region III and region IV are calculated.