Patent ID: 11922643
Assignee: nan
Field: Audio-visual technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 0:
1. A method for intelligently measuring vehicle speed based on a binocular stereo vision system, comprising:
step 1, inputting a dataset of images containing license plates into an SSD (Single Shot Multibox Detector) neural network, and training the SSD neural network with license plate as a detection feature to obtain a license plate recognition model;
step 2, installing the binocular stereo vision system on the right side, middle or above a lane, calibrating the binocular stereo vision system to acquire the internal and external parameters of two cameras; and recording videos of moving target vehicles by the calibrated binocular stereo vision system;
step 3, detecting the license plates in video frames recorded by the calibrated binocular stereo vision system with the license plate recognition model trained in step 1, and locating license plate positions of the target vehicles;
step 4, performing feature point extraction and stereo matching on the license plates in subsequent frames of the same camera by a feature-based matching algorithm, and reserving correct matching points after filtering with a homography matrix; performing feature point extraction and stereo matching on the license plates in corresponding left-view and right-view video frames of the binocular stereo vision system by the feature-based matching algorithm, and reserving correct matching points after filtering with the homography matrix;
step 5, screening the reserved matching point pairs in step 4, then eliminating screened matching point pairs by a distance measurement method of the binocular stereo vision system, and reserving one closest to a license plate center as the position of a target vehicle in a current frame;
step 6, performing stereo measurement on the screened matching point pairs by the binocular stereo vision system to get spatial position coordinates; calculating a distance that the target vehicle passes in a certain time according to the position information, and obtaining a moving speed of the target vehicle;
the calibration method for the binocular stereo vision system in the step 2 comprises: calibrating the two cameras to acquire parameters of the two cameras respectively, which includes coordinates of optical center, focal length, scale factor and/or lens distortion; after acquiring the parameters of the two cameras, calibrating the binocular stereo vision system again to acquire the displacement and rotation angle of a right-view camera relative to a left-view camera (taking the left-view camera as a reference),
a calibration method for the binocular stereo vision system comprises: taking the left-view camera as the reference, calibrating to acquire the external parameters (relative translation vector T1=(l,m,n)T and relative rotation vector V=(α,β,γ)T)) of the right-view camera and the internal parameters (focal length, optical-axis angle, and distortion) of the two cameras, wherein T refers to a transpose of a matrix, l,m,n refer to displacements of the right-view camera relative to the left-view camera in the three directions of x, y and z respectively, α,β,γ refer to rotation angles of the right-view camera relative to the left-view camera around the three axes of x, y and z respectively,
a convergence point of the binocular stereo vision system is determined according to the external parameters:, {
  
   
    
     
      B
      =
      
       
        
         l
         2
        
        +
        
         m
         2
        
        +
        
         n
         2
        
       
      
     
    
   
   
    
     
      ε
      =
      β
     
    
   
  
 

wherein, B is the baseline length between the two cameras, and ε indicates the angle between optical axes of the two cameras,
a transformation model is established for a target angle, imaging points of identical spatial point in the two cameras are respectively called a left corresponding point and a right corresponding point, which are intersections of the optical axes of the left-view camera and the right-view camera and respective imaging planes of the two optical axes, a′ represents the difference between the left corresponding point and a convergence point in the u direction of an image coordinate system, and b′ represents the difference between the right corresponding point and the convergence point in the u direction of the image coordinate system, if the left corresponding point or the right corresponding point is on the left side of the convergence point, a difference value is less than 0, otherwise the difference value is greater than 0,
the optical axes are perpendicular to respective imaging planes a line connecting an optical center to a target point is called a corresponding axis, angles a and b between corresponding axes and the optical axes are calculated as:, {
  
   
    
     
      a
      =
      
       arctan
       ⁡
       (
       
        
         a
         ′
        
        /
        
         f
         l
        
       
       )
      
     
    
   
   
    
     
      b
      =
      
       arctan
       ⁡
       (
       
        
         b
         ′
        
        /
        
         f
         r
        
       
       )
      
     
    
   
  
 

wherein, fl and fr indicate the focal length of the left-view camera and the right-view camera respectively,
for the target point in region I, a target angle c is calculated as, {
  
   
    
     
      
       c
       =
       
        
         ε
         +
         
          
           ❘
           "\[LeftBracketingBar]"
          
          b
          
           ❘
           "\[RightBracketingBar]"
          
         
         -
         
          
           ❘
           "\[LeftBracketingBar]"
          
          a
          
           ❘
           "\[RightBracketingBar]"
          
         
        
        =
        
         ε
         -
         b
         +
         a
        
       
      
      ;
     
    
   
   
    
     
      
       a
       <
       0
      
      ,
      
       
        
         
          b
          <
          0
         
         &
        
        ⁢
          
        ε
       
        
       >
       0
      
     
    
   
  
 

assuming world coordinates of a target point P in region I are (x, y, z), a depth calculation model for the target point P is established as:, tan
     ⁢
     a
    
    =
    
     x
     z
    
   
   ;
  
  ⁢
  

  
   
    
     tan
     ⁡
     (
     
      c
      -
      a
     
     )
    
    =
    
     
      l
      +
      x
     
     
      z
      -
      n
     
    
   
   ,
   
    a
    <
    0, then, z
  =
  
   
    
     n
     ⁢
     
      tan
      ⁡
      (
      
       ε
       -
       b
      
      )
     
    
    +
    l
   
   
    
     tan
     ⁡
     (
     
      ε
      -
      b
     
     )
    
    +
    
     tan
     ⁢
     a
    
   
  
 

a world coordinate x is calculated with a depth calculation model for the target point P as:, x
  =
  
   
    
     
      n
      ⁢
      
       tan
       ⁡
       (
       
        ε
        -
        b
       
       )
      
     
     +
     l
    
    
     
      tan
      ⁡
      (
      
       ε
       -
       b
      
      )
     
     +
     
      tan
      ⁢
      a
     
    
   
   ⁢
   tan
   ⁢
   
    a
    .
   
  
 

a world coordinate y are calculated according to a relation between a left corresponding point and a right corresponding point in the image coordinate system and a relation between the image coordinate system and the world coordinate system
the world coordinates of the target point P are:, {
  
   
    
     
      
       x
       =
       
        
         
          
           n
           ⁢
           
            tan
            ⁡
            (
            
             ε
             -
             b
            
            )
           
          
          +
          l
         
         
          
           tan
           ⁡
           (
           
            ε
            -
            b
           
           )
          
          +
          
           tan
           ⁢
           a
          
         
        
        ⁢
        tan
        ⁢
        a
       
      
     
    
    
     
      
       y
       =
       
        
         
          
           n
           ⁢
           
            tan
            ⁡
            (
            
             ε
             -
             b
            
            )
           
          
          +
          l
         
         
          
           tan
           ⁡
           (
           
            ε
            -
            b
           
           )
          
          +
          
           tan
           ⁢
           a
          
         
        
        ·
        
         
          v
          ′
         
         
          f
          l
         
        
       
      
     
    
    
     
      
       z
       =
       
        
         
          n
          ⁢
          
           tan
           ⁡
           (
           
            ε
            -
            b
           
           )
          
         
         +
         l
        
        
         
          tan
          ⁡
          (
          
           ε
           -
           b
          
          )
         
         +
         
          tan
          ⁢
          a
         
        
       
      
     
    
   
   ;
  
 

wherein, v′ indicates pixel difference between the target point P and an image center in the longitudinal direction of the image coordinate system, and fl is the focal length of the left-view camera, then world coordinates of the target point P in region II, region III and region IV are calculated.