Patent ID: 11910092
Assignee: BLACK SESAME TECHNOLOGIES INC.
Field: Audio-visual technology (Electrical engineering)
Classification: CPC H  B  G | IPC B  G  H

Claim 0:
1. A panoramic look-around view generation method, comprising steps of:
acquiring images of areas around a vehicle from a plurality of cameras installed on the vehicle, and acquiring steering wheel angle information and vehicle speed information from the vehicle;
generating a look-around view associated with a first time by transforming and mosaicking images acquired by the plurality of cameras, using the look-around view generated at a first time as the look-around view at the first time, and saving the look-around view at the first time;
(i) acquiring images of areas around the vehicle from a plurality of cameras installed on the vehicle, and acquiring steering wheel angle information and vehicle speed information from the vehicle;
(ii) generating the look-around view associated with a current time by transforming and mosaicking images acquired by the plurality of cameras, using the look-around view generated at the current time as the look-around view at the current time, and saving the look-around view at the current time;
(iii) performing a registration calculation on the look-around view at the current time and a saved look-around view at a previous time, wherein the registration calculation comprises following steps of:
(iv) respectively extracting a plurality of characteristic points of the look-around view at the previous time and a plurality of characteristic points of the look-around view at the current time by using an Oriented FAST and Rotated BRIEF (ORB) algorithm;
(v) calculating a position of a characteristic point of the look-around view at the previous time in the look-around view at the current time according to a vehicle motion model using the steering wheel angle information and the vehicle speed information;, selecting a characteristic point located near the position from the characteristic points of the look-around view at the current time to be matched with the characteristic point of the look-around view at the previous time; and
(vi) calculating an affine transformation matrix between the look-around view at the previous time and the look-around view at the current time;
(vii) performing affine transformation on the look-around view at the previous time by using the affine transformation matrix and performing weighted fusion with the look-around view at the current time to generate an updated look-around view and save the updated look-around view;
repeating above steps (i)-(vii) to obtain continuously updated panoramic look-around views; and
wherein a first transformation matrix of image coordinate systems is:, [
    
      
        
          
            u
            1
          
        
      
      
        
          
            v
            1
          
        
      
      
        
          1
        
      
    
    ]
  
  =
  
    
      [
      
        
          
            
              cos
              ⁢
              
                
              
              ⁢
              θ
            
          
          
            
              sin
              ⁢
              
                
              
              ⁢
              θ
            
          
          
            
              
                P
                x
              
              -
              
                
                  P
                  x
                
                ⁢
                
                  
                
                ⁢
                cos
                ⁢
                
                  
                
                ⁢
                θ
              
              -
              
                
                  P
                  y
                
                ⁢
                
                  
                
                ⁢
                sin
                ⁢
                
                  
                
                ⁢
                θ
              
            
          
        
        
          
            
              
                -
                sin
              
              ⁢
              
                
              
              ⁢
              θ
            
          
          
            
              cos
              ⁢
              
                
              
              ⁢
              θ
            
          
          
            
              
                P
                y
              
              -
              
                
                  P
                  y
                
                ⁢
                
                  
                
                ⁢
                cos
                ⁢
                
                  
                
                ⁢
                θ
              
              +
              
                
                  P
                  x
                
                ⁢
                
                  
                
                ⁢
                sin
                ⁢
                
                  
                
                ⁢
                θ
              
            
          
        
        
          
            0
          
          
            0
          
          
            1
          
        
      
      ]
    
    ⁡
    
      [
      
        
          
            
              u
              0
            
          
        
        
          
            
              v
              0
            
          
        
        
          
            1
          
        
      
      ]
    
  

wherein (Px,Py) is the coordinate of a rotation center of the vehicle motion; θ is a rotation angle of the vehicle from the previous time to the current time; (u0,v0) is a coordinate of a characteristic point of the look-around view at the previous time in the look-around view at the previous time; (u1,v1) is the coordinate of the characteristic point of the look-around view at the previous time in the look-around view at the current time.