Patent ID: 11908124
Assignee: CHONGQING UNIVERSITY
Field: Computer technology (Electrical engineering)
Classification: CPC G  Y | IPC G

Claim 3:
4. The pavement nondestructive detection and identification method based on the small samples according to claim 1, wherein an expression of multi-head attention features is:, {
  
   
    
     
      
       
        Q
        h
       
       =
       
        
         W
         Q
         h
        
        ⁢
        
         E
         T
        
       
      
     
    
    
     
      
       
        K
        h
       
       =
       
        
         W
         Q
         h
        
        ⁢
        
         E
         T
        
       
      
     
    
    
     
      
       
        V
        h
       
       =
       
        
         W
         V
         h
        
        ⁢
        
         E
         T
        
       
      
     
    
    
     
      
       
        Z
        i
        h
       
       =
       
        
         
          ∑
           
         
         
          j
          =
          1
         
         n
        
        ⁢
         
        soft
        ⁢
        
         max
         ⁡
         (
         
          
           
            Q
            i
           
           ⁢
           
            K
            j
            T
           
          
          
           
            d
            k
           
          
         
         )
        
        ⁢
        
         V
         j
        
       
      
     
    
    
     
      
       
        𝒵
        h
       
       =
       
        [
        
         
          Z
          0
          h
         
         ;
         
          Z
          1
          h
         
         ;
         …
            
         ;
         
          Z
          n
          h
         
        
        ]
       
      
     
    
    
     
      
       
        MHA
        ⁡
        (
        𝒵
        )
       
       =
       
        
         W
         O
        
        ⁢
        
         𝒵
         T
        
       
      
     
    
   
   ,
  
 

wherein n represents a number of embeddings, h is an output of an h-th head, dk represents a dimension of K, and a query Q, a key K and a value V are obtained by an embedding E and a learnable weight W.