Patent ID: 11967180
Assignee: SHANDONG COMPUTER SCIENCE CENTER (NATIONAL SUPERCOMPUTER CENTER IN JINAN)
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 6:
7. The dynamic FER method based on the DS theory according to claim 1, wherein the step f) comprises the following steps:
f-1) constituting, by a multi-branch convolution module, an uncertainty combination module, a multi-branch fusion module, and a determining module sequentially, the discriminator Dds guided by the DS theory;
f-2) constituting the multi-branch convolution module by a first branch, a second branch, and a third branch sequentially, wherein the first branch, the second branch, and the third branch each sequentially comprise a first convolutional layer with a 3*3 convolution kernel and a stride of 1, a first BN layer, a first ReLu activation function layer, a second convolutional layer with a 3*3 convolution kernel and a stride of 2, a second BN layer, a second ReLu activation function layer, an average pooling layer, a flatten function layer, and a linear layer sequentially; and inputting the spatio-temporal feature FstP into the first branch, the second branch, and the third branch of the multi-branch convolution module to obtain a first branch vector Vst1P, a second branch vector Vst2P, a third branch vector and Vst3P respectively;
f-3) inputting the first branch vector Vst1P, the second branch vector Vst2P and the third branch vector Vst3P into the uncertainty combination module; taking an exponent with e as a base for the first branch vector Vst1P to obtain a first evidence vector e1=[e11,e12, . . . ,e1k, . . . ,e1K]1, wherein e1k represents an ith evidence vector in the first branch vector, and k=[1, 2, . . . , K]; taking the exponent with e as the base for the second branch vector Vst2P to obtain a second evidence vector e2=[e21,e22, . . . ,e2k, . . . ,e2K], wherein e2k represents an ith evidence vector in the second branch vector; taking the exponent with e as the base for the third branch vector Vst3P to obtain a third evidence vector e3=[e31,e32, . . . ,e3k, . . . ,e3K], wherein e3k represents an ith evidence vector in the third branch vector, k=[1 2, . . . , k], K represents a quantity of sample categories, K=7, and values of k one-to-one correspond to numbers in a label sequence [1: surprise, 2: fear, 3: disgust, 4: happiness, 5: sadness, 6: anger, 7: neutral]; calculating a kth Dirichlet parameter α1k of the first evidence vector e1 according to a formula α1k=e1k+1, calculating a kth Dirichlet parameter α2k of the second evidence vector e2 according to a formula α2k=e2k+1, and calculating a kth Dirichlet parameter α3k of the third evidence vector e3 according to a formula α3k=e3k+1; obtaining Dirichlet strength S1 of the first evidence vector e1 according to a formula S1=Σk=1Kα1k, Dirichlet strength S2 of the second evidence vector e2 according to a formula S2=Σk=1Kα2k, and Dirichlet strength S3 of the third evidence vector e3 according to a formula S3=Σk=1Kα3k; obtaining first uncertainty u1 according to a formula, u
    1
   
   =
   
    K
    
     S
     1
    
   
  
  ,, second uncertainty u2 according to a formula, u
    2
   
   =
   
    K
    
     S
     2
    
   
  
  ,, and third uncertainty u3 according to a formula, u
    3
   
   =
   
    K
    
     S
     3
    
   
  
  ;, obtaining a first confidence coefficient b1 according to a formula, b
    1
   
   =
   
    
     
      α
      1
      k
     
     -
     1
    
    
     S
     1
    
   
  
  ,, a second confidence coefficient b2 according to a formula, b
    2
   
   =
   
    
     
      α
      2
      k
     
     -
     1
    
    
     S
     2
    
   
  
  ,, and a third confidence coefficient b3 according to a formula, b
    3
   
   =
   
    
     
      α
      3
      k
     
     -
     1
    
    
     S
     3
    
   
  
  ;, calculating a first conflict factor C12 according to a formula C12=b1b2 and a second conflict factor C23 according to a formula C23=b2b3; calculating a second prefix weight w2 according to a formula, w
    2
   
   =
   
    
     1
     
      1
      -
      
       C
       
        1
        ⁢
        2
       
      
     
    
    ⁢
    
     u
     1
    
    ⁢
    
     u
     2
    
   
  
  ,, and a third prefix weight w3 according to a formula, w
    3
   
   =
   
    
     1
     
      1
      -
      
       C
       
        2
        ⁢
        3
       
      
     
    
    ⁢
    
     u
     2
    
    ⁢
    
     u
     3
    
   
  
  ,, wherein a first prefix weight is w1=1; and multiplying the first branch vector Vst1P by the first prefix weight w1 to obtain a first weight vector V1P, multiplying the second branch vector Vst2P by the second prefix weight w2 to obtain a second weight vector V2P, and multiplying the third branch vector Vst3P by the third prefix weight w3 to obtain a third weight vector V3P;
f-4) inputting the first weight vector V1P the second weight vector V2P and the third weight vector V3P into the multi-branch fusion module, and calculating a fusion vector VfuseP according to a formula VfuseP=V1P+V2P+V3P; and
f-5) constituting the determining module by a Softmax function and a max function, inputting the fusion vector VfuseP into the Softmax function for normalization, inputting a normalized fusion vector VfuseP into the max function to obtain a subscript Ek of a maximum value, wherein k=[1, 2, . . . , K], and the values of k one-to-one correspond to the numbers in the label sequence [1: surprise, 2: fear, 3: disgust, 4: happiness, 5: sadness, 6: anger, 7: neutral], and comparing the subscript Ek of the maximum value with the label sequence [1: surprise, 2: fear, 3: disgust, 4: happiness, 5: sadness, 6: anger, 7: neutral] to find a corresponding expression label as a determining result R.