Patent ID: 11967180
Assignee: QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES)
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 2:
3. The dynamic FER method based on the DS theory according to claim 1, wherein the step c) comprises the following steps:
c-1) constituting the same-identity inter-frame sharing module Ms by a first convolution module, a second convolution module, and a third convolution module sequentially, and constituting the space-domain attention module Matt by a first FC module and a second FC module sequentially;
c-2) constituting the first convolution module of the same-identity inter-frame sharing module Ms by a convolutional layer with a 3*3 convolution kernel and a stride of 1, a batch normalization (BN) layer, and a rectified linear unit (ReLU) activation function layer sequentially, and inputting the facial expression image P into the first convolution module to obtain a feature Fs1P;
c-3) constituting the second convolution module of the same-identity inter-frame sharing module Ms by a downsampling module and a residual module sequentially, wherein the downsampling module comprises a first branch and a second branch, the first branch sequentially comprises a first convolutional layer with a 3*3 convolution kernel and a stride of 2, a first BN layer, a first ReLu activation function layer, a second convolutional layer with a 3*3 convolution kernel and a stride of 1, a second BN layer, and a second ReLu activation function layer, the second branch sequentially comprises a third convolutional layer with a 1*1 convolution kernel and a stride of 2, a third BN layer, and a third ReLu activation function layer, the residual module sequentially comprises a fourth convolutional layer with a 3*3 convolution kernel and a stride of 1, a fourth BN layer, a fourth ReLu activation function layer, a fifth convolutional layer with a 3*3 convolution kernel and a stride of 1, a fifth BN layer, and a fifth ReLu activation function layer; inputting the feature Fs1P into the first branch of the downsampling module of the second convolution module to obtain a feature Fsd2P1, and inputting the feature Fsd2P into the second branch of the downsampling module of the second convolution module to obtain a feature Fsd2P2; adding up the feature Fsd2P1 and the feature Fsd2P2 to obtain a feature Fsd2P2; and inputting the feature Fsd2P into the residual module of the second convolution module to obtain a feature Fs2P;
c-4) constituting the third convolution module of the same-identity inter-frame sharing module Ms by a downsampling module and a residual module sequentially, wherein the downsampling module comprises a first branch and a second branch, the first branch sequentially comprises a first convolutional layer with a 3*3 convolution kernel and a stride of 2, a first BN layer, a first ReLu activation function layer, a second convolutional layer with a 3*3 convolution kernel and a stride of 1, a second BN layer, and a second ReLu activation function layer, the second branch sequentially comprises a third convolutional layer with a 1*1 convolution kernel and a stride of 2, a third BN layer, and a third ReLu activation function layer, the residual module sequentially comprises a fourth convolutional layer with a 3*3 convolution kernel and a stride of 1, a fourth BN layer, a fourth ReLu activation function layer, a fifth convolutional layer with a 3*3 convolution kernel and a stride of 1, a fifth BN layer, and a fifth ReLu activation function layer; inputting the feature Fs2P into the first branch of the downsampling module of the third convolution module to obtain a feature Fsd3P1, and inputting the feature Fs1P into the second branch of the downsampling module of the third convolution module to obtain a feature Fsd3P2; adding up the feature Fsd3P1 and the feature Fsd3P2 to obtain a feature Fsd3P; and inputting the feature Fsd3P into the residual module of the third convolution module to obtain a feature Fs3P;
c-5) calculating the same-identity inter-frame shared feature FsP according to a formula, F
    s
    P
   
   =
   
    
     1
     
      1
      ⁢
      6
     
    
    ⁢
    
     
      ∑
       
     
     
      i
      =
      1
     
     
      1
      ⁢
      6
     
    
    ⁢
    
     F
     
      s
      ⁢
      3
     
     
      P
      ⁢
      i
     
    
   
  
  ,, wherein in the formula, Fs3Pi represents an ith video frame vector in the feature Fsd3P;
c-6) constituting the first FC module of the space-domain attention module Matt by a BN layer, a flatten function, an FC layer, and a ReLu activation function layer sequentially, and inputting the facial expression image P into the first FC module to obtain a feature Fatt1P;
c-7) constituting the second FC module of the space-domain attention module Matt by an FC layer and a Sigmoid function layer sequentially, and inputting the feature Fatt1P into the second FC module to obtain the space-domain attention feature FattP; and
c-8) multiplying the same-identity inter-frame shared feature FsP by the space-domain attention feature FattP to obtain the space-domain feature FsattPS.