# Spatio-Temporal Behavior Detection in Field Manual Labor Based on Improved SlowFast Architecture
## 3 Dataset and Methods
### Improvements to SlowFast Network Structure
#### 3.2.2 Action Attention Modules
The global spatial information tensor can be expressed as follows:  
\[F_{1}=\frac{1}{H\times W}\sum_{i=1}^{H}\sum_{j=1}^{W}X[:,:,:,i,j] \tag{10}\]  
The second step involves applying a \(1\times 1\) convolutional kernel \(K_{1}\) to \(F_{1}\in R^{N\times T\times C\times 1\times 1}\) with a channel reduction ratio r to compress the channel count of the feature map \(F_{1}\), resulting in a new feature map \(F_{r}\):  
Figure 10.0  
In the third step, \(F_{r}\in R^{N\times T\times T\times T\times T\times T\times T\times T\times T}\) is reshaped to obtain \(F_{r}^{*}\in R^{N\times T\times T\times T\times T\times T\times T}\), and in the fourth step, a one-dimensional convolutional kernel \(K_{2}\) with a kernel size of 3 is applied to process \(F_{r}^{*}\), resulting in a new feature map \(F_{\text{temp}}^{*}\):  
Figure 10.1  
In the fifth step, the dimension of \(F_{\text{temp}}^{*}\) is modified to yield a new feature map \(F_{temp}\in R^{N\times T\times T\times T\times T\times T}\). Subsequently, in the sixth step, a final feature map \(F_{o2}\) is derived through additional feature extraction and processing of \(F_{temp}\) using a 2D convolution kernel \(K_{3}\) with a size of 1:  
Figure 10.2  
and the mask \(M_{2}\in R^{N\times T\times C\times 1\times 1}\) is obtained through the Sigmoid activation function. The output of \(M_{2}\) can be expressed as follows:  
Figure 10.3  
Finally, \(M_{2}\) is multiplied element-wise with \(X\) and then summed with \(X\) to obtain the final output \(Y_{2}\). The output of \(Y_{2}\) can be expressed as follows:  
Figure 10.4  
The Motion Excitation (ME) attention module, depicted in Figure 12, primarily captures the motion information associated with the action's movement between two adjacent  
Figure 10.5 The structure of CE attention module. frames. This information is then concatenated as a branch with the two previously mentioned attention modules to form the ACTION module. The motion excitation (ME) attention module begins with dimension reduction through a 1 \(\times\) 1 convolution, followed by the computation of adjacent frame features using Equation (16). Here, \(F_{m}\) represents the motion feature map, \(K_{4}\) is a \(3\times 3\) convolutional kernel, \(F_{r}[:,t,:,:,:]\) and \(F_{r}[:,t+1,:,:,:]\) represent the features of the current frame and the previous frame, respectively. Figure 10.6  
Subsequently, motion features are concatenated along the temporal dimension and zero-padded up to the last position. The expression is given as:  
Figure 10.7  
where \(F_{M}\in R^{N\times T\times C\times 1\times 1}\) represents a series of motion feature maps concatenated along the temporal dimension, and t denotes the time dimension. Then, \(F_{M}\) undergoes spatial average pooling as described in Equation (10), followed by a 1 \(\times\) 1 convolution, and subsequent unsqueeze dimensionality upscaling, as specified in Equation (13), to obtain a new feature map \(F_{03}\), and the mask \(M_{3}\in R^{N\times T\times C\times 1\times 1}\) is obtained through the Sigmoid activation function. The output of \(M_{3}\) can be expressed as follows:  
Figure 10.8  
Finally, \(M_{3}\) is multiplied element-wise with \(X\) and then summed with \(X\) to obtain the final output \(Y_{3}\). The output of \(Y_{3}\) can be expressed as follows:  
Figure 10.9  
The final output \(Y\) of the ACTION module is represented as:  
Figure 8:0  
Figure 8:1 The structure of ME attention module. The ACTION attention module models internal features of the network at the feature level, enhancing the representation of diverse information in videos for better action representation. The original fast branch of the SlowFast network extracts spatio-temporal features to represent actions using stacked residual blocks with 3 \(\times\) 1 \(\times\) 1 convolution kernels. However, the extensive intra-class diversity necessitates learning more fine-grained spatio-temporal features.