# Spatio-Temporal Behavior Detection in Field Manual Labor Based on Improved SlowFast Architecture
## 3 Dataset and Methods
### Improvements to SlowFast Network Structure
#### 3.2.2 Action Attention Modules  
The ACTION [38] module comprises three complementary attention modules: STE (Spatio-temporal Excitation), CE (Channel Excitation), and ME (Motion Excitation). These consider three crucial aspects of temporal action: (a) spatio-temporal information, i.e., the temporal and spatial relationship of the action; (b) weighting of the temporal information of the action across different channels; and (c) the trajectory of the action between each pair of neighboring frames. Three paths--the spatio-temporal excitation path, channel excitation path, and motion excitation path--are utilized to extract the key spatio-temporal features of the video, the weights of the temporal features of the action across different channels, and the trajectory features of the action's changes between two adjacent frames. The network structure is illustrated in Figure 9.  
The STE module extracts spatio-temporal features from the video by generating a spatio-temporal mask \(M\), which results in a spatio-temporal attention map. The structure of the STE module is illustrated in Figure 10.  
Figure 8: Original and CA residual block network structure. (a) Original block network structure, (b) CA residual block network structure. Traditional spatio-temporal feature extraction typically involves using 3D convolution, but directly employing 3D convolution significantly increases the computational load of the model. Therefore, the Spatio-temporal Excitation (STE) module first performs channel global pooling on the input tensor \(X\in R^{N\times T\times C\times H\times W}\) obtaining the corresponding channel-wise global spatio-temporal feature map \(F\in R^{N\times T\times 1\times H\times W}\), Subsequently, \(F\) is reshaped into \(F^{*}\in R^{N\times 1\times T\times H\times W}\) to make it compatible with 3D convolution operations. \(F^{*}\) is then convolved with a \(3\times 3\times 3\) kernel \(K\) to produce the new spatio-temporal feature map \(F_{\text{o}}^{*}\). The output of this process can be expressed as follows:  
\[F_{\text{o}}^{*}=K*F^{*} \tag{7}\]  
Here, N represents the batch size, T denotes the temporal dimension, represents the number of channels, H denotes the height, W denotes the width, and R represents the real domain, indicating that the elements in the feature map belong to the set of real numbers. Then, \(F_{\text{o}}^{*}\) is reshaped to \(F_{\text{o}}\in R^{N\times T\times 1\times H\times W}\), and the mask \(M_{1}\in R^{N\times T\times 1\times H\times W}\) is obtained through the Sigmoid activation function. The output of \(M_{1}\) can be expressed as follows:  
\[M_{1}=\delta(F_{\text{o}}) \tag{8}\]  
Finally, \(M_{1}\) is multiplied element-wise with \(X\) and then summed with \(X\) to obtain the final output \(Y_{1}\). The output of \(Y_{1}\) can be expressed as follows:  
\[Y_{1}=X+X\odot M_{1} \tag{9}\]  
The CE module is an attention module acting on channels, adaptively recalibrating channel feature responses by explicitly modeling the interdependencies among channels in the temporal domain. The design of the CE module is similar to the SE module, with the distinction that, due to the temporal information present in video actions, the CE module  
Figure 10: The structure of STE attention module. Figure 9: The structure of ACTION attention module. inserts a \(1\times 1\) convolutional layer in the temporal domain between two fully connected (FC) layers. This convolutional layer enhances the temporal interdependence of channels, describing the temporal information of channel features. The network structure diagram of the CE module is shown in Figure 11.  
The CE module first performs spatial average pooling on the input \(X\in R^{N\times T\times C\times H\times W}\) to obtain the global spatial information tensor of the input features.