# Spatio-Temporal Behavior Detection in Field Manual Labor Based on Improved SlowFast Architecture
## 2 Related Work  
### SlowFast Network  
The SlowFast network architecture [27] is inspired by the distribution of retinal nerve cell species in biology. The network introduces two parallel branches, each dedicated to processing fast and slow movements, aiming to enhance spatial-temporal feature extraction. This is accomplished by configuring different sampling intervals and the number of convolutional channels for the two paths, leading to distinct temporal and spatial resolutions.  
Consequently, it captures the diverse characteristics of fast and slow movements in the video. The network structure is depicted in Figure 1.  
The slow path captures semantic features of spatial objects in the video with low temporal resolution and high spatial resolution. Let \(\tau\) represent the temporal resolution, D denote the number of channels, and T the number of sampled frames in this branch. Consequently, the length of the original video is \(\tau\times\) T frames. The 3D convolution in this branch is exclusively employed for high-level feature extraction to prevent a premature application of 3D convolution that may compromise accuracy. The temporal relationships between frames become evident only when the spatial receptive field is sufficiently large. The fast path captures the rapidly changing motion characteristics of the object in the video with high temporal resolution and low spatial resolution. Given the rapid changes in motion information within this branch, data with a high frame rate of \(\alpha\)T (where \(\alpha>1\)) frames and fewer channels, denoted as \(\beta\)D (where \(\beta=1/8\)), are input to capture finely detailed fast-changing motion information in the temporal dimension. Due to the limited number of channels in the fast branch, it remains highly lightweight, constituting only 20% of the overall computational workload. The fast branch, to ensure temporal fidelity for fine motion information, refrains from conducting any temporal downsampling operations and consistently maintains the time dimension as \(\alpha\)T until the ultimate global average pooling. This approach effectively preserves the temporal information of the motion and facilitates the capture of rapid motion features in the video.  
The fast and slow branches exhibit strong complementarity, with unidirectional information fusion occurring from the fast branch to the slow branch. Given the disparity between the dimensions D \(\times\) T \(\times\) S \(\times\) S of the slow branch and the dimensions \(\beta\)D \(\times\)\(\alpha\)T \(\times\) S \(\times\) S of the fast branch, a transformed transversal connection is necessary to align the feature vectors from the outputs of both branches. Information fusion employs time-step convolution, wherein the Pool\({}_{1}\), res\({}_{2}\), res\({}_{3}\), and res\({}_{4}\) layers of the network undergo time-step convolution, followed by unidirectional feature fusion. Eventually, the feature vectors from the slow and fast branches are concatenated and, after the fully connected layer, category prediction is executed. The SlowFast concept represents an idea, and network implementation is specifically based on 3D ResNet50, as illustrated in Table 1.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Stage** & **Slow Pathway** & **Fast Pathway** & **S/F Output T \(\times\) S2** \\ \hline raw clip & — & — & \(64\times 224^{2}\) \\ \hline Date layer & Stride16, \(1^{2}\) & Stride16, \(1^{2}\) & \(4/32\times 224^{2}\) \\ \hline conv\({}_{1}\) & \(1\times 7^{2}\), \(64\) & \(5\times 7^{2}\), \(64\) & \(4/32\times 112^{2}\) \\ \hline Pool\({}_{1}\) & \(1\times 3^{2}\), max & \(1\times 3^{2}\), max & \(4/32\times 56^{2}\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: SlowFast network structure parameters.  
Figure 1: SlowFast network structure.