# Spatio-Temporal Behavior Detection in Field Manual Labor Based on Improved SlowFast Architecture
## 2 Related Work
### Faster R-CNN + SlowFast Spatio-Temporal Behavior Detection Method  
In the realm of human behavior recognition, depending solely on a single behavior recognition model presents several limitations. It faces challenges in capturing intricate details about the target object [29] and the surrounding contextual environment, thereby constraining the overall scene comprehension and analysis [30]. Conversely, an approach that integrates behavioral recognition with a target detection model can acquire compre \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Stage** & **Slow Pathway** & **Fast Pathway** & **S/F Output T \(\times\) S2** \\ \hline \multirow{3}{*}{res\({}_{2}\)} & \([1\times 1^{2},64\) & \([3\times 1^{2},8\) & \\  & \(1\times 3^{2},64\) & \(1\times 3^{2},8\) & \\  & \(1\times 1^{2},256]\times 3\) & \(1\times 1^{2},32]\times 3\) & \\ \hline \multirow{3}{*}{res\({}_{3}\)} & \([1\times 1^{2},128\) & \([3\times 1^{2},64\) & \\  & \(1\times 3^{2},128\) & \(1\times 3^{2},64\) & \\  & \(1\times 1^{2},512]\times 4\) & \(1\times 1^{2},256]\times 4\) & \\ \hline \multirow{3}{*}{res\({}_{4}\)} & \([3\times 1^{2},256\) & \([3\times 1^{2},32\) & \\  & \(1\times 3^{2},256\) & \(1\times 3^{2},32\) & \\  & \(1\times 1^{2},1024]\times 6\) & \(1\times 1^{2},128]\times 6\) & \\ \hline \multirow{3}{*}{res\({}_{5}\)} & \([3\times 1^{2},512\) & \([1\times 1^{2},64\) & \\  & \(1\times 3^{2},512\) & \(1\times 3^{2},64\) & \\  & \(1\times 1^{2},2048]\times 3\) & \(1\times 1^{2},256]\times 3\) & \\ \hline \multicolumn{3}{c}{Global average pool, concat, fc} & \#classes} \\ \hline \hline \end{tabular}
\end{table} 
Table 1: _Cont._  
Figure 2: Faster RCNN network structure diagram. hensive information regarding the location, scale, and orientation of the target object. This integration leads to a more precise contextual environment for behavioral recognition. This not only aids in the precise analysis of human motion but also captures the interaction between the target object and the environment, enhancing the understanding of human behavior within the entire scene. Manual labor behavior in the field is a continuous motion process, encompassing both spatial and temporal information [31]. Therefore, extracting temporal features from field manual labor behavior is highly significant for behavior recognition. Simultaneously, utilizing the spatio-temporal information from the video can enhance the extraction of video features [32]. The spatio-temporal behavior detection algorithm serves two purposes: to target localization and multi-label behavior recognition for target classification. The schematic representation of the spatio-temporal behavior detection algorithm is illustrated in Figure 3. Initially, the video input frames undergo preprocessing to identify key frames and furnish restricted temporal context information. By handling the data within a relatively concise temporal context, this not only enhances the operational speed of the spatio-temporal behavior detection algorithm but also alleviates potential ambiguities in understanding arising from the sole reliance on static frames for behavior determination. Next is the selection of the target detection model. For target localization, two mainstream methods exist: two-stage and one-stage target detection. Two-stage target detection exhibits higher accuracy compared to that of one-stage, particularly in detecting larger targets where it produces superior results. In terms of target size and the number of targets, the two-stage target detection method performs well when dealing with larger targets and a limited number of targets. Given that the focus of this work is on human behavior involving larger targets, the two-stage target detection model, Faster R-CNN, is used in the subsequent experiments to accomplish the localization task by directly acquiring the target frame size. The target frame is mapped onto the feature blocks obtained from the 3D feature extraction network to derive the corresponding feature matrix.