# Research on Students' Action Behavior Recognition Method Based on Classroom Time-Series Images
## 3 Proposed Method
### Detection of Students' Action Behavior
#### 3.1.5. MsCSA Module  
The video data captured in this study have a large resolution; a large difference in pixel size occupied by front and back row students in space; and the need to extract small object features \(O_{t}\), such as books and mobile phones around the students, along with other types of features, to model the interaction. Literature [35] shows that multi-scale receptive field is helpful for the network to notice the spatial position of different size objects. Therefore, a Multi-scale Spatial Attention (MsSA) block is added to the slow pathway of the video backbone network to obtain contextual information of a larger receptive field; at the same time, to allow the network to learn the weight response of the feature maps of different channels in the process, the MsSA block is cascaded with Gaussian Context Transformer (GCT) [36] to form an MsCSA module. The MsCSA module is shown in Figure 7.  
**GCT block. ** The GCT block first performs the global average pooling operation on the input feature map \(X\) in the spatial dimension; then, it stabilizes the distribution of the global features by normalizing the channel vectors; finally, it obtains the attention map by using the Gaussian function to perform the excitation operation on the normalized global features. The specific process is as follows:  
(1) Global Context Aggregation (GCA). Let the input \(X\in R^{C\times T\times H\times W}\) reshape \(X\) into \(R^{CT\times H\times W}\); the GCA can be expressed as  
\[z=avg(X)=\left\{z_{k}=\frac{1}{H\times W}{\sum_{i=1}^{W}{\sum_{j=1}^{H}{X_{k}( i,j)}};k\in\{1,\cdots,CT\}}\right\} \tag{11}\]  
where \(CT\) is the number of channels, and \(H\) and \(W\) are the height and width of the feature map, respectively. (2) Normalize. The GCT block computes the attentional activation value of the channel using the function \(f(\cdot)\). Define \(\hat{z}\) as the input to the function \(f(\cdot)\), and \(\hat{z}\) is represented by Equation (12):  
\[\hat{z}=\frac{1}{\sigma}(z-\mu) \tag{12}\]  
Figure 6: MsTA module. Figure 7: MsCSA module. where \(\mu=\frac{1}{c}\sum_{k=1}^{C}z_{k}\) denotes the mean of the global context \(z\), and \(z-\mu\) is the mean shift. \(\sigma=\sqrt{\frac{1}{c}\sum_{k=1}^{C}\left(z_{k}-\mu\right)^{2}+\varepsilon}\) denotes the standard deviation of the global context \(z\), \(\varepsilon\) is a very small constant, and \(\sigma\) is introduced so that \(\hat{z}\) remains in distribution with mean 0 and variance 1 for different input samples, thus ensuring that the output of \(f(\cdot)\) is stable. Equation (11) is consistent with the result of normalizing \(z\) and, thus, can be expressed as \(\hat{z}=norm(z)\). (3) Gaussian Context Excitation (GCE). The GCT block substitutes the Gaussian function into \(f(\hat{z})\) and defines the GCE operation as  
\[g=f(\hat{z})=e^{-\frac{\hat{z}^{2}}{2\hat{z}^{2}}} \tag{13}\]  
where \(c\) is a constant or learnable parameter and \(g\) is the attentional activation value. The above steps are combined to form the GCT block, which is represented by Equation (14):  
\[F_{c}=e^{-\frac{norm(arg(X))^{2}}{2\hat{z}^{2}}}X \tag{14}\]  
where \(F_{c}\in R^{CT\times H\times W}\) is the feature map output by the GCT block. **MsSA block. ** After the GCT block, \(F_{c}\in R^{CT\times H\times W}\) is input to the Multi-scale Spatial Convolution Unit (MsSCU) in the MsSA block to extract the spatial information at different scales.