# Video Human Action Recognition Based on Motion-Tempo Learning and Feedback Attention
## 3 The Proposed Critical Action Information Extraction Network
### Multi-Granularity Adaptive Fusion Module (MgAFM)
#### 3.1.1 Motion-Extraction Module (MEM)  
The MEM is responsible for extracting motion features from sub-actions at different tempos. As illustrated in Figure 3, we employ motion transformations with \((n-1)\) frame intervals to represent sub-actions motion features at different tempos. For the video data, \(T\) frames are sampled as input to the model and subsequently converted into feature representations through convolution calculation. The features corresponding to different frame intervals are represented as \(\mathbf{F}^{t}\in\mathbb{R}^{C\times H\times W}\) and \(\mathbf{F}^{t+n}\in\mathbb{R}^{C\times H\times W}\), where \(C\), \(H\), and \(W\) represent the channel dimension, height, and width, respectively. The \(C\) dimensional features at spatial position \(x\) are denoted by \(\mathbf{F}_{x}\). The correlation score for position \(x\) with a displacement \(p\) is defined as:  
\[s(x,p,t)=\mathbf{F}^{t}_{x}\cdot\mathbf{F}^{t+n}_{x+p}. \tag{1}\]  
where \(\cdot\) denotes the dot product. To enhance computational efficiency, \(x\) is computed only within a neighborhood of size \(P=2R+1\), limiting the displacement to \(p\in[-R,R]^{2}\). For the \(t_{th}\) frame, the resulting correlation tensor \(\mathbf{S}^{(t)}\) has dimensions \(H\times W\times P^{2}\). In our implementation, considering the spatial resolution of \(28\times 28\), we set \(P=15\).  
To establish the motion relationship between video frames at different intervals, the correlation tensor \(\mathbf{S}^{(t)}\) is utilized to generate a displacement field that describes the motion trajectory. To mitigate the influence of noisy outliers in the correlation tensor on the displacement estimation, we adopt the kernel-soft-argmax method [32]. The kernel-soft-argmax for displacement computation is defined as:  
\[d(x,t)=\sum_{p}\frac{\exp(g(x,p,t)s(x,p,t)/\tau)}{\sum_{p^{\prime}}\exp(g(x,p^ {\prime},t)s(x,p^{\prime},t)/\tau)}p. \tag{2}\]  
where  
\[g(x,p,t)=\frac{1}{\sqrt{2\pi}\sigma}\exp(\frac{p-\mathrm{argmax}_{p}s(x,p,t) }{\sigma^{2}}). \tag{3}\]  
the function \(g(x,p,t)\) represents a Gaussian kernel, with the standard deviation \(\sigma\) empirically set to 5. The temperature factor \(\tau\) is introduced to adjust the softmax distribution, and based on empirical observations in our experiments, \(\tau\) is set to 0.01. Additionally, a  
Figure 3: The Motion-Extraction Module (MEM). The module takes two RGB video frame-level features, \(\mathbf{F}^{t}\) and \(\mathbf{F}^{t+n}\), with an interval of \(n-1\), as input. Through correlation computation, it estimates motion changes and transforms them into the correlation tensor \(S^{(t)}\). This tensor is then used as input to estimate the displacement tensor \(D_{i}\) via a Gaussian kernel.  
confidence map of the correlation tensor is applied as supplementary motion information. This confidence map is generated by pooling the highest correlation at each position \(x\):  
\[s^{*}(x,t)=\max_{p}s(x,p,t). \tag{4}\]  
To represent sub-action motion features at different tempos, we define a displacement tensor \(\mathbf{D}_{i}\in\mathbb{R}^{H\times W\times 3}\), which is formed by concatenating the single-channel confidence map and the two-channel displacement map.