## 2 Materials and Methods
### R3D-MHA Network Architecture  
The proposed R3D-MHA model has a modular structure, composed of the R3D module and the multi-head attention mechanism module. The first part involves using the R3D module as the backbone network to extract spatiotemporal features from video sequences. The R3D model is a 3D-CNN extended from the ResNet architecture, with a structure similar to 2D ResNet but replacing 2D convolutions with 3D convolutions, and using a \(1\times 3\times 3\) convolutional kernel in the temporal dimension. As shown in Figure 3, the main structure of the R3D network includes an STConv layer, four STResLayer layers, and an STPool layer. The STConv layer uses a standard 3D convolution module to extract features with a kernel size of \(3\times 7\times 7\). The STResLayer is composed of a basic residual module, as depicted in Figure 3, which consists of two 3D convolutional layers, each followed by a BatchNorm layer and a Softplus layer. Additionally, the STPool layer uses AdaptiveAvgPool3d to dynamically adjust the pooling layer size. Notably, in this study, all activation functions in R3D were replaced with Softplus instead of ReLU, enabling more effective weight updates and improving the model's generalization capability.  
The second part uses a multi-head attention mechanism module to further process the extracted features. It first performs three linear transformations on the input feature data, mapping the input (X) into matrices representing the query (Q), key (K) and value (V), respectively. Then, according to the following Equation (1), it calculates the attention weights for each head. \(Q_{i}=QW_{i}^{Q},K_{i}=KW_{i}^{K},V_{i}=VW_{i}^{V}\), where (\(d_{k}\)) is the dimension of the attention head, and (\(W_{i}^{Q},~{}W_{i}^{K},W_{i}^{V}\)) are the weight matrices learned by this head. Finally, multiple attention heads are used to compute attention weights in parallel to obtain multiple output results. These outputs are then linearly transformed and concatenated as  
Figure 3: R3D model structure.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Gesture** & **G1** & **G2** & **G3** & **G4** & **G5** & **G6** & **G7** & **G8** & **G9** & **G10** \\ \hline Instances & 168 & 163 & 220 & 218 & 219 & 203 & 326 & 218 & 217 & 665 \\ Overlapping segments & 1495 & 1347 & 2376 & 4144 & 3086 & 1490 & 2493 & 17,173 & 2857 & 13,846 \\ Mean frame count & 33 & 28 & 45 & 108 & 75 & 18 & 23 & 432 & 59 & 110 \\ Min frame count & 3 & 5 & 6 & 21 & 11 & 2 & 3 & 99 & 26 & 3 \\ Max frame count & 314 & 97 & 170 & 356 & 297 & 73 & 135 & 1435 & 216 & 593 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Data related to surgical gestures.  
shown in Equation (2) to obtain the final multi-head attention representation. Figure 4 is the schematic diagram of the multi-head attention mechanism.  
\[head_{i}=Attention(Q_{i},K_{i},V_{i})=softmax\left(\frac{Q_{i}K_{i}^{T}}{\sqrt{d_{ k}}}\right)V_{i} \tag{1}\]  
\[MultiHead(Q,K,V)=Concat(head_{1},head_{2},...,head_{h})W^{O} \tag{2}\]  
In summary, the two modules that constitute the R3D-MHA model are responsible for extracting spatiotemporal features from video sequences and performing global processing. By integrating these methods, the model can fully leverage the strengths of both modules, enhancing its capability to capture and process features from video sequences.