## 4 Experiments
### Ablation Study  
#### 4.3.1 Analysis of Multi-Scale Feature Fusion  
In our proposed model, we introduce the multi-scale feature fusion method with self-attention to capture more comprehensive information in sign language videos. We specifically choose \(3\times 3\) and \(7\times 7\) feature maps to represent coarse-grained and fine-grained features, respectively.  
To validate the effectiveness of using 3 \(\times\) 3 and 7 \(\times\) 7 feature maps for capturing coarse and fine details, we used class activation mapping (CAM) [49] to visualize the attention weights of feature maps at different scales, allowing us to observe the regions that the model focuses on at different feature scales. As shown in Figure 6, the 7 \(\times\) 7 feature maps capture finer details such as facial expressions and hand shapes. These fine-grained features are crucial for distinguishing subtle variations in sign language gestures, providing detailed information that is essential for precise recognition. In contrast, Figure 6 shows that the 3 \(\times\) 3 feature maps capture more coarse-level movements, such as overall body posture and hand trajectories. These coarse-grained features are essential for understanding the general structure and flow of sign language, providing a broader context that is necessary for recognizing overall gestures.  
Table 1 compares the performance of different feature selections on the PHOENIX-2014 benchmark. For fine-grained features, we only utilize the 7 \(\times\) 7 feature map with a receptive field of 31. For coarse-grained features, we use the 3 \(\times\) 3 feature map. The fusion process is shown in Figure 3. The results demonstrate that using the proposed feature fusion method can achieve better performance compared to using single features, with WER reductions of 4.7 and 10.5, respectively. This indicates that the feature fusion method enhances the spatial representation of the sign language videos and facilitates the model's understanding of sign language actions.  
The visualizations support our experimental findings: fine-grained features, although detailed, also include more noise, making them less effective when used alone. Coarse  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Spatial Features**} & \multicolumn{2}{c}{**Dev**} & \multicolumn{2}{c}{**Test**} \\ \cline{2-5}  & **del/ins** & **WER \(\downarrow\)** & **del/ins** & **WER \(\downarrow\)** \\ \hline Fine-grained features & 13.5/4.0 & 32.9 & 14.1/3.7 & 33.4 \\ Coarse-grained features & 12.0/3.6 & 27.1 & 11.8/3.6 & 27.0 \\ Fused features & **7.4/3.3** & **22.4** & **7.3/3.2** & **22.6** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Evaluation of various feature selections on PHOENIX-2014. The \(\downarrow\) means the lower the better.  
Figure 6: The CAM visualization of attention weights corresponding to feature maps at different scales. We applied this visualization to videos of different sign language performers to demonstrate the generalizability of the results. (**(A)** is sourced from ‘01 April_2010_Thursday_heute_default-1’ in the PHOENIX2014 validation set. (**B**) is sourced from ‘03 November_2010_Wednesday_tageschau_default-7’ in the PHOENIX-2014 validation set.)  
-grained features provide a clearer semantic representation but miss finer details. By combining both types of features, our model leverages the strengths of each, resulting in a more robust understanding of sign language videos. This comprehensive approach ensures that both the macro (coarse) and micro (fine) aspects of sign language are effectively captured, leading to improved recognition performance.