## 5 Experimental Results
In general, the performance difference on the validation set is not large but it is apparent that the VGGNet models are much better in generalizing to the test set compared to the ResNet models. Expectably, the translation models using the CNN extracted features show significantly worse translation performance than the models using the human keypoint features. It is well-known that CNN-based architectures such as ResNet and VGGNet have a huge number of trainable parameters (e.g., VGGNet-19 and ResNet-152 have over 143M and 60M parameters, respectively.) so that they easily fall into the overfitting problem due to the lack of a sufficient number of training examples. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Feature Type**} & \multicolumn{4}{c}{**Validation Set**} & \multicolumn{4}{c}{**Test Set**} \\ \cline{2-9}  & **ROUGE-L** & **METEOR** & **BLEU** & **CIDEr** & **ROUGE-L** & **METEOR** & **BLEU** & **CIDEr** \\ \hline VGGNet-16 [51] & 66.85 & 41.92 & 56.75 & 2.369 & **44.75** & **25.79** & **27.88** & **1.016** \\ VGGNet-19 [51] & 61.77 & 38.72 & 50.95 & 2.060 & 42.75 & 24.81 & 24.27 & 0.839 \\ ResNet-50 [50] & 62.26 & 38.79 & 51.99 & 2.124 & 38.76 & 21.85 & 19.45 & 0.664 \\ ResNet-101 [50] & 66.28 & 41.81 & 56.26 & 2.368 & 40.10 & 22.68 & 21.86 & 0.772 \\ ResNet-152 [50] & **74.10** & **48.03** & **66.73** & **2.841** & 38.44 & 22.71 & 20.78 & 0.753 \\ \hline OpenPose [32; 33; 34] & 96.92 & 72.14 & 96.11 & 4.380 & 73.95 & 46.66 & 64.79 & 2.832 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance comparison with translation models based on CNN-based feature extraction techniques. Note that the augmentation factor in this experiment was all set to 50. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Annotation**} & \multicolumn{4}{c}{**Validation Set**} & \multicolumn{4}{c}{**Test Set**} \\ \cline{2-9}  & **Accuracy** & **ROUGE-L** & **METEOR** & **BLEU** & **Accuracy** & **ROUGE-L** & **METEOR** & **BLEU** \\ \hline Sentence-level & 82.07 & 94.42 & 67.35 & 90.57 & 45.56 & **66.10** & **41.09** & **57.37** \\ Gloss-level & **93.28** & **96.03** & **71.04** & **93.85** & **55.28** & 63.53 & 38.10 & 52.63 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of sign language translation performance on different types of annotations. Note that bold numbers represent the best performance results. Moreover, the CNN-based models have a weakness for recognizing signs of previously unseen signers as they are weaker than our model in dealing with subtle variances of images. It is still interesting to know whether the combination of any CNN-based features and human keypoint features works better than when we solely rely on the human keypoint features. As the size of sign language dataset grows, we expect that the CNN-based models improve their performances and generalize much better. **Effect of feature normalization methods. ** To evaluate the effect of the feature normalization method on the keypoints estimated by OpenPose, we compared the following five cases: (1) no normalization; (2) feature normalization; (3) object normalization; (4) two-dimensional (2D) normalization; and (5) object 2D normalization. In the first case, we did not perform any normalization step on the keypoint feature generated by concatenating the coordinate values of all keypoints. In the feature normalization, we created a keypoint feature as in (1) and normalized the feature with the mean and standard deviation of the whole feature. In the object normalization, we normalized the keypoint features obtained from two hands, body, and face, respectively, and concatenated them to generate a feature that represents the frame. We also considered the case of 2D normalization in which we normalized the \(x\)- and \(y\)-coordinates separately. Lastly, the object 2D normalization is the normalization method that we propose in the paper. Table 5 summarizes the result of our experiments.