# Identification of 3D Lip Shape during Japanese Vowel Pronunciation Using Deep Learning
## 4 Discussion  
Tables 6 and 7 show that the accuracy rate of k-fold cross-validation exceeded 70% in both the mixed-gender and male-only experiments. As shown in Tables 8 and 9, the accuracy of the test data identification results was 82.67% for the mixed-gender experiment and 85.20% for the male-only experiment. From the training results using 2D images (the conventional method: Tables 10 and 11) and those from 3D point clouds (the proposed method: Tables 6 and 7), it can be seen that in the mixed results, accuracy, recall, and F-measure decreased by 2.945%, 3.835%, and 3.458%, respectively, while precision increased by 0.404% in accuracy compared to the conventional method. The male-only results show that the accuracy, precision, recall, and F-measure decreased by 2.538%, 0.487%, 2.795%, and 2.689%, respectively. The total parameters of the Xception model used in the experiment was 22,960,173, while the total number of parameters of the proposed method was 1,645,385. The total parameters of the proposed method was 92.834% less than that of the conventional method. The proposed method uses only the coordinates of the point cloud and does not use pixel information, while the conventional method uses image coordinates and pixel information. Therefore, the input information was reduced, contributing to a reduction in the number of parameters. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Fold No.** & **Accuracy** & **Precision** & **Recall** & **F-Measure** \\ \hline
1 & 0.600 & 0.630 & 0.600 & 0.609 \\
2 & 0.720 & 0.734 & 0.720 & 0.720 \\
3 & 0.760 & 0.801 & 0.760 & 0.758 \\
4 & 0.800 & 0.835 & 0.800 & 0.797 \\
5 & 0.800 & 0.803 & 0.807 & 0.791 \\
6 & 0.520 & 0.425 & 0.500 & 0.457 \\
7 & 0.840 & 0.863 & 0.847 & 0.842 \\
8 & 0.680 & 0.605 & 0.660 & 0.597 \\
9 & 0.720 & 0.787 & 0.720 & 0.710 \\
10 & 0.900 & 0.920 & 0.900 & 0.898 \\ \hline Average & 0.734 & 0.740 & 0.731 & 0.718 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Accuracy, precision, recall, and F-measure of Xception model (mixed-gender). \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & & **Predicted** & & \\ \hline  & \multicolumn{2}{c}{\"A\"} & \multicolumn{1}{c}{\"I\"} & \multicolumn{1}{c}{\"U\"} & \multicolumn{1}{c}{\"E\"} & \multicolumn{1}{c}{\"O\"} \\ \hline  & \multicolumn{1}{c}{\"A\"} & 46 & 0 & 0 & 4 & 0 \\  & \multicolumn{1}{c}{\"I\"} & 0 & 44 & 5 & 1 & 0 \\ Actual & \multicolumn{1}{c}{\"U\"} & 0 & 5 & 45 & 0 & 0 \\  & \multicolumn{1}{c}{\"E\"} & 2 & 2 & 1 & 40 & 5 \\  & \multicolumn{1}{c}{\"O\"} & 0 & 3 & 5 & 4 & 38 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Confusion matrix using test data (male-only). \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Fold No.** & **Accuracy** & **Precision** & **Recall** & **F-Measure** \\ \hline
1 & 0.800 & 0.817 & 0.810 & 0.803 \\
2 & 0.700 & 0.677 & 0.700 & 0.685 \\
3 & 0.850 & 0.853 & 0.850 & 0.848 \\
4 & 0.650 & 0.653 & 0.650 & 0.648 \\
5 & 0.850 & 0.867 & 0.850 & 0.846 \\
6 & 0.800 & 0.853 & 0.800 & 0.782 \\
7 & 0.850 & 0.870 & 0.843 & 0.848 \\
8 & 0.850 & 0.880 & 0.853 & 0.838 \\
9 & 0.800 & 0.843 & 0.800 & 0.793 \\
10 & 0.933 & 0.950 & 0.933 & 0.931 \\ \hline Average & 0.808 & 0.826 & 0.809 & 0.802 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Accuracy, precision, recall, and F-measure of Xception model (male-only). In addition, by generating a training model for vowel identification using a 3D point cloud, it is possible to maintain the order independence that provides degrees of freedom in the input array of point clouds. A point cloud of only the lip region can extract lip shape features with a few parameters and achieve 3.835% accuracy, as compared to conventional methods. In machine learning methods, models with a huge number of parameters have been proposed to improve accuracy, and research has also been conducted to reduce the number of parameters and decrease the computational cost [22].