# Efficient DNN Model for Word Lip-Reading
## 5 Evaluation Experiment
### Sssd  
SSSD [15] consists of 25 utterances, comprising 10 Japanese numeric words and 15 greetings ([[https://www.saitoh-lab.com/SSSD/index_en.html](https://www.saitoh-lab.com/SSSD/index_en.html)]([https://www.saitoh-lab.com/SSSD/index_en.html](https://www.saitoh-lab.com/SSSD/index_en.html)), accessed on 26 April 2023). The 25 words are (1) /ze-ro/ (zero), (2) /i-chi/ (one), (3) /ni/ (two), (4) /sa-N/ (three), (5) /yo-N/ (four), (6) /go/ (five), (7) /ro-ku/ (six), (8) /na-na/ (seven), (9) /ha-chi/ (eight), (10) /kyu/ (nine), (11) /a-ri-ga-to-u/ (thank you), (12) /i-i-e/ (no), (13) /o-ha-yo-u/ (good morning), (14) /o-me-de-to-u/ (congratulation), (15) /o-ya-su-mi/ (good night), (16) /go-me-N-na-sa-i/ (I am sorry), (17) /ko-N-ni-chi-wa/ (good afternoon), (18) /ko-N-ba-N-wa/ (good evening), (19) /sa-yo-u-na-ra/ (goodbye), (20) /su-mi-ma-se-N/ (excuse me), (21) /do-u-i-ta-shi-ma-shi-te/ (you are welcome), (22) /ha-i/ (yes), (23) /ha-ji-me-ma-shi-te/ (nice to meet you), (24) /ma-ta-ne/ (see you), and (25) /mo-shi-mo-shi/ (hello).  
\begin{table}
\begin{tabular}{l r} \hline \hline
**Model** & **Top-1 Acc. (\%)** \\ \hline Multi-Tower 3D-CNN [4] & 91.4 \\ AE + GRU [43] & 81.2 \\ FOMM \(\rightarrow\) AE + GRU [7] & 86.5 \\ [MF + AE + AU] + GRU [5] & 86.6 \\ \hline
3D-Conv + ResNet18 + MS-TCN (ours) & 90.1 \\
3D-Conv + ResNet18 + MS-TCN + RA (ours) & 93.1 \\
3D-Conv + ResNet18 + MS-TCN + FT (ours) & 95.1 \\
3D-Conv + ResNet18 + MS-TCN + RA + FT (ours) & 97.2 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Recognition results (OuluVS).  
\begin{table}
\begin{tabular}{l r} \hline \hline
**Model** & **Top-1 Acc. (\%)** \\ \hline AE + GRU [43] & 72.8 \\ FOMM \(\rightarrow\) AE + GRU [7] & 79.8 \\ [MF + AE + AU] + GRU [5] & 83.4 \\
3D-CNN (ours) & 84.4 \\ \hline
3D-Conv + ResNet18 + MS-TCN (ours) & 87.6 \\
3D-Conv + ResNet18 + MS-TCN + RA (ours) & 90.0 \\
3D-Conv + ResNet18 + MS-TCN + FT (ours) & 93.7 \\
3D-Conv + ResNet18 + MS-TCN + RA + FT (ours) & 94.1 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Recognition results (CUAVE).  
Unlike OuluVS, CUAVE, and LRW, SSSD is filmed using a smart device. An image of the lower half of a face of 300 \(\times\) 300 pixels extracted after normalization processing is applied for scale, and rotation is provided. The frame rate is 30 fps. The number of provided scenes is 72 speakers \(\times\) 25 words \(\times\) 10 samples = 18,000 scenes. As a competition using SSSD, the second machine lip-reading challenge was held in 2019, and 5000 scenes of test data were released.  
For the accuracy evaluation, we used 18,000 scenes as training data and an extra 5000 scenes as test data, using the same task as the second machine lip-reading challenge to obtain recognition accuracy. The results are shown in Table 6. \(N_{RA}\) gave 3 like OuluVS and CUAVE. From the table, 3D-Conv + ResNet18 + MS-TCN + FT obtained the highest recognition accuracy of 95.14%. While OuluVS and CUAVE obtained high recognition accuracy when RA was applied, SSSD obtained the highest recognition rate when RA was not applied. We presume that SSSD has more training data than OuluVS and CUAVE and can train sufficiently without applying RA.