## 4 Experiments
### Results and Discussion
#### 4.4.2 VSR  
Recognition accuracy of our and competitive VSR schemes is shown in Table 4. We firstly tested our models with and without view classification. Our method with the view classification part achieved almost the same or better performance, compared to ours without the classification, in which the classification result was correctly given. This indicates our feature extraction and recognition strategy can perform well.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & **Data** & **0\({}^{\circ}\)** & **30\({}^{\circ}\)** & **45\({}^{\circ}\)** & **60\({}^{\circ}\)** & **90\({}^{\circ}\)** & **Mean** \\ \hline CNN + Data Augmentation [3] & 85.6 & 82.5 & 82.5 & 83.3 & 80.3 & 82.84 \\ PCA + LSTM + GMM–HMM [6] & 73.1 & 75.6 & 67.2 & 63.3 & 59.3 & 67.7 \\ View2 View [9] & - & 86.11 & 83.33 & 81.94 & 78.89 & 82.57 \\ End-to-end Encoder + BLSTM [11] & **91.8** & 87.3 & 88.8 & 86.4 & **91.2** & 89.1 \\ End-to-End CNN–LSTM [14] & 82.8 & 81.1 & 85.0 & 83.6 & 86.4 & 83.78 \\ \hline Ours without view classification & 91.02 & 90.56 & 91.20 & 90.00 & 88.88 & 90.33 \\ Ours with view classification & 91.02 & **91.38** & **92.21** & **90.09** & 88.88 & **90.65** \\ \hline \hline \end{tabular}
\end{table}
Table 4: VSR accuracy (%) of our proposed method and conventional schemes.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Noise** & **SNR** & **0 dB** & **5 dB** & **10 dB** & **15 dB** & **20 dB** \\ \hline Kitchen & 1050 & - & 1050 & - & 1050 \\ \hline Park & - & 1050 & - & 1050 & - \\ \hline Office & 1050 & - & 1050 & - & 1050 \\ \hline Station & - & 1050 & - & 1050 & - \\ \hline Car & 1050 & - & 1050 & - & 1050 \\ \hline \hline \end{tabular}
\end{table}
Table 2: The amount of acoustically noisy data for ASR training: e.g., 1050 = 35 (speakers) \(\times\) 10 (utterances) \(\times\) 3 (times).  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Result** & **Label** & **0\({}^{\circ}\)** & **30\({}^{\circ}\)** & **45\({}^{\circ}\)** & **60\({}^{\circ}\)** & **90\({}^{\circ}\)** \\ \hline
0\({}^{\circ}\) & 360 & 0 & 0 & 0 & 0 \\
30\({}^{\circ}\) & 0 & 324 & 18 & 0 & 0 \\
45\({}^{\circ}\) & 0 & 36 & 242 & 0 & 0 \\
60\({}^{\circ}\) & 0 & 0 & 100 & 359 & 0 \\
90\({}^{\circ}\) & 0 & 0 & 0 & 1 & 360 \\ \hline \hline \end{tabular}
\end{table}
Table 3: A confusion matrix of view classification results.  
Next, we compared our approach with conventional methods. Focusing on the average of recognition accuracy, our proposed method achieved the highest accuracy regardless of the presence or absence of the view classification part. It is interesting that at 45\({}^{\circ}\) we found much more improvement than in the other conditions, and even the view classification performance was insufficient. Since 45\({}^{\circ}\) data were used as training data in the neighboring 30\({}^{\circ}\) and 60\({}^{\circ}\) conditions, we might obtain such an improvement even if the view classification fails. We also found that our method was particularly effective in the medium-angle (30\({}^{\circ}\), 45\({}^{\circ}\) and 60\({}^{\circ}\)) conditions, while the end-to-end system had higher accuracy for frontal and profile images.  
Figure 5 indicates F1 scores for each angle. Among all the angles, it is found that shorter utterances were relatively hard to classify, because there were fewer cues for recognition.