# Lip2 Speech: Lightweight Multi-Speaker Speech Reconstruction with Gabor Features
## 6 Results
### Individual Word Results  
In addition to complete sentences, GaborFea2 Speech and GaborCNN2 Speech performance with individual words was compared. The vocabulary from 1000 sentences of Grid Speaker 1 was extracted, which encompassed 6 categories and 51 distinct words, representative of the full set of vocabulary found in the Grid Corpus. The words were trained using GaborFea2 Speech and GaborCNN2 Speech models. Table 7 provides a comparison of GaborFea2 Speech and GaborCNN2 Speech across different word categories. The results indicate that GaborFea2 Speech consistently outperforms GaborCNN2 Speech in terms of accuracy, speech quality, overall quality, and intelligibility. Notably, adverbs were found to have the highest accuracy and speech quality, while letters had the best speech intelligibility based on GaborFea2 Speech. GaborCNN2 Speech demonstrates competitive performance in the command category for accuracy and the colour category for perceptual quality and intelligibility. These results highlight GaborFea2 Speech's superior performance in various aspects of word reconstruction. This indicates its effectiveness in capturing and reconstructing speech from different word categories. This is also shown in the radar graph in Figure 16.  
Figure 16 shows the accuracy of the reconstructed spectrograms for both methods across a predefined set of 51 words. Each spoke corresponds to a specific word, while the distance from the centre represents the Corr2D accuracy metric. The methods are differentiated by different colours, as shown in the legend. The results show that GaborFea2 Speech achieves higher overall accuracy than GaborCNN2 Speech across the word categories. Specifically, it exhibits superior performance in vocabulary such as \"B\", \"O\", and \"V\", while GaborCNN2 Speech performs more profitably in vocabulary like \"Y\", \"P\", and \"set\". Furthermore, certain vocabularies such as letters (D, G, H, J, Q, U and Y), digits (1, 2, 5 and 7), colours (white and red), prepositions (in) and adverbs (please) consistently perform better in both GaborCNN2 Speech and GaborFea2 Speech. Conversely, words such as \"F\", \"I\", \"X\", \"Z\", \"lay\" and \"soon\", demonstrate relatively lower performance in both methods. These limitations may be attributed to the limited visual variations that occur during the pronunciation of these vocabularies, which make it difficult for speech reconstruction software to capture subtle visual changes.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Measure** & **Model** & **Command** & **Colour** & **Preposition** & **Letter** & **Digit** & **Adverb** & **Average** \\ \hline Corr2D & GaborFea2 Speech & 79.15\% & 79.15\% & 76.15\% & 77.25\% & 80.50\% & **80.56\%** & **78.79\%** \\  & GaborCNN2 Speech & **73.32\%** & 72.85\% & 67.03\% & 69.52\% & 71.0\% & 73.01\% & 71.08\% \\ \hline PESQ & GaborFea2 Speech & **1.32** & 1.27 & 1.27 & 0.86 & 1.23 & 1.28 & **1.21** \\  & GaborCNN2 Speech & 0.80 & **0.82** & 0.75 & 0.75 & 0.78 & 0.81 & 0.79 \\ \hline OOSQ & GaborFea2 Speech & 0.29 & 0.31 & 0.26 & 0.26 & 0.34 & **0.45** & **0.32** \\  & GaborCNN2 Speech & 0.27 & 0.27 & 0.26 & 0.27 & 0.31 & **0.3** & 0.29 \\ \hline STOI & GaborFea2 Speech & 0.35 & 0.41 & 0.44 & 0.38 & **0.50** & 0.45 & 0.42 \\  & GaborCNN2 Speech & 0.39 & 0.44 & 0.43 & 0.39 & **0.48** & 0.43 & **0.43** \\ \hline \hline \end{tabular}
\end{table}
Table 7: Speech reconstruction from vocabulary: evaluating GaborFea2 Speech and GaborCNN2 Speech Networks on word category.