# Lip2 Speech: Lightweight Multi-Speaker Speech Reconstruction with Gabor Features
## 6 Results
### Multi-Speaker Model Results  
Multi-speaker speech reconstruction is important for evaluating model independence. Our proposed models are compared with the baseline, using the datasets described in Section 5.1, labelled as 1-S, 2-S, 4-S, 6-S, and 8-S. To validate performance, 100 samples from speaker S27 were randomly selected and the reconstructed speech from this speaker evaluated in various multi-speaker scenarios. The results are shown in Table 6 and Figure 14. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Measure**} & \multicolumn{4}{c}{Lip2 AndSpec} & \multicolumn{4}{c}{**GaborCNN2 Speech**} & \multicolumn{4}{c}{**GaborFea2 Speech**} \\ \cline{2-13}  & **1-S** & **2-S** & **4-S** & **6-S** & **8-S** & **1-S** & **2-S** & **4-S** & **6-S** & **8-S** & **1-S** & **2-S** & **4-S** & **6-S** & **8-S** \\ \hline Corr2D & 60.70\% & 39.22\% & 39.28\% & 33.05\% & 29.55\% & **66.18\%** & **64.67\%** & 51.47\% & 49.91\% & 46.18\% & 64.77\% & 61.26\% & **63.00\%** & **58.17\%** & **54.52\%** \\ PES & 1.627 & 1.414 & 1.458 & 1.476 & 1.431 & 1.696 & 1.697 & 1.591 & 1.703 & **1.699** & **1.739** & **1.714** & **1.800** & **1.778** & 1.684 \\ COSO & 0.071 & -0.155 & -0.304 & -0.170 & -0.384 & 0.300 & 0.243 & -0.049 & 0.156 & -0.042 & **0.302** & **0.252** & **0.221** & **0.198** & -0.031 \\ STOI & 0.242 & 0.213 & 0.244 & 0.260 & 0.251 & 0.269 & 0.283 & 0.285 & 0.339 & 0.315 & **0.278** & **0.287** & **0.330** & **0.355** & **0.320** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Speech reconstruction results from multiple speaker models. Figure 13: Comparative analysis of spectrogram reconstruction using proposed and baseline models. (**a(I)-a(3)**) show the original spectrograms. Reconstructed spectrograms from GaborFea2 Speech and GaborCNN2 Speech are displayed in (**b(1)-b(3)**) and (**c(1)-c(3)**), respectively. (**d(1)–d(3)**) show the Lip2 AudSpec spectrograms. Table 6 shows that GaborFea2 Speech consistently outperforms Lip2 AudSpec and GaborCNN2 Speech in all speaker scenarios. Under the Corr2D measure, GaborCNN2 Speech achieves the highest percentage (66.18%) in the 1-S scenario, followed by GaborFea2 Speech (64.77%). However, in the 2-S scenario, GaborFea2 Speech outperforms both Lip2 AudSpec and GaborCNN2 Speech with a percentage of 61.26%. In terms of PESQ, GaborFea2 Speech achieves the highest overall score (1.739), closely followed by GaborCNN2 Speech (1.696). GaborFea2 Speech consistently outperforms the other methods in terms of OOSQ, exhibiting the highest values across all scenarios. Similarly, under the STOI measure, GaborFea2 Speech demonstrates the highest scores, indicating superior speech intelligibility compared to Lip2 AudSpec and GaborCNN2 Speech. Figure 14(1-4) shows the average accuracy (Corr2D), PESQ scores, objective overall speech quality (OOSQ), and STOI measurement of the reconstructed speech for speaker S27 using models trained with different numbers of speakers. The results show the superior performance of the proposed GaborFea2 Speech and GaborCNN2 Speech models. Notably, the GaborFea2 Speech model consistently outshines the other models across all metrics and scenarios. The results show that with all datasets and metrics, our GaborFea2 Speech model delivers not only the best performance with multiple speakers but is also notably more stable with different datasets. There is a consistent pattern of both our proposed models outperforming the CNN model, even when trained with large datasets. To further compare performance, a sample sentence was randomly selected (\"bin blue by f 8 now\") from speaker S27 and the spectrogram and waveform analysed to visualise the change across various multi-speaker scenarios. Figure 15(0) shows the original spectrogram and waveform. Figure 15(1)-a(5) presents the reconstructed spectrograms  
Figure 14: Comparative analysis of speech reconstruction for Speaker S27: GaborFea2 Speech and GaborCNN2 Speech models versus Lip2 AudSpec with different datasets(1-S, 2-S, 4-S, 6-S, 8-S).