# Lip2 Speech: Lightweight Multi-Speaker Speech Reconstruction with Gabor Features
## 6 Results
### Speed and Memory Performance  
One key motivation for using Gabor features is the resulting lightweight models. Fast and lightweight models are valuable in scenarios demanding quick response times and minimal resource consumption. Evaluating training time and memory usage is valuable for assessing a model's lightweight and speed attributes. These metrics directly influence the practicality of model deployment, especially in resource-constrained environments. To compare the training time and memory usage, the GaborFea2 Speech, GaborCNN2 Speech, and Lip2 AudSpec models were trained across various speaker scenarios. Each model was trained repeatedly in the same speaker scenario to calculate the mean and Interquartile Range (IQR) values for both training time and memory usage. The mean provides a central tendency of the data, while the IQR offers insights into the variability and stability of the model performance. This training did not include time for pre-processing, and focuses only on model training performance. The results, as outlined in Table 8, reveal insightful contrasts in the training time and memory usage among the models, thereby allowing an evaluation of their lightweight and speed. Due to unavailability of the server used for other experiments, a different machine was used for these tests, using an Intel Xeon w3-2423 processor, 128 GB of RAM, and a NVidia RTX A5000 GPU. This improved training time for all models over the initial experiments. We calculated the mean of five runs of each configuration. These results demonstrate the advanced lightweight and speed capabilities of our proposed GaborFea2 Speech model, showcasing its superiority over the other models in  
Figure 16: Radar graph analysis of GaborCNN2 Speech and GaborFea2 Speech in vocabulary-based speech reconstruction. \begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline
**Model** & \multicolumn{2}{c}{**1-S**} & \multicolumn{2}{c}{**2-S**} & \multicolumn{2}{c}{**4-S**} & \multicolumn{2}{c}{**6-S**} & \multicolumn{2}{c}{**8-S**} \\  & **AVG** & **IQR** & **AVG** & **IQR** & **AVG** & **IQR** & **AVG** & **IQR** & **AVG** & **IQR** \\ \hline \hline \multicolumn{10}{c}{**Training Time (Seconds)**} \\ GaborFea2 Speech & **6.0** & 1.1 & **6.4** & 0.2 & **9.0** & 0.7 & **11.4** & **0.4** & **13.6** & **0.1** \\ GaborCNN2 Speech & 139.4 & **0.4** & 276.5 & 0.2 & 548.4 & 2.8 & 825.4 & 1.4 & 1102.7 & 1.8 \\ Lip2 AudSpec & 53.7 & 0.9 & 103.0 & **0.1** & 204.7 & **0.2** & 308.2 & 1.5 & 413.2 & 0.8 \\ \hline \multicolumn{10}{c}{**Memory Usage (MB)**} \\ GaborFea2 Speech & **2572.1** & **52.8** & **2629.1** & **23.0** & **2685.9** & **21.1** & **2739.0** & **25.5** & **2797.2** & **19.4** \\ GaborCNN2 Speech & 6042.6 & 319.9 & 7271.7 & 80.2 & 9334.6 & 204.1 & 12,047.3 & 551.8 & 14,917.3 & 92.3 \\ Lip2 AudSpec & 6241.6 & 456.6 & 7805.7 & 97.0 & 10,236.4 & 153.3 & 13,219.4 & 375.2 & 15888.1 & 645.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Comparison of training time and memory usage. terms of both training efficiency and memory optimization. It can be immediately seen that the GaborFea2 Speech approach uses a fraction of the memory and training time, thanks to the use of lower dimensionality features and no requirement for CNN layers. Even increasing the training set size to the full set of eight speakers did not make a notable impact. Conversely, the GaborCNN2 Speech approach performed much worse than expected. Less memory was required than the CNN-based approach, but the training time was almost three times slower. An investigation identified that although less CNN layers were used, the consequence was that with less pooling and abstraction, the input into the LSTM layer became much larger, resulting in a much slower training time. The significantly lower average training times and memory usage across various speaker scenarios underscore the GaborFea2 Speech model's suitability for efficient real-time speech reconstruction, particularly in environments where computational resources are limited.