# Domain Adaptation Speech-to-Text for Low-Resource European Portuguese Using Deep Learning
## 5 Experiments and Results
### Experiments and Results
#### 5.3.2 Transfer Learning  
Being that Portuguese is a low-resource language, experiments using both the Speech-Dat and MLS datasets have been developed using the transfer learning approach.  
In the current experiments, various combinations of both MLS and SpeechDat were used to develop models. The 100 h \"clean\" training set (ENG) from the LibriSpeech dataset was used to control the performance of the transfer technique on the English domain. NVIDIA's NeMo provides a model for the English language pre-trained with a dataset with \(\simeq\)3300 h of audio recordings ([[https://developer.nvidia.com/blog/jump-start-training-for-speech-recognition-models-with-nemo/](https://developer.nvidia.com/blog/jump-start-training-for-speech-recognition-models-with-nemo/)]([https://developer.nvidia.com/blog/jump-start-training-for-speech-recognition-models-with-nemo/](https://developer.nvidia.com/blog/jump-start-training-for-speech-recognition-models-with-nemo/))) (accessed on 23 January 2022). This model was used as the starting point for the transfer learning experiments. Table 10 shows the results of the transfer learning experiments to compare the performance of models developed from scratch and using transfer learning.  
Based on the previous conclusions, further experiments were conducted by mixing SpeechDat as defined in Definition 2 and the MLS (in these experiments \"MLS\" represents \"PT + BR\", both training and testing subsets) transfer and validation subsets. The goal of these experiments is to evaluate the impact of data quality and quantity on the performance of the models.  
The proportion of each subset is defined by a linear variation between 0% and 100% with a step of 25% (Table 11). This is carried out so that the number of instances remains constant across all mixes, 107,158 for training and 22,960 for validation.  
For each mix of the datasets, three models were developed and evaluated. Table 12 presents a summarized view of the average performance of each mix.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Train/Validation** & **Transfer** & **Test** & **WER** \\ \hline \multirow{8}{*}{Pre-trained ENG} & PT & PT & 1.0164 \\  & PT & BR & 0.9722 \\  & BR & PT & 0.7075 \\  & BR & BR & 0.5139 \\  & PT + BR & BR & 0.5025 \\  & PT + BR & PT + BR & 0.5083 \\ \cline{2-4}  & \multirow{4}{*}{28,123} & SpeechDat & 0.1603 \\  & & ENG & 1.0186 \\  & & PT + BR & 0.7912 \\ \cline{2-4}  & \multirow{4}{*}{28,123} & SpeechDat & 0.0557 \\  & & ENG & 1.006 \\ \cline{1-1}  & & PT + BR & 0.7680 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Performance of models developed using transfer learning with the MLS subsets and SpeechDat, according to Definitions 1 and 2.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**MIX ID** & \multicolumn{2}{c}{**Transfer**} & \multicolumn{2}{c}{**Validation**} \\  & **MLS** & **SpeechDat** & **MLS** & **SpeechDat** \\ \hline
0.00 & 0 & 107,158 & 0 & 22,960 \\
0.25 & 9374 & 97,784 & 206 & 22,754 \\
0.50 & 18,749 & 88,409 & 413 & 22,547 \\
0.75 & 28,123 & 79,035 & 619 & 22,341 \\
1.00 & 37,498 & 69,660 & 826 & 22,134 \\ \hline \hline \end{tabular}
\end{table}
Table 11: Audio instances from each dataset used on each training and validation mix.  
The performance of the models developed in these experiments was evaluated using the previously used test sets, SpeechDat, MLS (\"PT + BR\"), and LibriSpeech (ENG). Additionally, a full combination of the SpeechDat and MLS test sets (SpeechDat + MLS) was also used to evaluate the model's performance. Table 13 displays the number of audio instances present in each of the mentioned test sets.