# Domain Adaptation Speech-to-Text for Low-Resource European Portuguese Using Deep Learning
## 2 Literature Review
### Deep Learning in ASR  
Recent years have seen a growth in knowledge about the application of ANNs in a variety of research areas, including ASR, computer vision (image processing and recognition), and natural language processing. End-to-end (E2E) methods for ASR systems have increased in popularity due to these developments and the exponential growth in the amount of data available [7].  
Approaches to automatic speech recognition systems are largely based on four types of deep neural networks: convolutional neural networks (CNNs) [8]; recurrent neural networks (RNNs) [9]; time-delay neural networks (TDNNs) [10]; and most recently, transformers [11]. These architectures can employ E2E mechanisms such as attention-based encoders and decoders (AED), recurrent neural network transducer (RNN-T), and connectionist temporal classification (CTC) [7].  
The CTC algorithm is commonly used in ASR, as well as handwriting and other problems, the output of which consists of a sequence. When speech data only contain the audio and the transcript, there is no direct correlation between them. The correct alignment is possible by applying the CTC loss function on a DNN. This decoding process of aligning speech to words makes use of a tokenizer that contains the vocabulary (or alphabet) of the target audio language.  
The Jasper model architecture is a good example of this [12]. It uses a TDNN architecture trained with CTC loss. This model achieved best performance of 2.95% WER on a test clean LibriSpeech dataset and improved over various domains compared to other state-of-the-art models.  
Many applications in ASR infer pre-trained models from APIs due to their performance for the English language, as shown in Table 1.  
These models can also be fine-tuned to other languages if given new vocabulary and speech data. This process is also known as transfer learning (TL).  
These types of DNNs require large quantities of data to be trained and achieve good performance. When the available data are scarce, the transfer learning technique can be  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Architecture** & **Train Hours** \\ \hline Facebook wav2 vec 2.0 & 1041 \\ Google LAS & 2025 \\ Microsoft ASR System & 12,500 \\ \hline \hline \end{tabular}
\end{table}
Table 1: API architecture and respective training hours.  
applied to these networks to improve their performance. Transferring knowledge on ANN-based systems is the equivalent of reusing layers from previously trained models. This is accomplished by using previously calculated weights to initialize specified layers of new models, followed by training the remaining layers. The reused layers can either be fixed, in which case the pre-calculated weights will not be updated, or flexible, in which case the pre-calculated weights will be able to be updated according to the new data [13]. The weights of the remaining layers are randomly initialized as in a normal ANN.  
[PERSON] et al. [14] evaluated the APIs from Table 1 using two collaborative and public Portuguese datasets, Mozilla Common Voice ([[https://commonvoice.mozilla.org/pt](https://commonvoice.mozilla.org/pt)]([https://commonvoice.mozilla.org/pt](https://commonvoice.mozilla.org/pt))) (accessed on 11 November 2021) and the Voxforge ([[http://www.voxforge.org/pt](http://www.voxforge.org/pt)]([http://www.voxforge.org/pt](http://www.voxforge.org/pt))) (accessed on 11 November 2021). Each had a domain, the former being common words and utterances, while the latter was audiobooks. The result of each API over each dataset can be seen in Table 2.  
The model can adapt to new languages or domains if given enough training data for it to transfer to new vocabularies. Transferring knowledge from a high-resource language to a low-resource language, such as Portuguese, has been shown to improve the low-resource ASR model [15; 16].