# Evaluating Novel Speech Transcription Architectures on the Spanish RTVE2020 Database +
Footnote †: This work presents three novel speech recognition architectures evaluated on the Spanish RTVE2020 dataset, employed as the main evaluation set in the Albayzin S2T Transcription Challenge 2020. The main objective was to improve the performance of the systems previously submitted by the authors to the Challenge, in which the primary system scored the second position. The novel systems are based on both DNN-HMM and E2E acoustic models, for which fully- and self-supervised learning methods were included. As a result, the new speech recognition engines clearly outperformed the performance of the systems previously submitted by the DNNs DNN-HMM based system. This work presents three novel speech recognition architectures evaluated on the Spanish RTVE2020 dataset, employed as the main evaluation set in the Albayzin S2T Transcription Challenge 2020.  
mainly with the use of voice control applications integrated in machines, medical instruments or technical devices. These interests have triggered special challenges for the current ASR technology, mainly related to the need to optimise and reduce neural models in order to be integrated in devices with low computational power but without a noticeable loss of quality. With the aim of meeting the requirements of embedded systems, the most common optimisation techniques rely on architecture and format optimisation as well as quantisation [1]. In addition to the compression of neural models, another big topic in the ASR field today corresponds to the use of large pre-trained models generated with self-supervised learning methods and a big amount of unlabelled data. The main idea behind is that these models can provide a solution in those challenging scenarios in which the amount of labelled data are scarce in any language or domain. Self-supervised models like Wav2 vec2.0 [2] and HuBERT [3] learn first from a large scale unlabelled speech corpus in different languages. The knowledge acquired during this initial training is usually then transferred to downstream tasks by using the model as a feature extractor or by fine-tuning it with in-domain labelled data in the target language. In this paper, we explore and evaluate three novel ASR architectures on the Spanish RTVE2020 database [4]. The first system is based on the Kaldi toolkit [5], and it was constructed on the novel Multistream CNN architecture designed for robust acoustic modelling in speech recognition tasks. This architecture processes input speech with diverse temporal resolutions by having stream-specific dilation rates to convolutional neural networks (CNNs) across multiple streams. The second system presented in this paper is based on the NVIDIA's Quartznet E2E architecture [6], designed by the need to reduce the size and complexity of the ASR E2E models and to make them lighter, faster and more feasible to deploy on embedded hardware while maintaining the SoA-level accuracy. The Quartznet architecture is composed of multiple blocks with residual connections in between. Each block consists of one or more modules with 1D time-channel separable convolutional layers, batch normalisation, and ReLU layers. Finally, for the last architecture presented, we explored the application of the self-supervised E2E model Wav2 vec2.0 proposed by Facebook AI [2] as the pretrained model. This model was then fine-tuned with the same in-domain labelled data employed to train the first two ASR systems described above. The Wav2 vec2.0 model encodes speech audio via a multi-layer convolutional neural network and then it masks spans of the resulting latent speech representation, which are fed to a Transformer network to build contextualised representations. Throughout the paper, we compared the performance of these three novel architectures with the ones obtained by the primary system and the third contrastive system previously presented by the authors [7] to the Albayzin-RTVE 2020 Speech to Text Transcription Challenge.