# Evaluating Novel Speech Transcription Architectures on the Spanish RTVE2020 Database +
## 4 Systems Description and Configuration
### Novel ASR Architectures
#### 4.1.2 Quartznet Q15\(\times\)5 Based System  
The Quartznet family of models are E2E ASR architectures completely based on 1D Time-Channel Separable Convolutional layers with residual connections, as it is illustrated in Figure 2. This design is based on the Jasper architecture [20] but with many modifications focused on considerably reducing the number of parameters and, therefore, the computing resources needed. As the most novel architecture, we trained and evaluated the Q15\(\times\)5 architecture, which is described as follows.  
The model initially integrates a 1D Convolutional layer (kernel(\(k\)) = 33, output channels (\(c\)) = 256) processing the spectrogram input from each speech frame. This layer is then followed by five groups of blocks. On the Q15\(\times\)5 architecture, each group of block is composed of a block \(B_{i}\) repeated three times with residual connections in between. Each \(B_{i}\) is composed of a module repeated five times and composed of (i) a \(k\)-sized depth-wise convolutional layer, (ii) a point-wise convolution, (iii) a batch normalisation layer, and (iv) a ReLU. The configuration of the CNN for each \(B_{i}\) was: B1 (\(k\) = 33, \(c\) = 256), B2 (\(k\) = 39, \(c\) = 256), B3 (\(k\) = 51, \(c\) = 512), B4 (\(k\) = 63, \(c\) = 512) and B5 (\(k\) = 75, \(c\) = 512). Finally, there are three additional convolutional layers C1 (\(k\) = 87, \(c\) = 512), C2 (\(k\) = 1, \(c\) = 1025) and  
Figure 2: Architecture of the Quartznet ASR model of NVIDIA. The input spectrogram of the audio signal is first processed by a 1D CNN layer and then fed into a series of five groups of blocks. Each group is composed of three blocks with residual connection between them in the Q15\(\times\)5 architecture and one block for the Q5\(\times\)5 architecture. Blocks are formed by a module composed of a Time-Channel Separable CNN repeated five times. Finally, a sequence of 3 CNN feed the output to a trainable CTC layer. This figure is based on the one presented in [6].  
a point-wise convolutional layer (\(k=1\), \(c=\textit{labels}\)), followed by a Connectionist Temporal Classification (CTC) layer.  
During training, a CTC loss function was employed to measure the prediction errors, in addition to the Novograd optimiser with beta values of 0.8 and 0.5 and a triangular cyclical learning rate policy during five cycles of 60 epochs for a total of 300 epochs, as it was described in [32]. The initial and minimum learning rates were set to 0.015 and \(10^{-5}\), respectively, whilst the weight decay was set to \(10^{-3}\). The training process was performed on four GPU cards, applying a batch size of 20 each and mixed precision training. Our resulting Q15\(\times\)5 network configuration contained 18.9 million parameters.  
Additionally, a 5-gram language model was trained with the _Transcriptions_, _RTVE2018_ and _A la Carta_ corpora. This language model was employed during decoding with a CTC beam-search decoder based on the pyctdecode library [33]. The decoding was performed with a _beam-width_ of 256, a \(\alpha=0.5\) and \(\beta=1\).