# A Light-Weight Autoregressive CNN-Based Frame Level Transducer Decoder for End-to-End ASR
## 3 Experiments and Results
Also, In Table 6 the decoding times are compared along with the encoding times during the inference operation for the Librispeech test-other dataset (2939 sentences, around 5.1 h long); the AMD EPYC 7402P 24-Core Processor CPU was used for this inference operation. The proposed decoder works 30.9 and 1.65 times faster than the transformer and the RNN-T decoders, respectively, while the CTC decoder is 13.5 times faster than this work. In this work, the encoder spends 7.6 times longer time than the decoder. However, this work takes longer training time than other decoders due to the less optimized gradient computation step. The gradient computation equations (Equations (7) and (8)) of this work have the computational complexity of \(O(T_{enc}*U)\) while those of CTC, RNN-T and transformer decoders have \(O(T_{enc})\), \(O(T_{enc}+U)\) and \(O(U)\), respectively. It is expected that this long training time may be reduced by sharing intermediate computation results in future works. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{4}{c}{**Inference Time (s)**} & \multirow{2}{*}{**Training Time**} \\  & **Encoding** & **Decoding** & **Relative Decoding** & **(h)** \\  & **Time** & **Time** & **Time** & **(h)** \\ \hline
**This work** & **368** & **48.6** & **1.0** & **75.9** \\ Transformer [9] & 350 & 1500 & 30.9 & 41.7 \\ RNN-T [18] & 365 & 80.1 & 1.65 & 38.1 \\ CTC [12] & 371 & 3.63 & 0.074 & 24.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of decoding times in seconds during inference using LibriSpeech test other data set and training times in hours using Librispeech train-clean-100. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Params (M)**} & \multicolumn{2}{c}{**Word Error Rate (\%)**} \\ \cline{3-5}  & & & **Test-Clean** & **Test-Other** \\ \hline \multirow{3}{*}{Auto-regressive} & **This work** & **0.18** & **6.8** & **18.4** \\  & Transformer [9] & 9.6 & 7.2 & 18.0 \\  & RNN-T [18] & 1.0 & 6.6 & 18.3 \\ \hline \multirow{3}{*}{Non-Autoregressive} & CTC [12] & 0 & 7.7 & 20.7 \\  & Self-conditioned CTC [17; 21] & 0 & 6.9 & 19.7 \\ \cline{1-1}  & Intermediate CTC [16; 21] & 0 & 7.1 & 20.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of WER (%) and number of trainable parameters. The kernel size of the CNN block of the decoder was set to 6 in this work because the best WER was observed empirically at the kernel size 6 in the range from 4 to 24 for four datasets (Librispeech dev-clean, dev-other, test-clean, test-other) as shown in Figure 7; the number of the layers and the channels of the CNN block was set to 1 and 128, respectively. The kernel size 6 corresponds to 2.73 words on average; this corresponds to the 3.73-gram language model, which is close to the known-to-be-optimum 4-gram language model [25]. The optimum number of the layers and the channels of the CNN block was found empirically to be 1 and 128, respectively, with the kernel size set to 6 (Table 7). With the increase in the number of trainable parameters of the CNN block, the WER degrades due to the overfitting of the potentially wrong previous token sequence with the beam search not used in this work. The RNN-T decoder provides the best WER among the decoders for the end-to-end ASRs without using the beam search. There are two differences between this work and the RNN-T decoder. One is the decoding step is [\(T_{enc}\)] in this work and [\(T_{enc}+U\)] in the RNN-T decoder. The other is that a CNN block with 177 k trainable parameters is used in this work, while an LSTM block with 910 k parameters and a linear block with 90 k parameters is used in the RNN-T decoder. To assess the contribution of the two differences to the decoding time, the number of trainable parameters, and WER, the LSTM block of the RNN-T decoder was replaced by the CNN block of this work.