# A Light-Weight Autoregressive CNN-Based Frame Level Transducer Decoder for End-to-End ASR
## 4 Discussion  
A frame-level autoregressive decoder is proposed to reduce the decoding time for the inference operation of end-to-end ASRs while maintaining good WERs. This was achieved by adopting the autoregressive architecture and reducing the decoding steps and the number of trainable parameters of the decoder. To reduce the decoding steps, the decoder is operated at the frame level, that is, the decoder generates the probabilities of the token set once in each frame, at every 40 ms in this work. To reduce the number of trainable parameters, only a CNN block of 177 k parameters was employed in the proposed decoder of this work, while the transformer decoder has a 6-layer transformer with 9.6 M trainable parameters and the RNN-T decoder has an LSTM block (910 k parameters) and a linear block (90 k parameters). Two special tokens (space, re-appearance) are used along with 300-word piece tokens to represent the token string. The proposed frame-level autoregressive decoder was trained along with an encoder with 18 conformers (30 M parameters), by using the 100 h Librispeech dataset (train-clean). A comparison of the proposed decoder with the state-of-the-art autoregressive decoders combined with the same encoder used in this work demonstrates that the proposed decoder provides comparable WERs to other decoders with much less decoding time (1.7 and 30.9 times less than the RNN-T and the transformer decoders, respectively). A comparison with non-autoregressive decoders presents that the proposed decoder gives better WERs than other decoders with a longer decoding time.  
Conceptualization, H.-K.N. and H.-J.P.; software, H.-K.N.; project administration, H.-K.N. and H.-J.P. worked together during the whole editorial process of the manuscript. All authors were involved in the preparation of this manuscript. All authors have read and agreed to the published version of the manuscript.  
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2022 RI2C2003451).  
Not applicable.Not applicable.Not applicable.Data Availability Statement:The raw data supporting the conclusions of this article will be made available by the authors on request.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Relative Decoding Time**} & \multirow{2}{*}{**Params (M)**} & \multicolumn{2}{c}{**Word Error Rate (\%)**} \\ \cline{3-5}  & & & **Test-Clean** & **Test-Other** \\ \hline Original RNN-T (LSTM) & 1.65 & 1.0 & 6.6 & 18.3 \\ RNN-T (LSTM \(\rightarrow\) CNN) & 1.83 & 0.27 & 6.8 & 18.3 \\ This work (CNN) & 1.0 & 0.18 & 6.8 & 18.4 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Comparison of this work with RNN-T decoder.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**Word Error Rate (\%)**} \\ \cline{2-7}  & \multicolumn{3}{c}{**Beam Size = 1**} & \multicolumn{3}{c}{**Beam Size = 10**} \\ \cline{2-7}  & **Relative Time** & **Test-Clean** & **Test-Other** & **Relative Time** & **Test-Clean** & **Test-Other** \\ \hline
**This work** & **1.0** & **6.8** & **18.4** & **24.14** & **6.8** & **18.3** \\ RNN-T & 1.65 & 6.6 & 18.3 & 30.36 & 6.3 & 17.9 \\ Transformer & 30.9 & 7.2 & 18.0 & 347.77 & 6.7 & 17.8 \\ \hline \hline \end{tabular}
\end{table}
Table 9: WERs and decoding times with and without beam search, beam size = 10.