# Variable Scale Pruning for Transformer Model Compression in End-to-End Speech Recognition
## Appendix A Adaptive Pruning Settings  
In this appendix, we present the results obtained through adaptive pruning with the development data from the Libri-trans and VoxforgeIT databases. The approach involves fixing the pruning rate for the attention layers while systematically varying the pruning rate for the feedforward layers. This variable rate is determined by the parameters \(U_{0}\) and \(V_{0}\), which subsequently allow us to calculate the model's overall pruning rate. The error rate is duly recorded. As highlighted earlier in Section 5.2, among these findings, the most advantageous tradeoffs between WER and Sparsity are identified and underscored through bold formatting. To determine the most optimal pairs (WER, Sparsity), it is essential to select a specific WER value from the table and then choose the associated minimum sparsity level. \begin{table}
\begin{tabular}{l l c c c c c c c c c} \hline \hline  & & & & & & \multicolumn{3}{c}{\(\mathbf{U_{0}\) (\%)}} & & & \\ \cline{3-10} \(V_{0}\) (\%) & & **30** & **35** & **40** & **45** & **50** & **55** & **60** & **65** \\ \hline
30 & WER & 6.5 & 6.6 & 6.6 & 6.7 & 6.8 & 6.9 & 7.1 & **7.1** \\ Sparsity & 26.62 & 29.12 & 31.62 & 34.12 & 36.62 & 39.12 & 41.62 & **44.12** \\ \hline
35 & WER & 6.5 & 6.5 & 6.6 & 6.7 & 7.0 & 7.2 & 7.7 & 8.4 \\ Sparsity & 27.87 & 30.37 & 32.87 & 35.37 & 37.87 & 40.37 & 42.87 & 45.37 \\ \hline
40 & WER & 6.6 & **6.5** & **6.6** & **6.7** & 7.0 & 7.3 & 7.8 & 8.5 \\ Sparsity & 29.12 & **31.62** & **34.12** & **36.62** & 39.12 & 41.62 & 44.12 & 46.62 \\ \hline
45 & WER & 6.6 & 6.6 & 6.7 & **6.8** & 7.1 & 7.3 & 7.8 & 8.6 \\ Sparsity & 30.37 & 32.87 & 35.37 & **37.87** & 40.37 & 42.87 & 45.97 & 47.87 \\ \hline
50 & WER & 6.7 & 6.6 & 6.8 & 6.7 & 7.2 & 7.4 & 7.9 & 8.6 \\ Sparsity & 31.62 & 34.12 & 36.62 & 39.12 & 41.62 & 44.12 & 46.62 & 49.12 & 49.12 \\ \hline
55 & WER & 6.8 & 6.7 & 6.9 & **6.9** & 7.2 & 7.5 & 8.2 & 8.7 \\ Sparsity & 32.87 & 35.37 & 37.87 & **40.37** & 42.87 & 45.37 & 47.87 & 50.37 \\ \hline
60 & WER & 6.9 & 6.9 & 7.1 & 7.1 & 7.4 & **7.7** & **8.2** & 8.9 \\ Sparsity & 34.12 & 36.62 & 39.12 & 41.62 & 44.12 & **46.62** & 49.12 & **51.62** \\ \hline
65 & WER & 7.1 & 7.1 & 7.1 & 7.4 & 7.6 & 7.9 & 8.6 & **9.2** \\ Sparsity & 35.37 & 37.87 & 40.37 & 42.87 & 45.37 & 47.87 & 50.37 & **52.87** \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Adaptive pruning of the Libri-trans model: WER and Sparsity. The overall pruning rate (i.e., Sparsity) of the model is derived from the values of the pairs (\(U_{0}\), \(V_{0}\)). The fixed parameters are: attention layer pruning rate = 30% and \(\alpha=\beta=0.01\). \begin{table}
\begin{tabular}{l l c c c c c c c c} \hline \hline  & & \multicolumn{8}{c}{\(U_{0}\) (\%)} \\ \cline{3-10} \(V_{0}\) (\%) & & **45** & **50** & **55** & **60** & **65** & **70** & **75** \\ \hline
60 & CER & 10.5 & 10.5 & 10.6 & 10.9 & 11.4 & 11.4 & 11.5 \\  & Sparsity & 38.52 & 41.69 & 44.87 & 48.04 & 51.22 & 52.11 & 53.22 \\ \hline
65 & CER & 10.5 & 10.5 & 10.6 & 10.9 & 11.5 & 11.5 & 11.8 \\  & Sparsity & 39.15 & 42.30 & 45.50 & 48.68 & 51.85 & 55.03 & 57.12 \\ \hline
70 & CER & 10.5 & 10.5 & 10.6 & 10.9 & 11.5 & 11.5 & 12.0 \\  & Sparsity & 39.80 & 43.01 & 46.14 & 49.31 & 52.50 & 55.66 & 59.01 \\ \hline
75 & CER & 10.5 & 10.5 & 10.6 & 10.9 & 11.5 & 11.5 & 12.2 \\  & Sparsity & 40.62 & 43.60 & 46.77 & 49.95 & 53.12 & 56.30 & 59.2 \\ \hline
80 & CER & 10.5 & **10.5** & 10.6 & 10.9 & 11.5 & 11.5 & 12.2 \\  & Sparsity & 41.06 & **44.23** & 47.41 & 50.58 & 53.76 & 57.01 & 60.11 \\ \hline
85 & CER & 10.5 & 10.5 & **10.6** & **10.9** & 11.2 & **11.5** & 12.5 \\  & Sparsity & 41.70 & 44.87 & **48.04** & **51.22** & 54.40 & **57.57** & 60.74 \\ \hline
90 & CER & 10.6 & 10.6 & **10.7** & 11.0 & 11.2 & 11.6 & **12.3** \\  & Sparsity & 42.33 & 45.50 & **49.68** & 51.85 & 55.03 & 58.20 & **61.38** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Adaptive pruning of the VoxforgeIT model: CER and Sparsity. The overall pruning rate (i.e., Sparsity) of the model is derived from the values of the pairs (\(U_{0}\), \(V_{0}\)).