## 2 Related Work  
End-to-end automatic speech recognition models are increasingly being investigated by researchers due to their simplified training process and time, and their ability to effectively improve the recognition accuracy of models. Currently, the mainstream end-to-end speech recognition models mainly include connectionist temporal classification (CTC) [16; 17], the recurrent neural network transducer (RNN-T) model [18; 19], and the attention mechanism-based encoder and decoder model [20; 21]. Among the attention-based mechanism models, the most representative one is Transformer [1]. Transformer uses a multi-head attention mechanism, increases parallel computation, and achieves good results in sequence modeling. Transformer, although more effective in extracting long sequence dependencies, performs poorly in extracting local features. For this reason, [PERSON] et al. proposed a new model, Conformer, which is a convolutional module to enhance Transformer's learning in local features [2]. The proposed end-to-end speech recognition framework based on Conformer has further improved the recognition accuracy of end-to-end speech recognition models and has also been widely used in the field of speech recognition [22; 23]. The end-to-end model based on the attention mechanism is difficult to train due to its overly flexible alignment. Therefore, adding CTC to the decoder layer of the attention-based mechanism to assist in training not only helps the model converge quickly, but also improves the recognition rate of the model [24; 25]. With the continuous development of deep learning techniques, end-to-end speech recognition is becoming more and more popular among researchers, but it also increases the number of parameters and storage space for models, which leads to higher resources and a longer training time for end-to-end speech recognition models. There are many research works investigating the reduction in the number of redundant parameters in end-to-end speech recognition models. [PERSON] et al. proposed a method to compress large pre-trained models using non-negative matrix decomposition LSTM post-training compression [26]. [PERSON] et al. proposed a QuartzNet model with one-dimensional temporal channel separable convolution to decompose the convolution [27]. [PERSON] et al. proposed to compress the end-to-end speech recognition model using a low-rank decomposition method, with no reduction in word error rate [28]. [PERSON] et al. [5] constructed a lightweight but effective end-to-end speech recognition model using low-rank decomposition in Speech-Transformer [29]. In addition, common model compression methods include knowledge distillation [30; 31] and pruning operations [32], but knowledge distillation and pruning mostly require retraining of the model, and the training process is more complicated. To the best of our knowledge, most studies have focused on the feed-forward and convolutional layers in the network, and few studies have been conducted on the multi-head attention module, especially the multi-head self-attention module of the currently more advanced Conformer end-to-end speech recognition model. Therefore, in order to reduce the number of parameters and storage space in the Conformer model, we use low-rank decomposition to compress the multi-head self-attention module of the Conformer model for the study. Speech recognition has been a relatively popular research object, but most research objects are mainly focused in resource-rich languages, such as Chinese datasets [33; 34], English datasets [35] and other datasets [36; 37], and very little work has been conducted to study speech recognition of languages in Central and West Asia. The Kazakh language is currently used in two regions: the first is Kazakhstan and Mongolia, in which the Cyrillic alphabet is used; the second is the Xinjiang Uyghur Autonomous Region of China, in which the alphabet of Arabic origin is used [13]. The Kazakh language in the Cyrillic alphabet is used as a modeling unit in this paper.