# Cascade Speech Translation for the Kazakh Language
## 4 Materials and Methods
### Speech Recognition
#### 4.1.2 Methods  
DNN-HMM. The chain models can be considered as a separate design point in the space of acoustic models because they are a sort of DNN-HMM model that is implemented using nnet3 in Kaldi [34]. The neural network's output uses a frame rate that is three times lower, which dramatically reduces the amount of processing necessary during testing and makes real-time decoding considerably simpler. The log probability of the proper sequence is the sequence-level objective function with which the models are initially trained. By doing a full forward-backward on a decoding graph produced from a phone n-gram language model, it is essentially MMI implemented without lattices on the GPU. E2E transformer. An encoder-decoder architecture based on the Transformer was trained using the ESPnet framework [33], jointly trained with the connectionist temporal classification (CTC) objective function. The input speech was represented using 80-dimensional filter bank features with pitch computed every 10 ms over a 25 ms window. To process the acoustic features for the E2E architecture, a few initial blocks of VGG network were used. The E2E transformer ASR system was comprised of 12 encoder and 6 decoder blocks, with 4 heads in the self-attention layer, hidden states of 256-dimension and feed-forward network dimensions of 2048. The model was trained for 100 epochs using the Adam optimizer with an initial learning rate of 10 and warm-up steps of 30,000, with a dropout rate and label smoothing of 0.1 set. To assist decoding, a two-layer RNN with 650 long short-term memory (LSTM) units each was utilized to construct a character-level \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Structure**} & **Name of the Corpus/** & **Amount of** & **Overall Duration of** \\  & **Sets** & **Wav-Files** & **Wav-Files in Hour** \\ \hline
1 & ISSAI KSC & 153,853 & 332.6 \\ \hline
1.2 & train & 147,236 & 318.4 \\ \hline
1.3 & dev & 3283 & 7.1 \\ \hline
1.4 & test & 3334 & 7.1 \\ \hline
2 & M2 ASRKazakh-78 & 37,892 & 86 \\ \hline
2.1 & train & 34,392 & 78 \\ \hline
2.2 & test & 3500 & 8 \\ \hline
3 & Kazcorpus & 13,425 & 44.16 \\ \hline
3.1 & train & 11,741 & 37.96 \\ \hline
3.2 & dev & 799 & 2.36 \\ \hline
3.3 & test & 880 & 3.85 \\ \hline
4 & KazLibriSpeech & 575,243 & 992 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The structure of the additional speech recognition corpora for the Kazakh language. LM, based on the transcripts of the training set. Additionally, the LSTM LM was used during the decoding stage. Wav2 Vec 2.0. and XLSR-53. The experiments were conducted using the Fairseq platform [35]. The Wav2 Vec 2.0 base model [36] was pre-trained with unlabeled speech data using various configurations, such as encoder layerdrop set to 0.05, dropout_input, dropout_features and feature_grad_mult set to 0.1 and encoder_embed_dim set to 768. The training hyperparameters included a learning rate of 5 \(\times\) 10\({}^{-4}\) and warmed up in the first 10% of the training time. The number of updates was set to 800,000, and the maximum quantity of tokens was set to 1,200,000. Additionally, the Adam optimizer was used, as in the original work. Standard fine-tuning procedures were used with fine-tuning parameters defined using the following configurations: the number of updates was set to 160,000, and the maximum quantity of tokens was set to 2,800,000. The Adam optimizer was used, and other parameters included a learning rate of 3 \(\times\) 10\({}^{-5}\) and a gradient accumulation of 12 steps. The batch size during training was defined automatically by the framework, depending on the maximum quantity of predefined tokens. During training, the best model was selected based on the lowest WER obtained on the validation set. The XLSR model was pre-trained using the same configurations as the Wav2 Vec large model. The encoder block consisted of 24 layers with a dimension of 1024, and 16 attention blocks were used with no dropout.