## 4 Experimental Setup  
### Datasets  
Several corpora and languages are considered to evaluate the validity of the proposed approach.  
CommonVoice [28] is a multilingual reading speech corpus that currently has 17,127 h of speech data in 104 languages, including English, French, German, Dutch, and Chinese, and allows anyone to contribute their voice to it. The amount of data for each language ranged from 2 h for Kazakh (\"low resource\") to 3209 h for English (\"high resource\"). For Kazakh, the Kazakh data set in CommonVoice contains 2 h of 1400 pieces of speech data and their labels. Additionally, using the KSC dataset [29], which contains more than 330 h of Kazakh speech sounds and tags crowdsourced over the Internet, volunteers from different regions and age groups were asked to read sentences presented through a web browser. In total, there are 153,000 recordings from more than 1600 different devices. The recordings were collected by people from different regions and age groups who completed the recordings by reading sentences presented in web browsers, and all accepted recordings were manually checked by native Kazakh speakers. Additionally, all annotations are in the Cyrillic alphabet, and the voice is stored in WAV format. In this paper, 20 h speech data randomly selected from the training set is used as the voice data to generate adversarial training, and 10,000 unmatched texts from the training set are used as the unlabeled text data. An hour's worth of randomly selected data from KSC's test set was used for the evaluation.  
For unsupervised pretraining of encoders, consider 960 h of untagged data using Librispeech [30] and 792 h of untagged data from CommonVoice in ten languages including Kazakh Kazakh Kazakh (kk), Turkish (tr), Kyrgyz (ky), Tatar (tt), Dutch (du), Chinese (zh), Russian (ru), Italian (it), Spanish (es), and French (fr). In keeping with the Kazakh language, The supervised data duration for each language trained by Decoder-1 is set to 2 h, and the audio and transcription of the same data are used as unlabeled data to train Decoder-2. The assessment duration is set according to the partitioning method proposed by [PERSON] et al. [31]. The data duration for each language is shown in Table 1.  
For Decoder-1 and Decoder-2, TIMIT is first used to explore the most appropriate \(\alpha\) setting for calculating joint loss \(\mathcal{L}_{CTC+Att}\). Next, in order to explore the effectiveness of the JSUM in the use of supervised and unsupervised learning under low-resource conditions, experiments were conducted on several other corpora. LibriSpeech, KSC, TIMIT, and THUYG-20 data set specifications are shown in Table 2.  
The TIMIT dataset contains about five hours of recordings and time-aligned phoneme labels, a total of 6300 sounds and corresponding labels, including SX (sentences with tight phonemes), SI (sentences with divergent phonemes), and SA (dialect sentences), with both \"match\" and \"do not match\" settings. The training set, development set, and test set in the \"match\" setting contain 3696, 400, and 192 voices and their corresponding labels, respectively. Only SX and SI are included in this setting. For the \"mismatch\" setting, there are 3000 speeches and 1000 labels separated from the training portion of the complete data set, but these unlabeled text labels do not contain transcripts of audio data. In this paper, the standard \"matching\" setting is used to explore different alphas to calculate the joint loss \(\mathcal{L}_{CTC+Att}\).  
For the Uygur language, use THUYG-20 [32], an open Uygur speech database jointly created by Tsinghua University and Xinjiang University. The database includes about 20 h of training data and 1 h of test data. The full audio training data is used as unsupervised data to generate adversarial learning to explore the effectiveness of the proposed multi-tasking learning model in a cross-language setting.