ASR pipeline for low-resourced languages: A case study on Pomak (2023)

Abstract: Automatic Speech Recognition (ASR) models can aid field linguists by facilitating the creation of text corpora from oral material. Training ASR systems for low-resource languages can be a challenging task not only due to lack of resources but also due to the work required for the preparation of a training dataset. We present a pipeline for data processing and ASR model training for low-resourced languages, based on the language family. As a case study, we collected recordings of Pomak, an endangered South East Slavic language variety spoken in Greece. Using the proposed pipeline, we trained the first Pomak ASR model.

## introduction speech technologies have gained popularity in the past decade and several people use voice commands to communicate with their devices or to dictate messages. furthermore, such technologies can be of use in field and corpus linguistics. manually transcribing one minute of recorded speech takes on average 40 minutes; automatic speech recognition (asr) models can facilitate the transcription of spoken corpora by providing the first iteration of the transcription @xcite . if high-quality recordings are available, text-to-speech (tts) models can augment speech corpora by generating audio files from text. however, training robust models requires several hundred hours of recorded speech, while most languages do not have enough such resources. therefore, in low-resource settings, one typically bootstraps the process using a model that has been pre-trained in a related language with sufficient resources (e.g., wav2vec2 @xcite , xls-r @xcite , and whisper @xcite ). the pre-trained model is then fine-tuned on the target language data to obtain the final model (e.g., @xcite @xcite ). to aid linguists, @xcite proposed a pipeline ("elpis") to help train a kaldi-based @xcite asr model with minimal scripting. the pipeline assumes that the transcription has been done via elan foot_0 which includes timestamps. however, in case the available transcriptions lack time annotations, creating a dataset for a lowresource language can be a demanding task; one of the reasons is that, typically, asr systems require short audio segments for the training process. therefore, to create a dataset, any available recordings must be segmented into smaller parts while retaining the corresponding transcription. splitting an audio file on its own is a relatively straightforward task in specific conditions. one can use, for instance, a voice activity detection (vad) algorithm (e.g., using pyannote @xcite or praat foot_1 @xcite )) that segments based on whether speech is present in the signal. however, in the case of missing audiotranscription time alignments, vad alone cannot split the transcription. we propose a pipeline (section 2) for lowresourced languages that i) normalizes the available audio and transcription files, ii) extracts speechtext word-level alignments, iii) segments the audio files into smaller parts to create a dataset, and iv) fine-tunes an asr model based on the language family. as a use case, we have focused on pomak, an endangered south east slavic language variety spoken in greece @xcite . specifically, we have recorded over 14 hours of pomak read speech (section 3.1) and used the proposed pipeline to train the first pomak asr model (section 3.2). even though 14 hours of speech is considered a low-resourced setting in the field of automatic speech recognition, for many endangered languages the available recordings are even fewer. for this reason, we further trained an asr model using only 1 hour of speech to show the applicability of the proposed approach even in the case of a simulated endangered language scenario. ## asr pipeline typically, popular asr models use sample rates of 8khz (8000 samples/sec) or 16khz. the first step of the pipeline (figure 1 ) is to convert all recordings to the latter sample rate (16khz 16-bit mono channel wav files) because it provides more accurate high-frequency information and it matches the sample rate of the pre-trained models we use (see section 2.3). additionally, we normalize the text and convert dates and numbers to their literal equivalent. to be able to verbalize pomak dates and numbers, we have extended the num2words package foot_2 . this part is language-specific and will need to be customized for a new under-resourced language. if that is not possible, the conversion step needs to be done manually and the user will need to convert the numbers into their lexical equivalents. the most challenging part of the data preprocessing task is the segmentation of audio files while retaining the correct transcription of words. to do so, we first need to obtain speech-text alignments in order to get the exact onset and offset times of each word. ## conclusion we presented a pipeline that facilitates data processing and enables asr model training for lowresourced languages. using this pipeline, we created the first transcription model in pomak. we used the same dataset to train a tts model, which we plan on using to augment the pomak corpus.