Aswat: A rabic Audio Dataset for Automatic Speech Recognition Using Speech-Representation Learning (2023)

Abstract: Recent advancements in self-supervised speech-representation learning for automatic speech recognition (ASR) approaches have significantly improved the results on many benchmarks with low-cost data labeling. In this paper, we train two self-supervised frameworks for ASR, namely wav2vec, and data2vec, in which we conduct multiple experiments and analyze their results. Furthermore, we introduce Aswat dataset, which covers multiple genres and features speakers with vocal variety. Aswat contains 732 hours of clean Arabic speech that can be used in the pretraining task for learning latent speech representations, which results in achieving a lower word error rate (WER) in Arabic ASR. We report the baseline results and achieve state-of-the-art WERs of 11.7% and 10.3% on Common Voice (CV) and the second round of Multi-Genre Broadcast (MGB-2) respectively, as a result of including our dataset Aswat.

## introduction automatic speech recognition (asr) is the task of transcribing speech audio into text. supervised deep learning has shown a notable improvement in speech recognition, providing significant gains in tasks rich in labeled data. unfortunately, this reliance on labeled data limits the extent to which deep learning can advance, primarily because of the scarcity of labeled data in some tasks. recently, self-supervised approaches have overcome this problem and made it possible to reach outstanding results with a limited labeled dataset @xcite @xcite . self-supervised learning leverages raw waveforms to learn representation that captures low level features and underlying structure of the data. the learned representations in the pretraining phase are used in downstream tasks in a supervised phase with a minimal amount of labeled data. arabic is one of the most spoken languages worldwide, with over 400 million speakers @xcite . it is considered challenging to process automatically due to various internal factors, including multiple dialects, ambiguous syntax, syntactical flexibility, and diacritics @xcite . however, modern standard arabic (msa) is one formal dialect that is understood by the majority of arabic speakers. it is the formal spoken and written dialect that is often used in formal speech, news broadcasts, radio, and newspapers. it is also taught in schools and universities @xcite . in our work, we utilized the self-supervised frameworks wav2vec @xcite and data2vec @xcite , and released dataset aswat (voices), on which we trained the asr systems. aswat is a well-organized, unannotated dataset of arabic speech, of which 66% is in msa (modern standard arabic). we carefully curated and manually cleaned it, and it includes speakers from various demographic backgrounds. it has 732 hours of speech constructed from audio files on the internet; thus, it covers a variety of audio files recorded from different speakers and under various recording setup environments. aswat leads to the learning of useful latent speech representations during the pretraining task in wav2vec @xcite and data2vec @xcite . this results in state-of-the-art performance in arabic with a word error rate (wer) of 11.7% on common voice (cv) and 10.3% on mgb-2, achieved with fewer training instances compared to the second round of multi genre broadcast (mgb-2). the original audio files are crawled *equal contribution ⸭ work done in tahakom from youtube and soundcloud; therefore, they are subject to copyright. we made the dataset publicly available foot_0 for non-commercial purposes. this paper's contributions can be summarized as follows: ## related work in @xcite , the first transformerbased architecture was introduced to better parallelize self-attention mechanisms. furthermore, when applied to asr tasks, @xcite demonstrated that transformerbased models outperformed state-of-the-art recurrent neural networks (rnns). in the asr task, self-supervised approaches, such as [1, 3], have recently shown significant improvement. the main difference between them is that wav2vec learns discrete units of speech during pretraining through a quantitation process, and data2vec directly predicts contextualized latent representations without quantization. although the literature on e2e models trained on arabic speech is limited, researchers have done valuable work that is essential to the community. in @xcite , the authors used ctc and rnns, and the reported results were on the mgb-2 development set, without any further results on the test set. in @xcite , the authors analyzed the learned internal representations and compared phonemes and graphemes as well as various articulatory features using deepspeech2, an end-to-end asr model. in taha zouhair's work foot_2 , the author used wav2vec model on cv benchmark, achieving a wer of 24 %. @xcite utilized the transformer architecture with ctc and attention objectives resulting in a wer of 12.5 % in an msa task on mgb-2. more recently, chowdhury et al. (2021) proposed a multilingual strategy for dialectal code switching in arabic asr. using end-to-end transformer models reported in @xcite for arabic, they achieved state-of-the-art results with a wer of 12.1 % demonstrating the effectiveness of multilingual approaches. in our work, we constructed a high-quality dataset and reached state-of-the-art wers on two well-known benchmark datasets, by pretraining selfsupervised architectures, namely wav2vec2.0 @xcite and data2vec @xcite . ## conclusion in this work, we provide the community with 732 hours of a clean and organized arabic speech dataset. we report state-of-the-art results for asr with data2vec architecture, and by combining aswat with mgb-2 in the pretraining stage, we achieved a wer of 11.7% on cv and 10.3% on mgb-2. in the future work, we plan to improve our methods by using automatic audio cleaning tools 9 and tool in @xcite to collect bigger data and include more dialects. in addition, we plan to use the large data2vec and adjust the hyper-parameters based on the training data to enhance the results.