Yet Another Model for A rabic Dialect Identification (2023)

Abstract: In this paper, we describe a spoken Arabic dialect identification (ADI) model for Arabic that consistently outperforms previously published results on two benchmark datasets: ADI-5 and ADI-17. We explore two architectural variations: ResNet and ECAPA-TDNN, coupled with two types of acoustic features: MFCCs and features exratected from the pre-trained self-supervised model UniSpeech-SAT Large, as well as a fusion of all four variants. We find that individually, ECAPA-TDNN network outperforms ResNet, and models with UniSpeech-SAT features outperform models with MFCCs by a large margin. Furthermore, a fusion of all four variants consistently outperforms individual models. Our best models outperform previously reported results on both datasets, with accuracies of 84.7% and 96.9% on ADI-5 and ADI-17, respectively.

## introduction dialect identification can be viewed as a special case of language recognition @xcite . both tasks suffer from similar performance issues in the presence of background noise, channel mismatch, prosodic fluctuations, and so on. however, with closely related dialects having a small difference in both acoustic and linguistic feature space, dialect identification tasks are substantially more difficult in nature @xcite . the arabic language is spoken in various dialects across the arab world, in addition to modern standard arabic (msa) which is used in official and educational settings. speech recognition systems trained on msa data generally don't generalize well to dialectal arabic and specialized dialectal models may be needed for improving automatic speech recognition (asr) performance in systems developed for specific populations. dialect identification could facilitate the development of dialectal speech recognition systems in various ways, such as by identi-fying dialectal utterances in large multi-dialectal corpora, or online dialect identification for routing utterances to dialect-specific asr modules. to enable the development of spoken arabic dialect identification systems, two benchmark datasets have been developed: adi-5, which was deployed as part of the mgb-3 challenge @xcite and adi-17, deployed as part of the mgb-5 challenge @xcite . for both challenges, the top systems developed and submitted for the initial challenges remain the best performing systems reported in the research literature for these benchmarks. the adi-5 training set consists of 10 hours of dialectal speech from broadcast news, covering five dialects: egyptian (egy), levantine (lav), gulf (glf), north african (nor), and modern standard arabic (msa), in addition to two hours each for development and test sets. the adi-17 data set consists of 17 dialectal classes for a total of 3k hours extracted automatically from youtube. roughly 58 hours of data were manually verified for the development and test sets. in this paper, we describe spoken dialect identification models we developed and tested on these benchmarks, and we report results exceeding the best performing models submitted to both challenges. we experimented with the residual networks (resnet) @xcite and emphasized channel attention, propagation and aggregation (ecapa-tdnn) @xcite architectures. both architectures have been successfully employed for speaker verification tasks. in addition, resnet was used in the best performing dialect identification system in the mgb-5 challenge, and ecapa-tdnn has been recently explored for dialect classification, as in @xcite for irish dialects. in addition, we explored the use of acoustic features extracted from the unispeech-sat @xcite model, which have been shown to provide improvements in various tasks in the superb benchmark @xcite . we observe large improvements in accuracy by incorporating these features into our models. we also employ data augmentation via additive noise and speed perturbation, which generally help improve the generalization of speech classification models. our best model result is 84.7% accuracy in the adi-5 test set, compared to 75% previously reported as the best result in @xcite . in adi-17, our best model achieves 96.9% accuracy compared to 94.9% previously reported as the best model in @xcite . ## related work in this section, we describe the approaches proposed for adi tasks in mgb-3 and mgb-5 challenges, which are used as baseline systems in this work. we first describe the top two performing systems for the mgb-3 challenge (adi-5) @xcite , followed by the top two systems in the mgb-5 challenge (adi-17) @xcite . the mit-qcri adi system @xcite combines acoustic and linguistic features within a siamese neural network framework to reduce dimensionality based on ivectors. they used loss functions involving both euclidean and cosine distances and employed support vector machines as the backend classifier. in contrast, the university of texas at dallas (utd) submission @xcite to the mgb-3 challenge fused five systems, incorporating acoustic and lexical information through various techniques, including i-vectors, generative adversarial networks (gans), gaussian back-end (gb), and bnf i-vector features. the utd system obtained the second-best performance with an overall accuracy of 70.38% @xcite . duke kunshan university (dku) submitted four variants of resnets with different block sizes and datasets, which were fused to achieve the best performing system in the mgb-5 challenge @xcite . the dku system employed a resnet with global statistics pooling and a fully connected layer. they used the kaldi toolkit for data augmentation, including speed-perturbation and datasets such as musan and rir. the resnet system was trained using cross-entropy loss with a softmax layer, taking 64-dimensional mel-filterbank energy features as input. on the other hand, the university of kent (ukent) mgb-5 system @xcite used a neural network architecture combining convolutional neural networks (cnn) and long short-term memory (lstm) networks with time-scale modification (tsm). the ukent system reported an accuracy of 93.1% on the test set. while the best performing models reported in the original mgb-3 and mgb-5 challenges have not been outperformed in later publications (to the best of our knowledge), several other studies proposed model variants and analyzed the performance in various ways. regarding the use of pre-trained selfsupervised acoustic models, @xcite recently utilized the xls-r model @xcite , which is a multi-lingual pre-trained acoustic model that includes arabic as one of the languages used in pre-training, and hubert @xcite , which was pre-trained solely in english. they fine-tuned dialect classification models on the adi-17 dataset, and interestingly, the model based on hubert outperformed the xls-r-based model, in spite of the multi-lingual pre-training of the latter. this indicates that the quality of the features extracted from pre-trained acoustic models may depend more on the self-supervised training details rather than linguistic coverage. a model outperforming hubert on several benchmark tasks is the unispeech-sat acoustic model @xcite , which includes additional objectives on top of the hubert model to facilitate speaker-aware representations, which also generally embody nonlinguistic characteristics of utterances, such as tone and emotion. ## proposed model as the space of possible architectural or feature variations increases with the increasing volume of developments in the ml field, exhaustively searching all possible architectures is unfeasible. therefore, we draw inspiration from the best performing models in related literature to reduce the search space and increase the likelihood of finding a best performing model. we selected two neural network architectures, resnet and ecapa-tdnn, for their potential in speech classification tasks. for feature extraction, we compare classical mfcc features with the pre-trained unispeech-sat large acoustic model @xcite that has been shown to provide consistent improvements in various speech classification benchmarks. finally, as best models in previous works typically include a form of ensemble, we experimented with fusing all model variants to further improve performance. we describe the details of these parts in this section. ## results tables 1 and 2 show the performance of our model variants in adi-5 and adi-17 test sets, respectively. fusion refers to an ensemble model where scores from all four variants are combined, each with an equal weight of 0.25. we also show the performance of the best performing models from the original challenges, which have not been previously outperformed to the best of our knowledge. we observe consistent results in both datasets: ecapa-tdnn network consistently outperforms resnet, and the models using unispeech-sat features consistently outperform those using mfcc features. incorporating these pre-trained features results in 4% to 5% absolute improvement in accuracy for both models. we observe additional gains of 0.8% to 2% improvement in absolute accuracy by fusing all four model/feature combinations. the highest performance gain is observed by using unispeech-sat features as input, which leads to outperforming all previous baselines. ## conclusions this paper described variations of model architectures, namely resnet and ecapa-tdnn, employing two acoustic features: classical mfccs and self-supervised unispeech-sat, leading to state-ofthe-art performance in two spoken arabic dialect identification benchmarks: adi-5, and adi-17. unispeech-sat features, which are extracted from a large pre-trained model optimized for acoustic and speaker variability, consistently demonstrated superior performance compared to mfcc features. despite being pre-trained solely in english speech, unispeech-sat illustrates transfer learning capa-bility by extracting suitable feature representations for this discriminative task in the arabic language. this may also indicate that non-linguistic acoustic variability (such as speaking tone, for example) could play a role in dialect identification. consistent with previous models from the mgb-3 and mgb-4 challenge, fusing multiple models results in consistent improvements of overall performance.