A rabic Dialect Identification with a Few Labeled Examples Using Generative Adversarial Networks (2022)

Abstract: Given the challenges and complexities introduced while dealing with Dialect Arabic (DA) variations, Transformer based models, e.g., BERT, outperformed other models in dealing with the DA identification task. However, to fine-tune these models, a large corpus is required. Getting a large number high quality labeled examples for some Dialect Arabic classes is challenging and time-consuming. In this paper, we address the Dialect Arabic Identification task. We extend the transformer-based models, ARBERT and MARBERT, with unlabeled data in a generative adversarial setting using Semi-Supervised Generative Adversarial Networks (SS-GAN). Our model enabled producing high-quality embeddings for the Dialect Arabic examples and aided the model to better generalize for the downstream classification task given few labeled examples. Experimental results showed that our model reached better performance and faster convergence when only a few labeled examples are available.

## introduction while arabic is the first language of most of the middle east and north africa (mena) region, different countries have different dialects of arabic. these dialect arabic (da) forms are all different from the modern standard arabic (msa). msa is used in formal writing and speaking situations, like academia and media. in contrast, da is the language of the street. da is spoken by people informally in their daily conversations and on social media platforms. the task of automatically identifying the dialect of arabic is beneficial since it contributes to many downstream tasks and applications, such as speech recognition and machine translation. some arabic dialects are very close to each other (e.g. levantine region dialects such as lebanese and syrian). and moroccan dialects) like in some datasets are imbalanced with few classes dominating the whole dataset. figure 1 illustrates the classes distribution in the nadi @xcite 2021 dialect dataset. some other datasets suffer from a limited number of dialects. another problem is mislabeled da examples due to noise in the labeling procedure, e.g., depending only on the geographic location. given these challenges, getting a large corpus of labeled da examples for all arab countries is challenging and time-consuming. these complexities represent a major challenge in the arabic dialect identification task. we aim to improve the transformer-based models, i.e., bert @xcite , that handle the task given the lack of large enough datasets. in this paper, we extend bert-based models, arbert and marbert @xcite , with a generative adversarial setting using semi-supervised generative adversarial networks (ss-gan) @xcite . this setting makes use of a set of unlabeled data, which can easily be obtained, to better generalize for the arabic dialect identification task given a few labeled examples. semi-supervised learning with adversarial nets was previously used for some tasks and languages, but to the best of our knowledge, it has not been used for arabic dialect identification before. the contributions of this work are: the rest of the paper is organized as follows: in section 2, we discuss the related work in the dialect arabic identification task and variations of bertbased models. in section 3, we illustrate the system components and model architectures. we show the conducted experiments and their results, in section 4. finally, we give a brief conclusion based on our work and the obtained results.