LAMAD : A Linguistic Attentional Model for A rabic Text Diacritization (2021)

Abstract: In Arabic Language, diacritics are used to specify meanings as well as pronunciations. However, diacritics are often omitted from written texts, which increases the number of possible meanings and pronunciations. This leads to an ambiguous text and makes the computational process on undiacritized text more difficult. In this paper, we propose a Linguistic Attentional Model for Arabic text Diacritization (LAMAD). In LAMAD, a new linguistic feature representation is presented, which utilizes both word and character contextual features. Then, a linguistic attention mechanism is proposed to capture the important linguistic features. In addition, we explore the impact of the linguistic features extracted from the text on Arabic text diacritization (ATD) by introducing them to the linguistic attention mechanism. The extensive experimental results on three datasets with different sizes illustrate that LAMAD outperforms the existing state-of-the-art models.

## introduction arabic is one of the most widely spoken semitic languages, the official language for about 27 countries, and spoken by more than 400 million speakers around the world @xcite . diacritics are marks added above or below letters to give the word correct meaning and pronunciation @xcite . however, more than 97% of arabic texts (e.g., magazines, newspapers, books,etc ) are not diacritized @xcite increasing the text ambiguity which poses a challenge for diacritized texts based computational models @xcite . for example, translating the undiacritized arabic sentences using arabic machine translators face some difficulties. figure 1 shows two arabic sentences translation results without/with diacritization using google translate foot_0 . it is noticeable that the undiacritized sentences are wrongly translated. for example, the undiacritized sentence is translated into "according to ahmed, his money." while the correct meaning is "ahmed calculated his money." the main reason for the failure of machine translation is that there are too many possible meanings for an undiacritized word, and the exact meaning can be revealed by the word context and diacritization, which is difficult for machine translation to figure out even with the presence of some arabic contextual undiacritized words. on the other hand, the diacritized sentences are translated correctly due to the diacritics specify the word meanings. arabic diacritizers can help reveal this ambiguity and improve the performance of various diacritized-text based nlp applications: automatic speech recognition @xcite , arabic machine translation @xcite ), text to speech (zine and meziane, 2017), part-of-speech (pos) tagging (abuzeina and abdalbaset, 2019), and indexing diacritized text enable the search engine to exclude the unwanted matches. arabic diacritization problem has been addressed utilizing classical machine learning methods (e.g., hidden markov models, the dynamic programming method byte pair encoding (bpe), and support vector machine) @xcite or deep learning based approaches @xcite @xcite , which yield state-of-theart performance for arabic text diacritization using bidirectional lstm or convolutional neural networks. to improve the performance of atd, deep learning models are enhanced by arabic diacritization rules @xcite . recent diacritization models adapting convolutional or recurrent neural networks show an improvement of arabic text diacritization (atd) @xcite @xcite @xcite . however, most of those models are based on the character-level representation, which helps generalize the model but loses some useful linguistic features such as part of speech, word number, etc. to solve this problem, we propose a novel linguistic attentional model in which we introduce a linguistic feature representation at the character-level utilizing word and character linguistic features, and investigate their impact on atd. then, a linguistic attention mechanism for atd is proposed to capture the most crucial features which influence the word diacritization. our main contributions are summarized as the following: ## the proposed lamad figure 2 shows the architecture of the proposed linguistic attentional model. in our model, we introduce a new linguistic feature representation to extract the linguistic contextual features. then, the linguistic attention mechanism for atd is presented. the linguistic attention mechanism is adopted to distinguish the different importance of those features and capture the most crucial textual ones and have a decisive effect on diacritization, which is designed for the first time in arabic diacritization and proved its effectiveness. ## conclusion in this paper, we propose a linguistic attentional model to tackle the arabic diacritization problem. a new features representation method is presented, and the impact of morphological and syntactic information, extracted as features, is investigated. to evaluate the proposed system, three diacritized arabic corpora are used; two of them are small datasets and one large dataset with long sentences. the proposed lamad achieved the best results compared to the state-of-the-art models.