Investigating the Impact of Various Partial Diacritization Schemes on A rabic- E nglish Statistical Machine Translation (2016)

Abstract: Most diacritics in Arabic represent short vowels. In Arabic orthography, such diacritics are considered optional. The absence of these diacritics naturally leads to significant word ambiguity to top the inherent ambiguity present in fully diacritized words. Word ambiguity is a significant impediment for machine translation. Despite the ambiguity presented by lack of diacritization, context helps ameliorate the situation. Identifying the appropriate amount of diacritic restoration to reduce word sense ambiguity in the context of machine translation is the object of this paper. Diacritic marks help reduce the number of possible lexical word choices assigned to a source word which leads to better quality translated sentences. We investigate a variety of (linguistically motivated) partial diacritization schemes that preserve some of the semantics that in essence complement the implicit contextual information present in the sentences. We also study the effect of training data size and report results on three standard test sets that represent a combination of different genres. The results show statistically significant improvements for some schemes compared to two baselines: text with no diacritics (the typical writing system adopted for Arabic) and text that is fully diacritized.

## introduction resolving natural language ambiguity is at the crux of the nlp enterprise. ambiguity refers to the problem of possibly having different interpretations for different segments (words, phrases, etc.) of a sentence. languages such as arabic, hebrew and persian are typically written in a manner that exacerbates this ambiguity problem and increases the homograph rate by underspecifying some of the letters such as short vowels and consonantal gemination, which in turn increases the effect of having multiple interpretations for the same word. this renders text even more ambiguous than typically expected. while context helps native speakers of the language resolve some of the ambiguity, context alone does not always produce adequate clarity for interpretation. the problem is further complicated in arabic by the fact that there are no native speakers of modern standard arabic (msa), which is the language used in education and formal settings. instead, speakers of arabic converse in various dialects of arabic which are at times starkly different from msa. one solution for this problem is diacritic restoration, or diacritization, which refers to ren-dering the underspecified diacritics explicit in the text. we investigate the problem of diacritization within the context of arabic-to-english statistical machine translation (smt) system. we address the problem in msa texts, the majority of which are underspecified for these diacritic marks. we focus here on the most prominent arabic diacritics which are short vowels @xcite , the syllable boundary marker, known as sukoon (o), indefiniteness marker, known as nunation (f, k, n), and the consonantal doubling marker (gemination) known as shadda (∼) 1 . in this study, we aim to investigate what is the appropriate level and type of diacritic restoration that would have the biggest impact on natural language understanding as tested and evaluated via machine translation. hence we experiment with various diacritization schemes based on lexical and/or syntactic information. this current work is a follow on to the pilot work presented in @xcite . however it is different in the following respects: 1-we explore automatically diacritized data; 2-we define more schemes that target both lexical and/or syntactic properties of the arabic language. 3-we test the robustness of our observations taking into consideration varying training size and cross genre evaluation. ## related work automatic arabic diacritization has been addressed thoroughly in @xcite @xcite @xcite . full diacritization indicates rendering the text with all the most prominent diacritics, namely (a, i, u, o, ∼). 2 initial efforts in automatic diacritization include rulebased approaches to add all diacritics in the texts @xcite ; however, it is expensive to maintain these rules to be generalized for unseen instances. most studies focused on full diacritic restoration. for automatic speech recognition (asr), @xcite perform full diacritization on msa speech transcripts for language modeling. they show that developing asr models on fully diacritized datasets improves performance significantly. supervised classifiers such as hidden markov model (hmm) and maximum entropy (maxent) have been employed for diacritization @xcite @xcite . in a study conducted by @xcite , the researchers use maxent trained on msa with lexical and n-gram features to improve asr. another study uses decision trees and stochastic language models to fully diacritize texts in order to render graphemes to synthesized speech @xcite . the buckwalter arabic morphological analysis (bama) @xcite system has been used along with a single tagger or a language model to select amongst the diacritized analyses in context to render text fully diacritized @xcite . in @xcite , the authors show that some inflectional and lexical related morphological features improve the performance of syntactic parsing in arabic. although @xcite have not used diacritics directly in their work, they use the same essential information that is used to diacritize arabic texts. @xcite not only investigate the impact of full diacritization on statistical machine translation (smt) but also introduce the notion of partial diacritization. they also show that several schemes have a small positive effect albeit not significant on smt performance over none and full diacritization despite the significant increase in the number of types. although the results in @xcite are not statistically significant, they provide directions of research that we can exploit to increase the performance of arabic related nlp applications. in a study conducted by alhanai and glass (2014), three partial diacritic schemes have been defined and compared to both fully and non-diacritized versions of the words. in their study, it is found that fully-diacritized text without gemination have statistically better performance than fully diacritized texts including gemination in asr application. our work follows the same general procedure as @xcite where we study the impact of some aspects of diacritization information in nlp applications, smt in particular. for arabic reading comprehension, @xcite studies the impact of partial diacritics in improving arabic speakers' reading comprehension. their study shows the effectiveness of having some level of diacritization between none and fully diacritized forms that help the readers disambiguate homographs that cannot be understood by the surrounding contexts. this shows the importance of having accurate automatic partial diacritization not only in improving different nlp applications but also to diacritize texts to help readers understand arabic texts better. having the goal of helping other researchers develop partial diacritization, @xcite has conducted a pilot study that minimally diacritize the dataset to reduce lexical ambiguity and help generate models to find an optimal level of diacritization in some nlp applications. although the result of this minimally-diacritized annotation has been highly affected by the annotators' subjectivity and background, it has shown some promising results for future studies. the idea of integrating word sense disambiguation (wsd) technologies into the smt framework has been studied previously, tackling different aspects of the phenomenon and showing statistically significant improvement integrating explicit wsd into the smt system @xcite @xcite . mainly, wsd integration improves the ability of the system to choose the target translation if it has been incorporated efficiently. @xcite show an improvement in chinese-to-english smt system in eight different automatic evaluation metrics when they integrate wsd in their translation system at decode time. they use the same parallel corpus used for training and the phrase translation table generated by the smt tool to disambiguate senses of the words by using the aligned phrases in the target language. all of the previous work incorporates features that help disambiguate senses in a supervised or unsupervised manner to generate better quality translation. some of these studies change the smt pipeline to integrate wsd but others implement it as a pre-processing step at decode time. in this study, we have the same goal as theirs which is to appropriately select the correct sense of a target word at decode time. we implement this by adding a certain amount of diacritics in arabic as preprocessing in the data preparation step. thus, the translation quality is not only enhanced by the appropriate choice of target word but also by the fact that the word alignment procedure is improved. ## scheme extraction we investigate the impact of various partial diacritization schemes on smt application. we compare their performance against two baselines, specifically full diacritization where all the diacritics are present and none where no diacritics are present. similar to the extraction strategy of @xcite , each of these schemes is identified from fully diacritized arabic datasets. additionally, the extraction process of some schemes involves the full morphological analysis of the words' part of speech and their lemmas. to identify these morphological features, we use madamira, a morphological analyzer and disambiguator for the arabic language @xcite . the quality of diacritization schemes rely on the performance of the automatic diacritization to predict diacritics. it is important to note that we rely on the underlying diacritized lemma form for ensuring extraction accuracy. @xcite define six different diacritization schemes based on their usage prominence in the arabic treebank (atb) @xcite . namely, they are fully diacritized (full), passive voice diacritic marks (pass), consonant doubling or gemination (gem), pres-ence of the syllable boundary marker sukoon (suk), syntactic case and mood diacritics (cm), and the case of no diacritization (none). in this study, we adopt the same previously mentioned schemes in addition to introducing several new ones: full-cm, pass+cm, pass+gem, suk+gem, pass+suk, pass+suk+gem, full-cm-pass, tanween. 3 the following is a detailed explanation of these diacritic schemes. the schemes are linguistically-motivated reflecting lexical, syntactic, or both types of information. the arabic sentences are written in buckwalter transliteration foot_3 and are tokenized according to the atb style (arabic treebank tokenization). it is crucial to note that if the word is not affected by the defined diacritic pattern, we remove all of its diacritics (i.e. none scheme). baselines: none: indicates that no diacritics are kept at all in the sentence, including the removal of the naturally occurring diacritics.