A Context-Contrastive Inference Approach To Partial Diacritization (2024)

Abstract: Diacritization plays a pivotal role for meaning disambiguation and improving readability in Arabic texts. Efforts have long focused on marking every eligible character (Full Diacritization). Overlooked in comparison, Partial Diacritzation (‘PD‘) is the selection of a subset of characters to be annotated to aid comprehension only where needed. Research has indicated that excessive diacritic marks can hinder skilled readers—reducing reading speed and accuracy. We conduct a behavioral experiment and show that partially marked text is often easier to read than fully marked text, and sometimes easier than plain text. In this light, we introduce Context-Contrastive Partial Diacritization (‘CCPD‘)—a novel approach to ‘PD‘ which integrates seamlessly with existing Arabic diacritization systems. ‘CCPD‘ processes each word twice, once with context and once without, and diacritizes only the characters with disparities between the two inferences. Further, we introduce novel indicators for measuring partial diacritization quality to help establish this as a machine learning task. Lastly, we introduce ‘TD2‘, a Transformer-variant of an established model which offers a markedly different performance profile on our proposed indicators compared to all other known systems.

## introduction the arabic language is central to the linguistic landscape of over 422 million speakers. it plays a pivotal role in the religious life of over a billion muslims @xcite . as in other impure abjad writing systems, the arabic script omits from writing some phonological features, like short vowels and consonant lengthening. this can affect reading efficiency and comprehension. readers use context from neighbouring words, the domain topic, and experience with the language structure to guess the correct pronunciation and disambiguate the meaning of the text. the arabic nlp community has noticeably focused on the task of full diacritization (fd)-the modeling of diacritic marks on every eligible character in a text (for example @xcite @xcite . this is especially useful in domains where ambiguities are not allowed, or where deducing the correct forms might pose challenges for non-experts. such domains may include religious texts, literary works like poetry, or educational material. there are benefits for human readers, like facilitating learning. however, prior research suggests that extensive diacritization can inadvertently impede skilled reading by increasing the required processing time @xcite @xcite @xcite . nevertheless, diacritics are important morphological markers, even when excessive, and may benefit automated systems in language modeling, machine translation (mt), part-of-speech tagging, morpho- system sentence full diacritization truth -mv, hard-partial diacritization td2 d2 -d2-sp-hard @xmath0.1 @xmath1.01 ## motivation previous research underscores the substantial influence of diacritization on the reading process of arabic text. skilled readers exhibit a tendency to read highly diacritized text at a slower pace compared to undiacritized. the effect on reading speed and accuracy has been substantiated across numerous studies involving diverse age groups and linguistic backgrounds @xcite @xcite @xcite @xcite . in addition, studies monitoring eye movements during reading link the increased reading times for diacritized text to an increase in fixation frequency and duration-a key indicator of the word identification process @xcite . partial diacritization may thus lead to a better reading experience for a wide range of readers, including in cases with dyslexia or visual impairments. ## arabic preliminaries diacritics in modern arabic orthography the inventory of diacritics used in modern typeset arabic comprises at least four functional groups. first are h . arakāt and sukūn, which indicate vowel phonemes or their absence. then tanwīn and shaddah indicate case-inflection morphemes and consonant lengthening. there are other marks, e.g. for recitation features in the qur'an. only the marks in ## behavioral experiment on partial diacritization we start with the hypothesis that reading partially diacritized text may be easier than reading fully diacritized text, while providing some benefit in disambiguation or ease-of-reading. to test this hypothesis, we conduct a behavioral experiment using machine-predicted partial diacritization masks. the diacritical marks themselves are the true labels. this is to ensure that the presented text is accurate, even if the model output is sub-optimal. the question is then: does ccpd using f sent and f word select useful letters to be diacritized? see the results in figure 4 . ## related work the literature has dealth with both types of diacritization: full (fd) and partial (pd). the majority has focused on fd. in pd, a distinct approach that seeks to augment reading comprehension by incorporating only the minimal requisite diacritics, we note a few works focus @xcite . other works take a morphological analysis approach @xcite @xcite @xcite . neural methods recent literature has focused more on neural-based systems. @xcite were the first to show that recurrent neural models are suitable candidates to learn the task entirely from data without resorting to manually engineered features such as morphological analyzers and pos taggers. alkhamissi et al. (2020) used a hierarchical, bilstm-based model that operates on words and characters separately with a crosslevel attention connecting the two-enabling sota task performance and faster training and inference compared to traditional models. many prior works have used recurrent-based models for the fd task @xcite @xcite @xcite , others used the transformer architecture @xcite , while others used convnet architectures @xcite . hybrid methods some works have combined neural and rule-based or other methods to improve arabic diacritization @xcite @xcite @xcite . ## discussion in this work, we have presented a novel approach for partial diacritization using context-contrastive prediction. the task is motivated by a large body of literature on the impact of diacritization coverage on the reading process of arabic text, and the scarcity of research into systems to automate it. ## conclusion in conclusion, our ccpd approach for partial diacritization using context-contrastive prediction contributes to the field of diacritization in an area with far-reaching implications for enhancing the reading experience of arabic text. grounded in previous research on the influence of diacritization on reading and further supported by behavioral experiments conducted in this work-our approach paves the way for advancements that benefit readers of diverse backgrounds and abilities. since our approach integrates seamlessly with existing arabic diacritization systems, it can be used by models trained on domain-specific text as well. we also propose a battery of performance indicators to gauge the competence of partial diacritization systems using fully-diacritized test data to mitigate the lack of publicly available benchmarks. finally, we introduce td2-a transformer adaptation of the d2 model which offers a different performance profile as shown by our proposed indicators. as arabic nlp continues to evolve, our approach serves as a promising direction for enhancing diacritization and its impact on text accessibility and comprehension.