Proper Noun Diacritization for A rabic W ikipedia: A Benchmark Dataset (2025)

Abstract: Proper nouns in Arabic Wikipedia are frequently undiacritized, creating ambiguity in pronunciation and interpretation, especially for transliterated named entities of foreign origin. While transliteration and diacritization have been well-studied separately in Arabic NLP, their intersection remains underexplored. In this paper, we introduce a new manually diacritized dataset of Arabic proper nouns of various origins with their English Wikipedia equivalent glosses, and present the challenges and guidelines we followed to create it. We benchmark GPT-4o on the task of recovering full diacritization given the undiacritized Arabic and English forms, and analyze its performance. Achieving 73% accuracy, our results underscore both the difficulty of the task and the need for improved models and resources. We release our dataset to facilitate further research on Arabic Wikipedia proper noun diacritization.

## introduction arabic wikipedia, like other language editions, has been a valuable resource for both its readers and nlp research. in this paper, we focus on a particular limitation rooted in arabic's abjad orthography, where diacritics are typically omitted @xcite except for children's books and religious texts. this omission leads to ambiguity in pronunciation and interpretation, especially for proper nouns. some arabic wikipedia articles address this issue by providing partial or full diacritization in their lead sentences. for instance, ςman 2 can refer to either ςumaan 'oman' or ςam∼aan 'amman' depending on the diacritization (figure 1 ). but more often than not, 1 https://github.com/camel-lab/camelprop 2 arabic hsb romanization @xcite . these diacritics are missing. in our dataset we found 99.45% of all entries had no diacritics. our intention is to solve this limitation. the work presented in this paper lies at the intersection of three commonly but often independently studied arabic nlp tasks: transliteration, diacritization, and lemmatization. transliteration is the mapping of words, primarily proper nouns, from one script to another, usually in the context of machine translation @xcite @xcite . it poses challenges due to misalignments between scripts, differences in representing phonology and morphology, and historical ad hoc conventions. diacritization, or diacritic restoration, aims at recovering omitted diacritics in languages that rely on them for disambiguation @xcite @xcite . while both transliteration and diacritization have been well studied for arabic, they are typically treated in isolation. an exception is the work of @xcite , which considers both in the context of arabic to english proper noun transliteration. lemmatization maps inflected words to their base forms. this is particularly important for morphologically rich languages such as arabic @xcite . in the context of wikipedia entries, providing the lemmas is useful to readers as it gives them a grounding on how to interpret and later inflect the word forms properly. more concretely, we focus here on mapping pairs of undiacritized arabic proper nouns and their english glosses to fully diacritized and lemmatized arabic forms. the task can be viewed as partial transliteration, where roman-script vowels help infer (or transliterate into) arabic diacritical marks. for example, nxjwan 'nakhchivan' (from figure 1 ) should ideally be mapped to nax.jiwaan, rather than incorrect alternatives like nix.jawaan or nux.jiwaan. we present a new dataset of 3,000 unique arabic wikipedia proper nouns annotated with gold lemma-level diacritizations. each entry is paired with its english wikipedia equivalent, enabling the study of joint diacritization and transliteration. we benchmark @xcite , which shows promising results but struggles with spelling variants and ambiguity. the dataset covers a range of named entities (people, places, and organizations) and includes 3,362 total pairs to reflect multiple valid diacritizations based on the gloss. our contributions are: the remainder of this paper is structured as follows. section 2 outlines arabic linguistic aspects. section 3 reviews related work. sections 4 and 5 describe our dataset and annotation process. section 6 presents evaluation results and error analysis. diacritic example fatha ba /ba/ damma bu /bu/ kasra bi /bi/ shadda b∼ /bb/ sukun b. /b/ dagger alif bá /ba:/ shadda + fatha b∼a /bba/ shadda + damma b∼u /bbu/ shadda + kasra b∼i /bbi/ long vowel /a/ baa /ba:/ long vowel /u/ buw /bu:/ long vowel /i/ biy /bi:/ shadda + long vowel /a/ bbaa /bba:/ shadda + long vowel /u/ bbuw /bbu:/ shadda + long vowel /i/ bbiy /bbi:/ glide w baw. /baw/ glide y bay. /bay/ ## datasets we work with the camelprop dataset, released as part of camelmorph, an arabic morphological analyzer, by @xcite . as noted in section 3, it consists of two parts: cp-sama and cp-wiki. we randomly selected 3,000 unique arabicscript proper nouns from cp-wiki for manual annotation, forming our dataset cp-wiki-d3k. in addition to frequency and phonological similarity, we examined the distribution of named en-tity categories, namely, personal and family names, locations, and organizations, across both the original cp-wiki dataset and the manually annotated subset, cp-wiki-d3k. the distributions were broadly similar, with location entities being the majority in both (cp-wiki: 77.1%, cp-wiki-d3k: 85.2%), followed by names and organizations. this consistency supports the representativeness of cp-wiki-d3k for studying diacritization across entity types. ## data annotation in this section, we discuss the diacritization guidelines we used, as well as the setup for initial automatic processing followed by manual correction. ## conclusion and future work we presented a new 3,362 entry dataset of arabic wikipedia proper nouns annotated with goldstandard lemma diacritizations, paired with their english equivalents. this resource enables the joint study of diacritization and transliteration in a realistic setting characterized by ambiguity and spelling variation. we benchmarked gpt-4o on this task, providing insights into its capabilities and limitations. while the model performs reasonably well, especially on frequent names, it struggles with rarer entries and variant mappings. looking ahead, we plan to expand the dataset with more diverse names, integrate it into a morphological analyzer, and explore fine-tuned models for diacritizing proper nouns in broader contexts. we also plan to fine-tune dedicated models for this task and develop more robust approaches to name ambiguity, especially with multiple valid diacritizations. we hope this resource advances arabic nlp and name normalization in multilingual settings like wikipedia.