Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition (2025)

Abstract: We introduce , a novel multi-dimensional corpus covering 16 Arabic dialects across 10 domains, resulting in 160 distinct corpora. The corpus comprises about 777k tokens, carefully collected and manually annotated with 21 entity types using both nested and flat annotation schemes - using the Wojood guidelines. While is useful for various NLP tasks like domain adaptation and transfer learning, this paper primarily focuses on benchmarking existing Arabic Named Entity Recognition (NER) models, especially cross-domain and cross-dialect model performance. Our benchmarking of four Arabic NER models using reveals a significant drop in performance of up to 38% when compared to the in-distribution data. Furthermore, we present an in-depth analysis of domain and dialect divergence and the impact of resource scarcity. We also measured the overlap between domains and dialects using the Maximum Mean Discrepancy (MMD) metric, and illustrated why certain NER models perform better on specific dialects and domains. is open-source and publicly available at https://sina.birzeit.edu/wojood/#download

## introduction ner is crucial in various nlp tasks, including machine translation @xcite , word sense disambiguation @xcite , data extraction @xcite , language understanding @xcite , question answering @xcite , and interoperability @xcite . state-of-the-art arabic ner models demonstrate impressive performance, achieving f 1 -scores above 90% @xcite . however, these models continue to face challenges across various domains and dialects @xcite . arabic dialects are low-resource in many nlp tasks, including arabic ner. msa benefits from a large pool of annotated ner resources, while dialects suffer from a lack of such datasets @xcite . additionally, existing datasets have mainly focused on a limited number of domains, such as wojood @xcite which covers two dialects and five domains. other resources such as ontonotes @xcite and @xcite focus exclusively on msa and are limited to the political news domain. the lack of labeled datasets for multiple dialects and domains makes developing and evaluating ner models in cross-domain and cross-dialect more challenging @xcite . konooz is a novel multi-dimensional ner corpus designed for benchmarking ner models across various domains and dialects. to the best of our knowledge, konooz is the first rich corpus that contains 10 domains in 16 different dialects. as shown in figure 1 , each corpus contains about 4k tokens. the msa corpus contains about 8k tokens. the corpus was manually collected from diverse sources, reflecting the diversity of arabic dialects. then, it was manually annotated by a team of 45 people. the annotation process involved labeling tokens with 21 distinct entity types, utilizing both flat and nested ner tags, adhering to the annotation guidelines presented in @xcite . we used konooz for benchmarking four arabic ner models, which revealed low performance across domains and dialects. the variations in the results underscore the urgent need for more diverse arabic ner datasets. furthermore, we leveraged konooz to conduct an in-depth analysis of lexical similarity across domains and dialects. we hypothesized that (i) dialects from the same geographic region are expected to exhibit low divergence, and (ii) the named entities from the same country or region in the training data improve model performance. by measuring the divergence between domains and dialects, we uncovered several insightful patterns and correlations. these divergences reveal the linguistic variations that directly impact the performance degradation of the trained models. efficiently measuring and reducing divergence is crucial for adapting models to the new domain-the topic of domain adaptation @xcite . in short, the main contributions of this paper are: the paper is organized as: section 2 reviews related work; section 3 presents konooz; section 4 reveals the lexical similarity; section 5 benchmarks ner models; and we conclude in section 6. ## ner benchmarks using konooz this section benchmarks four state-of-the-art arabic ner models in cross-domain and cross-dialect scenarios using konooz. first, we conducted in-distribution evaluations by training and evaluating four models using their testing datasets (wojood n ested , wojood f lat , ontonotes and an-ercorp ). second, we benchmarked these models against the 10 domains (out-of-domain evaluation) and the 16 dialects of konooz (out-of-dialect evaluation). since the four ner models follow different annotation guidelines, we mapped konooz entity types to align with each training data accordingly. the mapping and the benchmarking preparations are provided in appendix §a.6. we named the models based on their corresponding trained dataset. ## conclusion we presented konooz, a novel multi-dimensional corpus covering 16 dialects across 10 domains, resulting in 160 distinct corpora (777k tokens) annotated with 21 entity types. our in-depth lexical similarity analysis reveals both distinctions and similarities across different domains and dialects. our benchmarking of four arabic ner models in cross-domain and cross-dialect scenarios highlighted the challenges inherent to arabic dialects, revealing that models trained on msa struggle to generalize effectively to dialectal arabic.