B angor U niversity at W ojood NER 2024: Advancing A rabic Named Entity Recognition with CAM e LBERT -Mix (2024)

Abstract: This paper describes the approach and results of Bangor University’s participation in the WojoodNER 2024 shared task, specifically for Subtask-1: Closed-Track Flat Fine-Grain NER. We present a system utilizing a transformer-based model called bert-base-arabic-camelbert-mix, fine-tuned on the Wojood-Fine corpus. A key enhancement to our approach involves adding a linear layer on top of the bert-base-arabic-camelbert-mix to classify each token into one of 51 different entity types and subtypes, as well as the ‘O’ label for non-entity tokens. This linear layer effectively maps the contextualized embeddings produced by BERT to the desired output labels, addressing the complex challenges of fine-grained Arabic NER. The system achieved competitive results in precision, recall, and F1 scores, thereby contributing significant insights into the application of transformers in Arabic NER tasks.

## introduction named entity recognition (ner) is a crucial task in natural language processing (nlp) aimed at identifying and categorizing named entities in text. the arabic language, with its rich morphology and variety of dialects, poses significant challenges for ner, necessitating tailored solutions. the wojoodner 2024 shared task, as established by @xcite , seeks to advance arabic ner by emphasizing fine-grained entity recognition. this year's challenge features the newly introduced wojood-fine corpus, which offers more detailed annotations and an expanded range of entity types compared to previous versions. bangor university's response to subtask-1 of this challenge involves a strategic use of the bertbase-arabic-camelbert-mix, a transformer-based model specifically optimized for arabic. this model is fine-tuned on the wojood-fine corpus to handle the intricacies of fine-grained ner. unique to our approach is the integration of a linear classification layer atop the pre-trained model, enhancing its ability to classify each token into one of 51 different entity types and subtypes, as well as the non-entity 'o' label. our methodological advancements are driven by the need to address specific challenges posed by the arabic script's absence of capitalization, its morphological diversity, and contextual ambiguities. the approach adopted for the wojood-fine corpus is meticulously designed to improve model generalization across varied arabic texts, thereby enhancing performance metrics such as precision, recall, and f1 scores. the paper further discusses specific challenges encountered during the task, such as managing the complex nature of fine-grained entities and optimizing the model to achieve robust performance across diverse dialects and textual genres. through this participation, bangor university not only contributes to advancing the state of arabic ner but also offers insights that are valuable to the broader nlp community, showcasing effective strategies to tackle the linguistic complexities of arabic ner. ## related work ner has significantly advanced from identifying simple flat entities to complex nested entities, particularly in arabic, a language rich with script and dialect variations. historically limited by the scope of available corpora, early arabic ner datasets like anercorp @xcite and ace2005 @xcite were focused on modern standard arabic from news and political domains, offering primarily flat annotations. the introduction of the caner corpus @xcite expanded to 14 entity types in ## data the wojoodner 2024 shared task utilizes the wojood-fine corpus introduced by @xcite , a robust dataset tailored for arabic ner challenges. the dataset is structured to enhance the granularity and scope of entity recognition within arabic text, catering to a fine-grained classification scheme that extends beyond traditional ner datasets. ## our model in our approach to the ner shared task 2024 subtask-1 (closed-track flat fine-grain ner), we utilized a transformer-based model, specifically the camel-lab's bert-base-arabic-camelbert-mix, which was introduced by @xcite due to its robust pre-training on a diverse arabic language corpus. this choice was motivated by the model's proven capability in understanding the complexities and nuances of the arabic language, which is crucial for the fine-grained named entity recognition required in this task. ## results the results obtained from the ner system developed for the wojoodner 2024 competition exhibit strong performance across training, validation, and testing phases, as highlighted in the