UM 6 P & UL at W ojood NER shared task: Improving Multi-Task Learning for Flat and Nested A rabic Named Entity Recognition (2023)

Abstract: In this paper, we present our submitted system for the WojoodNER Shared Task, addressing both flat and nested Arabic Named Entity Recognition (NER). Our system is based on a BERT-based multi-task learning model that leverages the existing Arabic Pretrained Language Models (PLMs) to encode the input sentences. To enhance the performance of our model, we have employed a multi-task loss variance penalty and combined several training objectives, including the Cross-Entropy loss, the Dice loss, the Tversky loss, and the Focal loss. Besides, we have studied the performance of three existing Arabic PLMs for sentence encoding. On the official test set, our system has obtained a micro-F1 score of 0.9113 and 0.9303 for Flat (Sub-Task 1) and Nested (Sub-Task 2) NER, respectively. It has been ranked in the 6th and the 2nd positions among all participating systems in Sub-Task 1 and Sub-Task 2, respectively.

## introduction named entity recognition (ner) is a fundamental component for many natural language processing (nlp) applications, including information extraction, information retrieval, question-answering, and text summarization, among others @xcite . ner is a sequence labeling task that involves identifying and assigning predefined class labels to named entity mentions (individual words or spans of words), such as names of persons, locations, organizations, and more. based on the structure of named entities, they can be categorized as either flat or nested entities. flat named entities consist of contiguous word spans with non-overlapping structures. in contrast, nested named entities exhibit a more complex structure where a named entity encompasses or is part of other named entities @xcite . therefore, several tools, models, and datasets have been introduced to address both flat and nested ner tasks @xcite @xcite @xcite . however, most existing research in this field has primarily focused on languages with high resources, such as english. the arabic language encompasses three distinct language varieties: modern standard arabic (msa), classical arabic, and dialectal arabic. the latter refers to the diverse spoken dialects of arabic across the arab world. over the past two decades, significant attention has been paid to the arabic ner task, where several models, tools, and datasets have been proposed @xcite @xcite . however, it is important to note that most available resources have primarily focused on the modern standard arabic, including anercorp foot_0 , ace2004 foot_1 , ace2005 foot_2 , ontonotes5 foot_3 , and aqmar @xcite datasets. for dialectal arabic ner, @xcite have introduced a dataset sourced from twitter, covering both msa and dialectal arabic. similarly, salah and binti zakaria (2018) have compiled a ner corpus from religious texts specifically for classical arabic. most of the previously mentioned datasets have been introduced for the flat ner task and are limited to a single arabic language variety. to overcome these limitations, @xcite have presented the wojood dataset, specifically created for both flat and nested arabic ner tasks. this dataset has been collected from diverse sources, spanning various domains and topics. moreover, it is considered the largest available multi-domain and multi-dialectal arabic ner corpus. in this paper, we introduce our participating system to wojoodner shared task @xcite . our system is built upon a bert-based multi-task learning model, where each entity type is associated with a multi-class classification head that predicts the iob2 tag of a given input token. we have employed the same model for both the flat and nested ner sub-tasks. to encode input sentences, we have explored three arabic pretrained language models (plms): qarib @xcite , camelbert-mix @xcite , and ar-bertv2 @xcite . our final model is trained to minimize a multi-task variance loss penalty and loss function that combines the cross-entropy loss, the dice loss @xcite , the tversky loss @xcite , and the focal loss @xcite . our system is evaluated using the micro-average precision, recall, and f1 score. it has achieved a micro-f1 score of 0.9113 and 0.9303 on the test sets of flat (sub-task 1) and nested (sub-task 2) ner, respectively. our system achieved the 6th and 2nd positions, respectively, among all participating systems in sub-task 1 and sub-task 2 of the wojoodner shared task. it is worth mentioning that the best results were obtained using the ar-bertv2 sentence encoder @xcite for both sub-tasks. ## data wojoodner shared task organizers provide a rich and large dataset for arabic ner @xcite . the shared task organizers propose two sub-tasks: one for flat ner (sub-task 1) and one for nested ner (sub-task 2) in arabic. the provided dataset, namely wojood @xcite , is collected from various sources and covers several domains and topics. it consists of approximately 550k tokens, comprising sentences from msa and dialectal arabic. the authors have followed the ldc's ontonotes 5 annotation guidelines @xcite to label the wojood dataset. the dataset tokens are labeled using 21 entity types. additionally, they provided labels for both flat (wojood-flat) and nested (wojood-nested) arabic ner. to evaluate the annotation quality of the wojood dataset, the authors measured inter-annotator agreement using cohen's kappa. they have reported a macro kappa of 0.98 and 0.979, with and without the 'o' entity tag, respectively. both wojood-flat and wojood-nested have been split into 70%, 10%, and 20% for model training, development, and evaluation, respectively. it is worth mentioning that we have trained, vali-dated, and evaluated our models using the officially provided splits for training, validation, and development, respectively. furthermore, we do not employ any text preprocessing or normalization technique. ## system overview in this section, we present the overall architecture of our system's model and the employed training objectives. ## experiments and results in this section, we present the experimental settings and discuss the obtained results. ## discussion the results have shown that combining loss functions that deal with the class imbalance problem ## conclusion in this paper, we have presented our participating system to wojoodner shared task. our system relies on a bert-based multi-task learning model for both flat and nested arabic ner. for the input sentence encoding, we have assessed the performance of three arabic plms: qarib, camelbert-mix, and arbertv2. our best model is trained to minimize a multi-task variance loss penalty and loss function that linearly combines the cross-entropy loss, the dice loss, the tversky loss, and the focal loss. the proposed system is evaluated using the micro-average precision, recall, and f1 score. it has achieved a micro-f1 score of 0.9113 and 0.9303 on the test sets of flat (sub-task 1) and nested (sub-task 2) ner, respectively. besides, it has been ranked 6th and 2nd out of all participating systems in sub-task 1 and sub-task 2, respectively. besides, our best results are obtained using the arbertv2 sentence encoder for both sub-tasks.