ELYADATA at W ojood NER Shared Task: Data and Model-centric Approaches for A rabic Flat and Nested NER (2023)

Abstract: This paper describes our submissions to the WojoodNER shared task organized during the first ArabicNLP conference. We participated in the two proposed sub-tasks of flat and nested Named Entity Recognition (NER). Our systems were ranked first over eight and third over eleven in the Nested NER and Flat NER, respectively. All our primary submissions are based on DiffusionNER models (Shen et al., 2023), where the NER task is formulated as a boundary-denoising diffusion process. Experiments on nested WojoodNER achieves the best results with a micro F1-score of 93.73%. For the flat sub-task, our primary system was the third-best system, with a micro F1-score of 91.92%.

## introduction named entity recognition is the task of locating a word or a phrase that references a particular entity within a given text. it is among the most prominent challenges in natural language processing (nlp). ner has been an active research area with growing interest and significant development over the past twenty years. this increased interest has led to the organization of multiple ner shared tasks and evaluation campaigns, especially for english @xcite and some other languages @xcite @xcite . as for the used techniques and methods applied to ner, there are mainly four approaches according to @xcite : rule-based algorithms, supervised and unsupervised machine learning algorithms, and deep-learning algorithms. the rule based approach relies on predefined grammatical rules and dictionaries to identify named entities. these systems are precise, but do not generalize well. supervised learning approaches can achieve high accuracy, yet demand labelled data and may give poor results for unseen domains. in contrast, unsupervised learning techniques retrieve named entities without requiring labelled data. however, the absence of labels makes it challenging to assess the performance of unsupervised models effectively @xcite . the advent of deep learning models like transformers @xcite which excel across domains and languages, enabled researches to make bigger strides towards better performance. multiple studies have addressed the challenges of identifying and classifying named entities in arabic text. notable initiatives include the work by @xcite , which introduced a ner system relying only on n-grams and maximum entropy, and it achieved an f1-score of 54.11% on anercorp dataset @xcite . additionally, @xcite has used deep learning methodologies to enhance the performance of arabic ner with an f1-score 88.64% on the same dataset @xcite . furthermore, multiple pretrained models such as arabert, marbert, and jaber were fine-tuned to achieve respectively 90.51%, 80.5% and 84.20% f1-score on anercorp @xcite . despite these previous efforts and progress made for arabic ner, nested ner still constitutes a challenging task not well studied in arabic. nested ner refers to the particular case of ner where entities are nested within each other, possibly with different tags. another common challenge in arabic ner lies the specific nature of the language itself: arabic is a morphologically rich and highly ambiguous language with numerous dialectal variants and a significant amount of code-switching @xcite . these factors, combined, make ner in its nested or flat variants, a particularly challenging task when performed on arabic and dialectal arabic. hence, the motivation for the 2023 ner shared task @xcite which aims to produce models that can overcome the aforementioned challenges. the main contribution of this work can be summarized as follows: this paper is organized as follows. section 2 describes the wojood dataset. section 3 presents the implemented flat and nested tasks, and section 4 details the experiments and the obtained results. the overall results are discussed in section 5 before concluding the paper in section 6. ## dataset the ner shared task dataset "wojood" @xcite consists of 16817 sentences with an average sentence length of 23.45 words and a vocabulary size of 44881 for training, 3133 sentences with an average length of 17.8 and a 13134 vocabulary size for validation. the test set has 5990 sentences with an average length of 18.68 and a vocabulary size of 20920. it comprises both modern standard arabic (msa) and dialectal arabic sentences. however, as detailed in ## ner systems in this work, we approached the ner task with two different methods: a data-centric approach encompassing dataset re-sampling and data preprocessing and a model-centric approach using several model architectures. ## experiments and results in order to reproduce the results obtained in @xcite , we started by training a multitaskbased baseline model for each evaluation condition (flat and nested) using @xcite . ## discussion we started our experiments by re-sampling the data set in order to alleviate the data imbalance problem. our experiments show that not all the tested resampling improves the f1 score of the wojood data-set. we also tested model-centric methods using two encoder-only models, namely piqn and diffusionner. both models have been tested using various pretrained encoders: biobert, arabert and marbert. using biobert has decreased the model performance. this is not surprising, given the fact that biobert is mainly trained on biomedical english text, which does not fit our use-case. we also noted that the results of piqn and dif-fusionner are better without dataset re-sampling. we observed this when we trained piqn and diffu-sionner models after re-sampling the flat ner dataset. however, due to time constraints, we didn't run the same experiments for nested ner. another observation made during the experiments is that the use of an arabert encoder yields better results compared to marbert despite the presence of dialectal sentences in the latter. this can be attributed to the pretraining of marbert being exclusively on tweets, or to the possibility that the wojood dataset has more msa content than dialectal. ## conclusion this paper presents results obtained on two ner sub-tasks of the wojood shared task, namely flat and nested ner. our submission relies on the usage of data-centric and model-centric approaches. data-centric consists of a set of re-sampling methods intended to mitigate the unbalanced nature of wojood data-set. various re-sampling method are implemented and led to only limited success. model-centric approaches, for their part, are designed to train the best model for a given dataset. we experimented with piqn and diffusionner models trained using various pre-trained encoder. remarkably, the diffusionner fine-tuned with an arabert sentence encoder module without any re-sampling or pre-processing, yielded to significant improvements over the baseline results for both nested and flat ner. this allows us to be ranked first out of eight for the nested ner sub-task with a 93.93% f1-score and third out of eleven for the flat ner sub-task with a 91.92% f1-score.