Ertim at S em E val-2023 Task 2: Fine-tuning of Transformer Language Models and External Knowledge Leveraging for NER in F arsi, E nglish, F rench and C hinese (2023)

Abstract: Transformer language models are now a solid baseline for Named Entity Recognition and can be significantly improved by leveraging complementary resources, either by integrating external knowledge or by annotating additional data. In a preliminary step, this work presents experiments on fine-tuning transformer models. Then, a set of experiments has been conducted with a Wikipedia-based reclassification system. Additionally, we conducted a small annotation campaign on the Farsi language to evaluate the impact of additional data. These two methods with complementary resources showed improvements compared to fine-tuning only.

## introduction entity recognition and linking has now become a standard task for nlp, either as a goal in itself or as a preprocessing step for other goals. as for many other nlp tasks, developing an accurate model for a specific dataset (by language or domain) is still a challenge that heavily relies on resources, either pre-computed in a language model or as an external dataset. as the nlp research lab of inalco, ertim was particularly interested in multiconer ii because it gave us a good opportunity to: ## fine-tuning of huggingface language models we present experiments based on the fine-tuning of transformer language models for the multiconer ii task on specific languages. ## conclusion our work is based on three complementary directions: the fine-tuning of huggingface transformer language models, a reclassification system based on wikipedia, and an effort to provide additional annotations. the first axis allowed us to highlight key results regarding the use of different models on a variety of languages. the second and third axes correspond to the original contributions of our work about integrating complementary data sources. with our wikipedia-based reclassification system, we found that using the information available in wikipedia could correct the classification from a fine-tuned transformer model. we have shown the significant contribution of additional annotations to the fine-tuning of transformer language model, even if these annotations are modest in terms of volume and categorical coverage. we consider sharing as open-source the annotated data, the models produced and our experimental codes. as a follow-up to this work, it would be interesting to evaluate the evolution of pre-trained model performance according to the quantity and variety of annotations we include. also, our wikipediabased reclassification system has a lot of room for improvement, especially when it comes to handling mentions with no wikipedia page available and cases of ambiguity. overall, this work would be all the more interesting and robust when applied to more languages. we consider experimenting further with spacy as it is a user-friendly wrapper, including pytorch and huggingface components.