m R e F in ED : An Efficient End-to-End Multilingual Entity Linking System (2023)

Abstract: End-to-end multilingual entity linking (MEL) is concerned with identifying multilingual entity mentions and their corresponding entity IDs in a knowledge base. Existing works assumed that entity mentions were given and skipped the entity mention detection step due to a lack of high-quality multilingual training corpora. To overcome this limitation, we propose mReFinED, the first end-to-end multilingual entity linking. Additionally, we propose a bootstrapping mention detection framework that enhances the quality of training corpora. Our experimental results demonstrated that mReFinED outperformed the best existing work in the end-to-end MEL task while being 44 times faster.

## introduction end-to-end entity linking (el) is the task of identifying entity mentions within a given text and mapping them to the corresponding entity in a knowledge base. end-to-end el plays a crucial role in various nlp tasks, such as question answering @xcite @xcite and information retrieval @xcite . to clarify the terminology used in this paper and previous work, it should be noted that when referring to el in previous work, we are specifically referring to entity disambiguation (ed) where the entity mentions are given. we will only refer to our definition of el (mention detection with ed) using the full name "end-to-end el". existing el research has extended models to support over 100 languages in a single model using wikipedia as the training corpus. we call this task multilingual entity linking (mel). recent work proposed mel frameworks by minimizing discrepancy between mention and entity description representation @xcite or the same mention but in different contexts @xcite based on bi-encoder pre-trained language models @xcite . an alternative method is predicting the target entity's wikipedia title in an auto-regressive manner @xcite by extending the sequence-to-sequence pipeline of @xcite . however, none of the existing works perform end-to-end mel because of a lack of high-quality multilingual entity mention training resources. for example, we found that wikipedia suffers from an unlabelled entity mention problem, i.e. not all entity mentions have hyperlink markups to train a reliable mention detection model. thus, devising an end-to-end mel system remains a challenging task. in this paper, we propose the first end-to-end mel system. to address the unlabelled mention problem in end-to-end mel, we propose a bootstrapping mention detection (md) framework. our framework leverages an existing multilingual md model to create a bootstrapped dataset, which we use to train a new mention detection model for annotating unlabelled mentions in wikipedia. the framework provides an improvement for detecting named and non-named entities in wikipedia, compared to previous multilingual md approaches @xcite @xcite . to construct the end-to-end mel system, we extend refined @xcite since it is comparable to the state-of-the-art (sota) models in the english end-to-end el setting, and significantly faster than any other methods to date. we call this new model mre-fined. our code is released at: https: //github.com/amazon-science/ refined/tree/mrefined. to demonstrate mrefined's effectiveness, we compare it with sota mel @xcite on the end-to-end mel task across two datasets, mewsli-9 @xcite and tr2016 hard @xcite . experimental results show that mrefined outperforms a two-stage model (combining sota md and mel models) on both datasets. moreover, mrefined's inference speed is 44 times faster than sota mel. our contributions are as follows: we propose the first end-to-end mel in a single model by extending refined to multilingual refined. in addition, we propose a bootstrapping mention detection framework to solve the unlabelled mention problem in end-to-end mel. ## methodology overview. we first fine-tune a mention detection (md) model -based on a multilingual pre-trained language model (plm) -with our bootstrapping md framework, as shown in figure 1 . we then use the bootstrapped md model to annotate unlabelled mentions in wikipedia. finally, we use the data bootstrapped to train mrefined in a multitask manner @xcite , which includes mention detection, entity type prediction, entity description, and entity disambiguation. ## conclusion in this paper, we propose mrefined, the first multilingual end-to-end el. we extend the monolingual refined to multilingual and add the new bootstrapping md framework to mitigate the unlabelled mention problem. mrefined outperformed sota mel in the end-to-end el task, and the inference speed is faster than sota 44 times.