Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem (2022)

Abstract: We propose an autoregressive entity linking model, that is trained with two auxiliary tasks, and learns to re-rank generated samples at inference time. Our proposed novelties address two weaknesses in the literature. First, a recent method proposes to learn mention detection and then entity candidate selection, but relies on predefined sets of candidates. We use encoder-decoder autoregressive entity linking in order to bypass this need, and propose to train mention detection as an auxiliary task instead. Second, previous work suggests that re-ranking could help correct prediction errors. We add a new, auxiliary task, match prediction, to learn re-ranking. Without the use of a knowledge base or candidate sets, our model sets a new state of the art in two benchmark datasets of entity linking: COMETA in the biomedical domain, and AIDA-CoNLL in the news domain. We show through ablation studies that each of the two auxiliary tasks increases performance, and that re-ranking is an important factor to the increase. Finally, our low-resource experimental results suggest that performance on the main task benefits from the knowledge learned by the auxiliary tasks, and not just from the additional training data.

## introduction entity linking @xcite is the task of linking entity mentions in a text document to concepts in a knowledge base. it is a basic building block used in many nlp applications, such as question answering @xcite @xcite , word sense disambiguation @xcite ), text classification @xcite , and social media analysis @xcite . early definitions decompose the task of entity linking (el) into two subtasks: mention detection (md) and entity disambiguation (ed). many statistical and lstm-based methods propose to cast soccer -japan get lucky win, china in surprise defeat. japan began the defence of their asian cup title with a lucky 2-1 win against syria in a group c championship match on friday. but china saw their luck desert them [...] ## multi-task learning for autoregressive entity linking we propose an autoregressive entity linking model, that is trained along with two auxiliary tasks, and uses re-ranking at inference time. in this section, we first describe the main entity linking task. then, we define the two auxiliary tasks: mention detection and a new task, called match prediction. third, we train our multi-task learning architecture with a weighted objective. finally, we propose to use the match prediction module for re-ranking during inference. an overview of our architecture is in figure 2 . ## conclusions we propose a multi-task learning and re-ranking approach to autoregressive entity linking. our main two novelties address two weaknesses in the literature. first, whereas the two-step method of de cao et al. (2021a) improves performance, it relies on predefined sets of entity candidates. we propose to instead train mention detection as an auxiliary task to autoregressive el, in order to bypass the need for entity candidate sets, and to preserve the knowledge learned by mention detection. second, previous work suggests that a sizeable portion of errors could be corrected with re-ranking. we propose to use samples generated at training time to teach the model to re-rank outputs. our model establishes a new state of the art in both cometa and aida-conll. the increases in performance across both datasets show that our model can learn and leverage domain-specific knowledge, without using a candidate set or a knowledge base. to analyse our model, we devise three ablation study experiments, and show that our model benefits from both auxiliary tasks and re-ranking. in particular, we show that reranking plays a major role in increasing entity linking scores. then, we propose three low-resource experiments for auxiliary tasks. the results show that our model's performance is not only due to additional training datapoints, but also due to how we defined our auxiliary tasks.