Polar Ducks and Where to Find Them: Enhancing Entity Linking with Duck Typing and Polar Box Embeddings (2023)

Abstract: Entity linking methods based on dense retrieval are widely adopted in large-scale applications for their efficiency, but they can fall short of generative models, as they are sensitive to the structure of the embedding space. To address this issue, this paper introduces DUCK, an approach to infusing structural information in the space of entity representations, using prior knowledge of entity types. Inspired by duck typing in programming languages, we define the type of an entity based on its relations with other entities in a knowledge graph. Then, porting the concept of box embeddings to spherical polar coordinates, we represent relations as boxes on the hypersphere. We optimize the model to place entities inside the boxes corresponding to their relations, thereby clustering together entities of similar type. Our experiments show that our method sets new state-of-the-art results on standard entity-disambiguation benchmarks. It improves the performance of the model by up to 7.9 F1 points, outperforms other type-aware approaches, and matches the results of generative models with 18 times more parameters.

## introduction state-of-the-art approaches to entity linking, namely the task of linking mentions of entities in a text to the corresponding entries in a knowledge base (kb) @xcite , are nowadays large generative models @xcite which perform entity retrieval in a autoregressive way. this category of methods achieves the best results, as it allows effectively capturing relations between the context of a mention and entity descriptions. however, the preferred choice in large-scale applications are often methods based on dense retrieval @xcite @xcite , as they are easier to train and can be more than one order of magnitude faster @xcite . these approaches learn to represent entities and mentions separately in the same embedding space, so that, at inference time, the method only requires encoding the mention and retrieving the most similar entity. methods based on dense retrieval have the drawback of being very sensitive to the structure of the embedding space, thereby reaching lower accuracy compared to generative models @xcite . in this paper, we aim to close the gap with generative approaches by infusing structural information in the latent space of retrieval-based methods. recent work @xcite has shown the benefit of infusing prior factual knowledge in the models. in particular, @xcite reported that prior knowledge of the type of a mention would result in nearly perfect disambiguation performance. prior methods used type labels extracted from knowledge graphs (kgs) @xcite @xcite , but as kgs can be highly incomplete, we aim to define type information in a fuzzy and more fine-grained manner. the first time a world cup final was settled in a penalty shootout was in 1994, when italy lost to brazil. ## preliminaries we start by formalizing the entity-disambiguation problem, then we outline the main intuitions underlying methods based dense-retrieval. problem statement. the goal of entity disambiguation (ed) is to link entity mentions in a piece of text to the entity they refer to in a reference kb. for each entity e, we assume we have an entity description expressed as a sequence of tokens s @xmath0 dense-retrieval methods. methods based on dense retrieval @xcite learn to represent mentions and entities in the same latent space, often optimizing a cross-entropy loss of the form: ## duck: enhancing entity disambiguation with duck typing our approach builds on dense-retrieval methods and aims to enhance their performance using finegrained type information. ## a bi-encoder model with duck typing building on prior work, we used the method described in section 3 to train a bi-encoder model with the same architecture of @xcite . compared to @xcite , duck adds just a relation encoder which is only used at training time to represent relations as boxes. ## experiments this section provides a thorough evaluation of our approach. first, we show that duck achieves new state-of-the-art results on popular datasets for entity disambiguation, closing the gap between retrievalbased methods and more expensive generative models. then, we discuss several ablation studies, showing that incorporating type information using box embeddings in polar coordinates improves the performance of the model. finally, we dig into qualitative analyses, showing that our model is able to place entities in the correct boxes despite the incompleteness of the information in the kg. ## conclusion this paper introduced duck, a method to improve the performance of entity disambiguation models using prior type knowledge. the overall idea underlying our method was inspired by the concept of duck typing, as we defined types in a fuzzy manner, without any need for type labels. we introduced box embeddings in spherical polar coordinates and we demonstrated that using this form of representation allows effectively clustering entities of the same type. crucially, we showed that infusing structural information in the latent space is sufficient to close the gap between efficient methods based on dense retrieval and generative models. as a future line of research, it might be interesting to explore methods to infuse prior knowledge of entity types in generative models as well.