Entity Disambiguation via Fusion Entity Decoding (2024)

Abstract: Entity disambiguation (ED), which links the mentions of ambiguous entities to their referent entities in a knowledge base, serves as a core component in entity linking (EL). Existing generative approaches demonstrate improved accuracy compared to classification approaches under the standardized ZELDA benchmark. Nevertheless, generative approaches suffer from the need for large-scale pre-training and inefficient generation. Most importantly, entity descriptions, which could contain crucial information to distinguish similar entities from each other, are often overlooked.We propose an encoder-decoder model to disambiguate entities with more detailed entity descriptions. Given text and candidate entities, the encoder learns interactions between the text and each candidate entity, producing representations for each entity candidate. The decoder then fuses the representations of entity candidates together and selects the correct entity.Our experiments, conducted on various entity disambiguation benchmarks, demonstrate the strong and robust performance of this model, particularly +1.5% in the ZELDA benchmark compared with GENRE. Furthermore, we integrate this approach into the retrieval/reader framework and observe +1.5% improvements in end-to-end entity linking in the GERBIL benchmark compared with EntQA.

## introduction entity linking (el) extracts references (a.k.a. mentions) to entities within a document and associates these mentions with their corresponding entries in a knowledge base (kb). el is a fundamental component in automatic text comprehension, with various practical applications such as question answering, text analysis, recommender systems, semantic search, and information retrieval. as the most critical component of el workflows, entity disambiguation (ed) aims to select the correct entity from a set of candidate entities, given textual references. for instance, the entity mention 'bert' may stand for 'the famous language model' @xcite or 'the golden yellow muppet character' depending on the given context. therefore, models need to understand context to disambiguate entities correctly. owing to its practical significance in the industry and the latest developments in utilizing pre-trained language models @xcite @xcite , various approaches for entity disambiguation have been introduced in recent years. primarily, existing methods can be categorized into two styles: classification approaches @xcite @xcite or generative approaches @xcite . classification approaches such as @xcite predict the masked entity titles while generative approaches such as @xcite directly decode entity titles. the recently proposed zelda benchmark @xcite standardizes the experimental setup (consistent training data, entity vocabulary, and candidate lists) and shows that generative approaches such as @xcite have significantly stronger performance under this experimental setup. however, @xcite argues that generative approaches require large scale pre-training. in particular, de @xcite critically relies on a prefix tree (also known as a trie) derived from wikipedia to constrain the beam search in order to produce a valid entity title in a given knowledge base (kb), which might be inefficient memorywise. in addition, since it directly generates a valid entity without reading their descriptions, crucial information in the descriptions might be ignored. therefore, disentangling significantly similar enti-ties proves challenging with this method @xcite . to better disentangle similar entities, in this paper we propose an encoder-decoder model that decodes entities by utilizing their descriptions. our approach is mainly inspired by a recent work on question answering @xcite . in particular, we make the following contributions: we summarize our contributions in the following: our approach outperforms strongest ed baselines @xcite @xcite on zelda benchmark and el baselines @xcite @xcite on gebil benchmark @xcite . for classification approaches, luke @xcite and fevry @xcite are two of the most well-known approaches due to their strong performance. luke is based on masked entity prediction. during the pre-training, luke combines input text and ground-truth entities as input tokens. then, it randomly masks entities from those ground-truth entities and predict those masked entities by leveraging both the input text and those unmasked entities. their model is trained on a large entity-annotated corpus obtained from wikipedia and achieves the current sota in several ed benchmark datasets. for generative approaches, genre @xcite uses bart weights from @xcite and is trained on a wikipedia corpus, learning to generate entity names in an autoregressive manner, conditioned on the provided context. at inference, genre employs a constrained beam search strategy that forces each generated name to be in a predefined entity set. conventionally, ed methods are evaluated on six datasets, msnbc, aquaint, ace2004, wned-cweb (cweb) and wned-wiki (wiki) @xcite . nevertheless, as shown in @xcite , those different ed methods use significantly different amounts of training data (ranging from 2 to 20 million annotated text) obtained with diverse sampling methodologies and enhanced weak labels @xcite , and completely different knowledge bases (ranging from few thousands to over 6 million) from different sources, @xcite or kilt @xcite and different candidate lists @xcite . thus, comparing various approaches is highly challenging. it is impossible to conclude which approach performs best @xcite . zelda @xcite ) benchmark is proposed to unify the training data set, entity vocabulary, and candidate lists to facilitate direct comparability of ed approaches. for this reason, we compare our approach with sota approaches on zelda benchmark. our experiment is rigorously conducted using the same training data, entity vocabulary, and candidate lists without additional information from wikipedia or using weak labels. entity linking. different from ed, the key challenge of el is its significantly large search space. a system can potentially generate any subset of conceivable spans in the document, each of which could correspond to an entity in a large kb, typically containing millions of entities. to manage this overwhelming scale, existing approaches break down el into two stage tasks: mention detection (md) and entity disambiguation (ed). these tasks are often tackled with varying degrees of independence. in most of these approaches, the sequence of subproblems is consistent: first, the system identifies possible entity mentions, and then it links these mentions to specific entries in the given knowledge base. this @xmath0. they either assume that mentions are provided in advance, following the example of @xcite or take a different route by employing readily available entity recognition systems to first identify mentions and then disambiguate them through the ed process, as evidenced in the works of @xcite . furthermore, some research @xcite trains an end-to-end autoregressive model that jointly performs @xmath1. recently, @xcite has shown that the classic @xmath2, which is unnatural and challenging. to fix this problem, the authors flip the order of md and ed, and propose an @xmath3. their key observation is that finding relevant candidate entities is easy without the knowledge of their specific mentions. their @xmath4. although their retriever (select top-k candidate entities) performs remarkably well, the majority of errors are attributed to their reader (which predicts the final entities and mention spans). a recent work @xcite ) proposes a structured prediction approach and achieves 88.6% on aida-conll test-b by using the pprforned @xcite candidate list. however, @xcite ; @xcite question this candidate list since it is unclear how candidates were pruned. the entity candidates generated by pprforned @xcite were found to be well-tailored to the aida-conll test-b evaluation dataset, with high recall and low ambiguity. models @xcite improve significantly when using these lists instead of the more generic lists by @xcite and @xcite , respectively. without the handcrafted pprforned @xcite candidate list, the result of aida-conll test-b in @xcite is the same as @xcite , 85.8%. as discussed in zelda @xcite , using additional signals makes comparison unfair and indirect. moreover, in real world entity linking applications, additional signals such as pruned candidate lists may not be available. therefore, same as our comparison methodology in ed, we do not bring any additional signals and aim to conduct an end-to-end direct entity linking comparison precisely by using the same training data and same knowledge base, kilt @xcite as entqa @xcite and genre @xcite . ## experiment we conduct extensive experiments to demonstrate the performance of our proposed approach (fusioned) over 20 datasets, addressing both single-entity disambiguation and end-to-end entity linking. the goal of our experiments is to facilitate a direct comparison, illustrating that under identical conditions (without incorporating extra training data or taking additional signals into account), our approach outperforms the current sota. ## conclusion we propose an encoder-decoder model architecture to enhance the disambiguation of entities by providing more detailed descriptions. the encoder, when given text and candidate entities learns the interactions between the text and each entity candidate, generating representations for each candidate. the decoder then combines these representations to produce the correct entity. our experiments, conducted on various entity disambiguation benchmarks, demonstrate the model's strong and robust performance. furthermore, we integrate this approach into the retrieval/reader el framework and observe improvements on the gerbil benchmark compared with previous sota. we also propose entity retrieval-augmented large language models (llms) for el. results show that compared to fusioned, llms generally underperform while they demonstrate strong improvements compared to sota over some datasets.