R e L i K : Retrieve and L in K , Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget (2024)

Abstract: Entity Linking (EL) and Relation Extraction (RE) are fundamental tasks in Natural Language Processing, serving as critical components in a wide range of applications. In this paper, we propose ReLiK, a Retriever-Reader architecture for both EL and RE, where, given an input text, the Retriever module undertakes the identification of candidate entities or relations that could potentially appear within the text. Subsequently, the Reader module is tasked to discern the pertinent retrieved entities or relations and establish their alignment with the corresponding textual spans. Notably, we put forward an innovative input representation that incorporates the candidate entities or relations alongside the text, making it possible to link entities or extract relations in a single forward pass and to fully leverage pre-trained language models contextualization capabilities, in contrast with previous Retriever-Reader-based methods, which require a forward pass for each candidate. Our formulation of EL and RE achieves state-of-the-art performance in both in-domain and out-of-domain benchmarks while using academic budget training and with up to 40x inference speed compared to competitors. Finally, we show how our architecture can be used seamlessly for Information Extraction (cIE), i.e. EL + RE, and setting a new state of the art by employing a shared Reader that simultaneously extracts entities and relations.

## introduction extracting structured information from unstructured text lies at the core of many ai problems, such as information retrieval @xcite , knowledge graph construction @xcite , knowledge discovery @xcite , automatic text summarization @xcite , language modeling @xcite , automatic text reasoning @xcite , and semantic parsing @xcite , inter alia. looking at the variety of applications in which ie systems are used, we argue that such systems should strive to satisfy three fundamental properties: inference speed, flexibility, and performance. this work focuses on two of the most popular ie tasks: entity linking and relation extraction. while tremendous progress has recently been made on both el and re, to the best of our knowledge, recent approaches only focus on at most two out of the aforementioned three properties simultaneously (usually either performance and inference speed @xcite , or performance and flexibility @xcite ), hindering their applicability in multiple scenarios. here, we show that by harnessing the retriever-reader paradigm @xcite , it is possible to use the same underlying architecture to tackle both tasks, improving the current state of the art while satisfying all three fundamental properties. most importantly, our models are trainable on an academic budget with a short experiment life cycle, leveling the current playing field and making research on these tasks accessible for academic groups. our relik system frames el and re similarly to recent open domain question answering (odqa) systems @xcite where, given an input question, a bi-encoder architecture (retriever) encodes the input text and retrieves the most relevant text passages from an external index containing their encodings. then, a second encoder (reader) takes as input the question and each retrieved passage separately and extracts the answer, if it is present, from a specific passage. for our tasks, el and re, the input query corresponds to the sentence in which we have to link entities and/or extract relations; the retrieved passages are the entities' or relations' definitions; and predicting an answer translates into linking the entities and/or extracting the relations. however, our framing differs from most famous odqa ones in two main respects: i) for both el and re, the input text contains multiple questions simultaneously since there might be multiple entities to link, and/or multiple relations to extract; ii) we encode the input text with all its retrieved passages (i.e., the textual representations of the candidate entities or relations), linking all the entities or extracting all the relational triplets in a single forward pass. our architecture can thus be divided conceptually into two main components: relik innovates and integrates various unique properties and benefits: first, leveraging the nonparametric memory, i.e., the knowledge base accessed by the retriever component, considerably lowers the number of parameters required by the final model in order to achieve state-of-the-art performance (inference speed). second, using textual representations for entities/relations combined with the retriever component makes it easier for the model to zero-shot on unseen entities/relations (flexibility). finally, using our novel input formulation we exploit to the fullest the contextualization capabilities of novel language models such as deberta-v3 @xcite . indeed, by way of an extensive array of experiments, we show that encoding the input text and the textual representation of entities/relations and linking/extracting them in the same forward pass improves both model's final performance and processing speed (performance and inference speed). to foster research and usage of relik, we release the code and models' weights at https: //github.com/sapienzanlp/relik . relation extraction (re) is the task of extracting semantic relations between entities found within a given text from a closed set of relation types coming from a reference knowledge base. formally, for an input text q and a closed set of relation types r, re consists of identifying all triplets {(m, m', r) : (m, m') ∈ m(q) × m(q), @xmath0, respectively, the subject and object spans and r a relation between them. the combination of both el and re as a unified task is known as closed information extraction (cie). ## the reader-retriever (rr) paradigm in this section, we introduce relik, our retriever-reader architecture for el, re, and cie. while the retriever is shared by the three tasks (section 3.1), the reader has a common formulation for span identification, but differs slightly in the final linking and extraction steps (section 3.2). figure 1 shows a high-level overview of relik as a unified framework for el, re and cie. ## entity linking we now describe the experimental setup (section 4.1) and compare our system to current state-ofthe-art solutions (section 4.2) for el. ## relation extraction and closed information extraction in this section, we present the experimental setup (section 5.1) for re and cie, and compare the results of our systems to the current state of the art (section 5.2). ## future work the results presented in this paper demonstrate strong performance on held-out benchmarks; however, the robustness of our approach needs further testing across different domains and text varieties. this is further discussed in the limitations section (8). we see this as an opportunity for future research. the performance of recent systems for both el and re is reaching a plateau on many benchmarks. we believe a framework like relik, which is both fast and cost-effective to train and use, will facilitate a renewed focus on the nature of the data used for training and testing el and re systems. we encourage research in this direction. in particular, we identify emerging entities @xcite and the automatic generation of entity and relation verbalizations @xcite as promising areas for further exploration. addressing these issues would reduce the reliance on static indexes and human-generated descriptions. ## conclusion in this work, we presented relik, a novel and unified retriever-reader architecture that attains state-of-the-art performance seamlessly for both entity linking and relation extraction. furthermore, taking advantage of the common architecture and using a shared reader, our system is capable of achieving unprecedented performance and efficiency even on the closed information extraction task (i.e., entity linking + relation extraction). our models are considerably lighter, an order of magnitude faster, and trained on an academic budget. we believe that relik can advance the field of information extraction in two directions: first, by providing a novel framework for unifying other ie tasks beyond el and re, and, second, by providing accurate information for downstream applications in an efficient way.