Direct Fact Retrieval from Knowledge Graphs without Entity Linking (2023)

Abstract: There has been a surge of interest in utilizing Knowledge Graphs (KGs) for various natural language processing/understanding tasks. The conventional mechanism to retrieve facts in KGs usually involves three steps: entity span detection, entity disambiguation, and relation classification. However, this approach requires additional labels for training each of the three subcomponents in addition to pairs of input texts and facts, and also may accumulate errors propagated from failures in previous steps. To tackle these limitations, we propose a simple knowledge retrieval framework, which directly retrieves facts from the KGs given the input text based on their representational similarities, which we refer to as Direct Fact Retrieval (DiFaR). Specifically, we first embed all facts in KGs onto a dense embedding space by using a language model trained by only pairs of input texts and facts, and then provide the nearest facts in response to the input text. Since the fact, consisting of only two entities and one relation, has little context to encode, we propose to further refine ranks of top-k retrieved facts with a reranker that contextualizes the input text and the fact jointly. We validate our DiFaR framework on multiple fact retrieval tasks, showing that it significantly outperforms relevant baselines that use the three-step approach.

## introduction knowledge graphs (kgs) @xcite @xcite , which consist of a set of facts represented in the form of a (head entity, relation, tail entity) triplet, can store a large amount of world knowledge. in natural language applications, language models (lms) @xcite are commonly used; however, their knowledge internalized in parameters is often incomplete, inaccurate, and outdated. therefore, several recent works suggest augmenting lms with facts from kgs, for example, in question answering @xcite and dialogue generation @xcite . however, despite the broad applications of the kgs, the existing mechanism for retrieving facts from them are, in many cases, unnecessarily complex. in particular, to retrieve facts from kgs, existing work @xcite @xcite relies on three sequential steps, consisting of span detection, entity disambiguation, and relation classification, as illustrated in figure 1a . for example, given an input text: "where was michael phelps born?", they first detect a span of an entity within the input, which corresponds to "michael phelps". then, they match the entity mention in the input to an entity id in the kg. those two steps are often called entity linking. finally, among 91 relations associated with the entity of michael phelps, they select one relation relevant to the input, namely "place of birth". the aforementioned approach has a couple of drawbacks. first, all three sub-modules in the existing pipeline require module-specific labels in addition to query-triplet pairs for training. however, in real-world, high-quality training data is limited, and annotating them requires significant costs. second, such a pipeline approach is prone to error propagation across steps @xcite . for example, if the span detection fails, the subsequent steps, such as relation classification, are likely to make incorrect predictions as well. third, certain modules, that match entities in queries to kgs or predict relations over kgs, are usually not generalizable to emerging entities and relations and cannot be applied to different kgs. it would be preferable to have a method that does not require kg-specific training and inference. to tackle these limitations, we propose to directly retrieve the relevant triplets related to a natural language query by computing their similarities over a shared representation space (see figure 1b ). the design of our direct retrieval framework is motivated by a pioneering work of open-domain question answering with documents @xcite , which showed the possibility of dense retrieval with simple vector similarities between the question and document embeddings. however, in contrast to the document retrieval scenario where documents have sufficient contexts to embed, it is unclear whether the lm can still effectively embed facts represented in the short triplet form for retrieval. also, compared to the document retrieval which additionally requires a reader to extract only the relevant piece of knowledge, our fact retriever itself can directly provide the relevant knowledge. to realize our fact retriever, we train it by maximizing similarities between representations of relevant pairs of input texts and triplets while minimizing irrelevant pairs, where we use lms for encoding them. we note that this process requires only text-triplet pairs without using extra labels, unlike the conventional pipeline approach for fact retrieval. after training, we index all triplets in the kg with the trained encoder in an offline manner, and, given the input query, we return the nearest triplets over the embedding space. this procedure simplifies the conventional three steps for retrieving facts from kgs into one. to further efficiently search the relevant triplets, we approximate the similarity calculation with vector quantization and hierarchical search based on clustering @xcite . we further note that, since we embed triplets using the lm, our retriever can generalize to different kgs without any modification, unlike some conventional retrieval systems that require additional training to learn new kg schema about distinct entities and relations types. we refer to our framework as direct fact retrieval (difar). we experimentally demonstrate that our direct retrieval on kgs works well; however, the fact represented in the triplet form has a limited context, since it consists of only two entities and one relation. also, similarity calculation with the independently represented input text and triplets is arguably simple, and might be less effective. therefore, to further improve the retriever performance, we additionally use a reranker, whose goal is to calibrate the ranks of retrieved triplets for the input text. in particular, we first retrieve k nearest facts with the direct retriever, and then use another lm which directly measures the similarity by encoding the input text and the triplet simultaneously. moreover, another objective of the reranker is to filter out irrlevant triplets, which are the most confusing ones in the embedding space of the direct retriever. therefore, to effectively filter them, we train the reranker to minimize similarities between the input text and the most nearest yet irrelevant triplets. we evaluate our difar framework on fact retrieval tasks across two different domains of question answering and dialogue, whose goals are to retrieve relevant triplets in response to the given query. the experimental results show that our di-far framework outperforms relevant baselines that use conventional pipeline approaches to retrieve facts on kgs, and also show that our reranking strategy significantly improves retrieval performances. the detailed analyses further support the efficacy of our difar framework, with its great simplicity. our contributions in this work are as follows: since some natural language applications require factual knowledge @xcite , existing literature proposes to use knowledge in kgs, and sometimes along with language models (lms) @xcite . to mention a few, in question answering domains, facts in kgs can directly be answers for knowledge graph question answering tasks @xcite , but also they are often augmented to lms to generate knowledge-grounded answers @xcite . similarly, in dialogue generation, some existing work augments lms with facts from kgs @xcite . however, prior to utilizing facts in kgs, fact retrieval -selection of facts relevant to the input context -should be done in advance, whose results substantially affect downstream performances. in this work, we propose a conceptually simple yet effective framework for fact retrieval, motivated by information retrieval. ## experimental setups we explain datasets, models, metrics, and implementations. for additional details, see appendix a. ## conclusion in this work, we focused on the limitations of the conventional fact retrieval pipeline, usually consisting of entity mention detection, entity disambiguation and relation classification, which not only requires additional labels for training each subcomponent but also is vulnerable to the error propagation across submodules. to this end, we proposed the extremely simple direct fact retrieval (difar) framework. during training, it requires only pairs of input texts and relevant triplets, while, in inference, it directly retrieves relevant triplets based on their representational similarities to the given query. further, to calibrate the ranks of retrieved triplets, we proposed to use a reranker. we demonstrated that our difar outperforms existing fact retrieval baselines despite its great simplicity, but also ours with the reranking strategy significantly improves the performances; for the first time, we revealed that fact retrieval can be easily yet effectively done. we believe our work paves new avenues for fact retrieval, which leads to various follow-up work.