Entity Retrieval for Answering Entity-Centric Questions (2025)

Abstract: The similarity between the question and indexed documents is a key factor in document retrieval for retrieval-augmented question answering. Although this is typically the only method for obtaining the relevant documents, it is not the sole approach when dealing with entity-centric questions. We study Entity Retrieval, an alternative retrieval method, which rather than relying on question-document similarity, depends on the salient entities within the question to identify the retrieval documents. We conduct an in-depth analysis of the performance of both dense and sparse retrieval methods in comparison to Entity Retrieval. Our findings reveal the great potential of entity-driven methods for improving augmentation document retrieval in both accuracy and efficiency.

## introduction information retrieval has significantly enhanced the factual reliability of large language model (llm) generated responses @xcite in question answering @xcite . this improvement is particularly evident in retrieval-augmented generation (rag; @xcite @xcite , which typically employs the retriever-reader architecture @xcite . rag retrievers can be sparse @xcite , dense @xcite , or hybrid @xcite , while the readers are usually generative language models 1 such as bart @xcite , t5 @xcite , or @xcite that generate answers based on the documents identified by the retriever. recent rag methodologies leverage the in-context learning capabilities of llms to incorporate retrieved documents into the 1 the readers in the original architecture were designed to extract answer spans rather than generate answers. prompt @xcite @xcite . entity-centric questions seek concise factual answers about the real world, typically in the form of single words or short phrases. these answers often reference or directly stem from a knowledge base entity @xcite , and retrieval-augmentation enhances llm performance in answering such questions, particularly for rare entities that appear infrequently in llm training and fine-tuning data @xcite . but is there a correlation between the quality of the retrieved documents and the generated response quality? @xcite found that dense retrievers retrieve less relevant documents for answering entity-centric questions than simpler sparse retrievers. additionally, @xcite show that the presence of irrelevant documents leads to worse answers. these findings underscore the crucial role of the retrieval module, particularly for entity-centric questions. in this paper, we propose entity retrieval (figure 1b ), which uses salient entities in the question to lookup knowledge base (e.g., wikipedia) articles that correspond to each entity. each article is truncated to the first w words to form a document set that augments the question passed to the llm. our contributions are as follows: (1) we propose entity retrieval, a novel method of acquiring augmentation documents using salient entities in the questions, (2) we compare the retrieval performance quality of several retrieval techniques (both dense and sparse) to entity retrieval for questions within two entity-centric question answering datasets, (3) we study the retrieval-augmentation quality of the compared techniques and entity retrieval, using salient entity annotations of the questions, and (4) we examine the application of a recent state-of-the-art entity linking method for entity retrieval in the absence of entity annotations in entity-centric questions. 2 retrieval for retrieval-augmentation retrieval-augmentation @xcite ) can be employed as a method of converting closedbook question answering foot_0 @xcite into extractive question answering @xcite , where the answers can be directly extracted from the retrieved documents. despite the abundance of effective retrieval techniques for retrieval-augmented question answering in existing literature @xcite @xcite @xcite , this section will concentrate on a select few methods foot_1 utilized to study answering entitycentric questions in this paper. bm25 @xcite is a probabilistic retrieval method that ranks documents based on the frequency of query terms appearing in each document, adjusted by the length of the document and overall term frequency in the collection. it operates in the sparse vector space, relying on precomputed term frequencies and inverse document frequencies to retrieve documents based on keyword matching. dpr (dense passage retrieval; karpukhin et al., 2020) leverages a bi-encoder architecture, wherein the initial encoder processes the question and the subsequent encoder handles the passages to be retrieved. the similarity scores between the two encoded representations are computed using a dot product. typically, the encoded representations of the second encoder are fixed and indexed in faiss @xcite , while the first encoder is optimized to maximize the dot-product scores based on positive and negative examples. ance @xcite is another dense retrieval technique similar to dpr foot_2 . it employs one encoder to transform both the questions and passages into dense representations. the key distinction from dpr is that ance uses hard negatives generated by periodically updating the passage embeddings during training, which helps the model learn more discriminative features, thereby enhancing retrieval performance over time. ## entity retrieval for question answering while quite powerful, most retrieval-augmented systems are notably time and resource-intensive, necessitating the storage of extensive lookup indices and the need to attend to all retrieved documents to generate the response (see section 4.7). this attribute renders such methods less desirable, particularly given the drive to run llms locally and on mobile phones @xcite . entity recognition has been an integral component of statistical question answering systems @xcite . additionally, the extensively studied field of knowledge base question answering @xcite has underscored the significance of entity information from knowledge bases in question answering @xcite . a traditional neural question answering pipeline may figure 2 : the first paragraph of the wikipedia article typically provides an informative summary for the entity. for example, the first paragraph of swan lake wikipedia article contains the answer to "who is the composer of the swan lake ballet?" contain entity detection, entity linking, relation prediction, and evidence integration @xcite , where entity detection can employ lstm-based @xcite or bert-based @xcite encoders. inspired by this body of work, we investigate the relevance of retrieval based on entity information as an alternative strategy to the proposed retrieval methods of section 2, especially for answering entity-centric questions with llms. our proposed method, entity retrieval, leverages the salient entities within the questions to identify and retrieve their corresponding knowledge base articles. we will then truncate these articles to the first w words foot_3 to form the list of the documents augmenting entity-centric questions when prompting llms. figure 1 presents a schematic comparison between entity retrieval and other retrieval methods in identifying retrieval documents to enhance question answering with llms. figure 2 provides an intuitive example to motivate the effectiveness of entity retrieval. ## related work similar to our studies, @xcite investigate the impact of salient entities on question answering, and propose constructing oracle retrieval documents as the 300-word segment surrounding the ground-truth answer from the wikipedia page that contains the answer (entity name). our approach leverages salient entities from questions without directly involving answers. additionally, they primarily use entities to classify questions 10 our empirical results demonstrate that dpr follows the same trend. similar to our studies, @xcite ; @xcite focus on answering questions with minimal lexical overlap between the retrieved documents and the question text. however, they emphasize multi-hop question answering, using entity linking to extract entities from the question and leveraging knowledge base articles to guide the multi-hop process. in contrast, we utilize entity links to directly identify augmentation documents. @xcite employ entity linking to identify entities in the question, generating a set of seed entities, which are then expanded using the ppr algorithm to create a subgraph of the knowledge base containing relevant entities. a graph propagation algorithm subsequently learns representations for each node in the subgraph, and each representation is binary classified to determine if it answers the question. our approach differs as we focus on using llms, employing entity linking in a retrieval-augmented setting without relying on graph propagation. ## conclusion in this study, we focused on retrieval-augmented question answering, and explored various retrieval methods that rely on the similarity between the question and the content of the passages to be retrieved. we introduced a novel approach, entity retrieval, which deviates from the conventional textual similarity-based mechanism. instead, it capitalizes on the salient entities within the question to identify retrieval documents. our findings indicate that our proposed method is not only more accurate but also faster in the context of entity-centric question answering.