Knowledge-Aware Query Expansion with Large Language Models for Textual and Relational Retrieval (2025)

Abstract: Large language models (LLMs) have been used to generate query expansions augmenting original queries for improving information search. Recent studies also explore providing LLMs with initial retrieval results to generate query expansions more grounded to document corpus. However, these methods mostly focus on enhancing textual similarities between search queries and target documents, overlooking document relations. For queries like “Find me a highly rated camera for wildlife photography compatible with my Nikon F-Mount lenses”, existing methods may generate expansions that are semantically similar but structurally unrelated to user intents. To handle such semi-structured queries with both textual and relational requirements, in this paper we propose a knowledge-aware query expansion framework, augmenting LLMs with structured document relations from knowledge graph (KG). To further address the limitation of entity-based scoring in existing KG-based methods, we leverage document texts as rich KG node representations and use document-based relation filtering for our Knowledge-Aware Retrieval (KAR). Extensive experiments on three datasets of diverse domains show the advantages of our method compared against state-of-the-art baselines on textual and relational semi-structured retrieval.

## introduction large language models (llms) have been utilized to expand original queries with additional contexts, capturing similar semantics of target documents and hence improving retrieval performance @xcite . while direct generations of llms introduce problems such as hallucination, out-dated information, and lack of domain knowledge, recent methods @xcite @xcite explore augmenting llms with initial retrievals as contexts, e.g., pseudo relevance feedback (prf) @xcite : example query expansions generated by hyde @xcite , rar @xcite , and kar (ours) given a semi-structured product search query with both textual and relational requirements @xcite . while hyde and rar enrich the textual information, e.g., "wildlife" and "highly rated", they make up incorrect document relations, e.g., compatibility of "nikon coolpix p1000" with "f-mount lenses". in contrast, our kar utilizes document relations from knowledge graph, e.g., customers bought "nikon z7 ii" and "f-mount lenses" together, to generate semantically similarly and structurally related query expansions. 2010; @xcite , to generate expansions more grounded to domain-specific document corpus. though effective, existing methods mostly focus on enhancing the semantic similarities between expanded queries and target document texts. however, in real-world search scenarios, besides textual descriptions, documents are often inter-connected with certain types of relations @xcite . both textual and relational details are often queried by users in semi-structured manner @xcite @xcite to help better describes their search intents, which are overlooked by existing query expansion methods. for exam-ple, the user query in figure 1 specifies both textual requirements of the product "highly rated" and "wildlife photography", and relational requirement of the product "compatible with nikon f-mount lenses". while existing methods may generate semantically similar expansions on this product, they tend to make up incorrect product relations, i.e., compatibility between cameras and lenses, leading to suboptimal retrieval results. to handle such semi-structured queries, we propose a knowledge-aware query expansion framework, augmenting llms with structured document relations from knowledge graphs (kg). we first parse entities explicitly mentioned the original query with an llm and then retrieve textual documents of these entities as well as their associated nodes and relations on kg. while existing kg-based question answering methods @xcite @xcite filter out irrelevant relations by scoring the node relevance based on semantic similarity between the query and entity names, e.g., "nikon camera", they overlook the rich textual details of entities queried by users, e.g., "highly rated" and "wildlife". to address this, we leverage document texts as rich kg node representations and use documentbased relation filtering to extract query-focused relations. then, with collected textual and relational knowledge as inputs, llm generates query expansions that are grounded to the document corpus while preserving user-specified document relations. the expanded queries are then utilized for the final retrieval as our knowledge-aware retrieval (kar). extensive experiments are conducted on three textual and relational semi-structured retrieval datasets in the stark benchmark @xcite for product, academic paper, and biomedical search, respectively. the results show that our method outperforms state-of-the-art query expansion methods and achieves at least on par performances compared to llm-based retrieval agent. in summary, we make the following contributions: i) to handle complex search queries with both textual and relational requirements, we propose a knowledge-aware query expansion framework augmenting llms with kg; ii) to address the limitation of entity-based scoring, we use document texts as kg node representations and adopt document-based relation filtering for knowledge-aware retrieval (kar); iii) experiments on three semi-structured retrieval datasets show the advantages of our method and its practical applicability. ## problem definition we define our studied query expansion for textual and relational semi-structured retrieval as follows. follwing @xcite , suppose a knowledge base contains a collection of textual documents d and a knowledge graph @xmath0, r), where d @xmath1, and v @xmath2, with r being a set of relations between different nodes. for example, in a paper search scenario as figure 2 , each paper i has a textual document d i that contains abstract, venue, publication date, etc., and the corresponding node v i on the knowledge graph g encodes its relations with other nodes such as paper citations, authorship, and field of study. now given a query q with requirements from both unstructured texts in d and structured relations in g, the semi-structured retrieval @xcite @xcite is to out-put a set of documents a ⊆ d such that the entity described in each document satisfies both textual and relational requirements specified by query q. to bridge the gap between query and documents, we aims to augment the original query q with query expansions q e based on available textual and relational knowledge as ## methodology in this section, we describe our knowledge-aware query expansion framework as in figure 2 . entity parsing by llm as the initial step, we first utilize an llm to extract explicitly mentioned entities from the original query q given the document structures, denoted by e q . following similar ideas in @xcite , we also consider the original query q itself as a pseudo entity representing the target entity document to be retrieved, then, entities for a paper search query may include author names, paper titles, and the query itself. entity document retrieval for each mentioned entity @xmath3, we then use an off-the-shelf text embedding model to retrieve its associated textual document d i from d. as shown in figure 2 , author documents contain information of paper and citation counts while paper documents contain abstract and publication information such date and venue. ## results we show the results on test sets of synthetic queries in for query expansion baselines, we find that simply using initial retrieval results as expansions, i.e., prf, has little or even negative impact on final retrieval accuracy as low-quality initial retrievals can introduce noise for final retrievals. meanwhile, hyde employs an llm to generate query expansions directly with its intrinsic knowledge. however, without grounded textual knowledge from the document corpus, hyde only improves retrieval performance marginally. for more advanced llm-based methods, i.e., rar and agr, we observe that augmenting llm with initial retrievals as contexts before expansion and utilizing its selfrefinement abilities can indeed improve expansion quality and thus retrieval accuracy. however, the lack of relational knowledge can still lead to incorrect document relations limiting their performance on textual and relational semi-structured retrieval. for supervised baselines, according to @xcite , training challenges of encoding both textual and relational information as texts for dense retrievers and computational demands for graph neural network in qagnn lead to significant performance gaps. while llm-based agent avatar shows promising results after being optimized on training data, the cost and efficiency remain challenging with a high number of llm inferences. moreover, compared to mag and prime, we observe on amazon dataset higher general performance of all methods and smaller performance gaps between kar and baselines, e.g., avatar outperforms kar on hit@5 and r@20 metrics as shown in ## conclusion in this paper, we develop a knowledge-aware query expansion framework for textual and relational retrieval, utilizing kg relations between textual documents to enhance llms' query expansion generation. leveraging collaboratively textual and relational knowledge, we filter query-focused relations with document texts as rich kg node representations for our knowledge-aware retrieval. experiments on three semi-structured retrieval datasets of diverse domains demonstrate the advantages of our method compared against state-of-the-art query expansion methods, and showcase its applicability to handle real-world complex search queries.