Corpus-Steered Query Expansion with Large Language Models (2024)

Abstract: Recent studies demonstrate that query expansions generated by large language models (LLMs) can considerably enhance information retrieval systems by generating hypothetical documents that answer the queries as expansions. However, challenges arise from misalignments between the expansions and the retrieval corpus, resulting in issues like hallucinations and outdated information due to the limited intrinsic knowledge of LLMs. Inspired by Pseudo Relevance Feedback (PRF), we introduce Corpus-Steered Query Expansion (CSQE) to promote the incorporation of knowledge embedded within the corpus. CSQE utilizes the relevance assessing capability of LLMs to systematically identify pivotal sentences in the initially-retrieved documents. These corpus-originated texts are subsequently used to expand the query together with LLM-knowledge empowered expansions, improving the relevance prediction between the query and the target documents. Extensive experiments reveal that CSQE exhibits strong performance without necessitating any training, especially with queries for which LLMs lack knowledge.

## introduction query expansion enhances the effectiveness of information retrieval systems by incorporating additional texts into the original query, which are traditionally identified via pseudo-relevance feedback @xcite or by leveraging external lexical knowledge sources @xcite . recent studies @xcite @xcite show query expansions generated by llms are able to significantly boost retrieval effectiveness, especially in zero-shot scenarios. for instance, gao figure 1 : overview of csqe. given a query biology definition and the top-2 retrieved documents, csqe utilizes an llm to identify relevant document 1 and extract the key sentences from document 1 that contribute to the relevance. the query is then expanded by both these corpus-originated texts and llm-knowledge empowered expansions (i.e., hypothetical documents that answer the query) to obtain the final results. @xcite demonstrates the effectiveness of utilizing llms to generate hypothetical documents answering the original query as additional texts to augment the query. @xcite show the efficacy of applying pseudo-relevance feedback upon the llm-generated answers for expansion. despite variations in prompts or expansion methods, a common foundational element across these approaches is the reliance on the intrinsic knowledge of llms. despite their effectiveness, generations that rely on the intrinsic parametric knowledge within llms encounter various issues. these include hallucination @xcite , inability to update @xcite , and a deficiency in long-tail knowledge @xcite . such generations may introduce irrelevant or misleading texts, degrading retrieval performance @xcite . these query expansions can be seen as an evolution of earlier query expansions reliant on external lexical knowledge. in contrast, tradition prf that typically chooses additional texts from the top-retrieved documents, has received less attention. however, given that the expanded texts are sourced directly from the original documents, these methods hold significant potential for enhancing factuality. to this end, we propose corpus-steered query expansion (csqe). unlike methods that rely on the intrinsic parametric knowledge of llms, csqe exclusively leverages the strong relevance assessing capability of llms @xcite . as illustrated in figure 1 , given a query and its initially retrieved documents, csqe utilizes a llm to first identify relevant documents to the query and then extracts pivotal sentences that contribute to their relevance. these corpus-originated texts are then combined together with llm-knowledge empowered expansions to expand the original query. by incorporating query expansions that strictly originate from the corpus, csqe balances out the limitations commonly found in llm-knowledge empowered expansions. to sum up, our contributions are 3-fold: 1) we propose csqe, which exclusively exploits the relevance assessing capability of llms to overcome the hinderance posed by llm-knowledge empowered expansions. 2) experimental results reveal that csqe combined with a simple bm25 model, without necessitating any training, outperform not only llm-knowledge empowered expansion methods but also the sota supervised contriever ft model across two highresource web search datasets and six low-resource beir datasets. ## method in this section, we first describe how we implement a knowledge empowered query expansion baseline based on llms (keqe), then present the details of csqe to enhance bm25. keqe inspired by recent works that directly generate hypothetical documents to answer the query via llms for boosting retrieval @xcite @xcite , we implement a keqe baseline in a similar pattern for fair comparison. given a query q, we use llms to generate the hypothetical answer a via a task-agnostic prompt shown in it is worth noting that these hypothetical documents are inevitably susceptible to issues like hallucination that can adversely affect retrieval performance, due to the limitation of llms' intrinsic knowledge. to mitigate such problems, we propose csqe to incorporate corpus-originated expansions with knowledge embedded in the corpus.