MILL : Mutual Verification with Large Language Models for Zero-Shot Query Expansion (2024)

Abstract: Query expansion, pivotal in search engines, enhances the representation of user information needs with additional terms. While existing methods expand queries using retrieved or generated contextual documents, each approach has notable limitations. Retrieval-based methods often fail to accurately capture search intent, particularly with brief or ambiguous queries. Generation-based methods, utilizing large language models (LLMs), generally lack corpus-specific knowledge and entail high fine-tuning costs. To address these gaps, we propose a novel zero-shot query expansion framework utilizing LLMs for mutual verification. Specifically, we first design a query-query-document generation method, leveraging LLMs’ zero-shot reasoning ability to produce diverse sub-queries and corresponding documents. Then, a mutual verification process synergizes generated and retrieved documents for optimal expansion. Our proposed method is fully zero-shot, and extensive experiments on three public benchmark datasets are conducted to demonstrate its effectiveness over existing methods. Our code is available online at https://github.com/Applied-Machine-Learning-Lab/MILL to ease reproduction.

## introduction query expansion is a critical technique in search systems, aiming to effectively capture and represent users' information needs @xcite . search engines employ query expansion to resolve ambiguities in queries and align the vocabulary of queries and documents. central to this task is the development of contextual documents, comprising additional query terms, to enhance effectiveness @xcite . specifically, existing research predominantly falls into two categories: retrieval-based and generation-based methods. retrieval-based methods @xcite @xcite * *corresponding author 2022) typically construct contextual documents from the targeted corpus, assuming that the topretrieved documents (i.e., pseudo-relevance feedback (prf)) are reasonable expansions of a given query. generation-based methods @xcite @xcite often utilize advanced generative models, such as large language models, as an external knowledge base for producing contextual documents. however, both methods have clear limitations. for retrieval-based methods, it has been observed in practice that the documents retrieved with the original query do not align well with the information needs, particularly when the original query itself is brief and ambiguous @xcite . for generation-based methods, directly using off-the-shelf llms in a few-shot or zero-shot manner can hardly align the model with a specific corpus @xcite . in contrast, the llms could easily generate useless out-of-domain information. to this end, we propose a novel query expansion framework based on large language models (llms), integrating both retrieved and generated documents to mitigate their respective limitations. first, to improve contextual document generation, we design a query-query-document prompt that leverages an llm as a zero-shot reasoner to decompose a query into multiple sub-queries during contextual document generation. this helps the llm generate diverse contextual information that is more likely to cover the underlying search intent. next, we propose a mutual verification framework that exploits generated and retrieved contextual documents for query expansion. to be more specific, we propose to filter out the uninformative generated documents via comparing their relevance with the top-retrieved documents. by doing this, the selected generated documents are intuitively more aligned with the target corpus. conversely, we also filter out the noisy retrieved documents via comparing their relevance with the generated documents. the external contextual knowledge embedded in the generated documents can facilitate the retrieved documents to more accurately reveal search intent. we evaluate the proposed method on the downstream information retrieval task in a zeroshot manner. the results on three public datasets demonstrate that our proposed method significantly outperforms the state-of-the-art baselines. overall, the contributions can be summarized as follows: ## problem definition given a user query q, query expansion is to apply a function f to expand q with additional contextual information: q ′ = f θ (q), where θ represents the parameters. using the expanded query q ′ should be able to achieve better downstream retrieval performance compared to the original query q. more formally, such an objective can be defined as argmax θ m(q ′ , r), where q ′ = f θ (q). (1) ## methodology in this section, we introduce our proposed query expansion method in detail. specifically, we give an overview of mill in section 3.1, elaborate the query-query-document generation in section 3.2, and introduce the mutual verification framework in section 3.3. ## conclusion in this paper, we propose a novel zero-shot llmsbased framework for query expansion. first, we design a qqd prompt scheme that allows llms to generate diverse contextual documents via zeroshot reasoning. next, we introduce a mutual verification method that allows retrieved and generated contextual documents to complement each other as query expansion. the experimental results show that our method is superior to the state-of-the-art baselines on three public datasets.