Analyze, Generate and Refine: Query Expansion with LLM s for Zero-Shot Open-Domain QA (2024)

Abstract: Query expansion (QE) is a critical component in the open-domain question answering (OpenQA) pipeline, enhancing the retrieval performance by broadening the scope of queries with additional relevant texts. However, existing methods like GAR and EAR rely heavily on supervised training and often struggle to maintain effectiveness across domains and datasets. Meanwhile, although large language models (LLMs) have demonstrated QE capability for information retrieval (IR) tasks, their application in OpenQA is hindered by the inadequate analysis of query’s informational needs and the lack of quality control for generated QEs, failing to meet the unique requirements of OpenQA. To bridge this gap, we propose a novel LLM-based QE approach named AGR for the OpenQA task, leveraging a three-step prompting strategy. AGR begins with an analysis of the query, followed by the generation of answer-oriented expansions, and culminates with a refinement process for better query formulation. Extensive experiments on four OpenQA datasets reveal that AGR not only rivals in-domain supervised methods in retrieval accuracy, but also outperforms state-of-the-art baselines in out-domain zero-shot scenarios. Moreover, it exhibits enhanced performance in end-to-end QA evaluations, underscoring the superiority of AGR for OpenQA.

## introduction open-domain question answering (openqa) is a key task in natural language processing, aiming to provide accurate answers to a wide range of factual questions across different domains @xcite . the challenge in openqa is to retrieve relevant information from large text corpora without specific contexts @xcite . retrieval methods are therefore es1 our code and data are publicly available at https:// github.com/process-cxr/agr . in 1771, he wrote ""the army in general and the army in particular "" (1771), in which he advocated the use of high-heated engines hundredhorsepower per hour (hp) for the war, as a mean to bring about the ... what does hp mean in war and order ## methodology as shown in figure 2 , the proposed agr leverages the concept of multi-step task decomposition, encompassing a progressive sub-task workflow to analyze, generate, and refine the query. this approach thoroughly exploits the abilities of llms @xcite , like semantic comprehension, knowledge retrieval, and integration, as well as contextual understanding and reasoning. the result is an expanded query that is anticipated to be more aligned with the answer, thereby improving the performance of sparse retrieval mechanisms within the openqa task. ## conclusion in this paper, we introduce a multi-step generative framework agr with analyze, expand and refine phases, which deeply mines the inherent knowledge of llms for answer-oriented query expansion. agr eschews the need for additional in-domain data for supervised training and demonstrates significantly higher retrieval accuracy compared to state-of-the-art qe methods under out-of-domain zero-shot scenes. moreover, when integrated with the fusion-in-decoder as the reader component, agr achieves a nearly comprehensive lead in endto-end performance, showcasing its effectiveness on openqa task. in future work, we plan to explore a more comprehensive framework, further integrating the requirements of the reader component to achieve a unified architecture and enhance the end-to-end performance for openqa task.