CHIQ : Contextual History Enhancement for Improving Query Rewriting in Conversational Search (2024)

Abstract: In this paper, we study how open-source large language models (LLMs) can be effectively deployed for improving query rewriting in conversational search, especially for ambiguous queries. We introduce CHIQ, a two-step method that leverages the capabilities of LLMs to resolve ambiguities in the conversation history before query rewriting. This approach contrasts with prior studies that predominantly use closed-source LLMs to directly generate search queries from conversation history. We demonstrate on five well-established benchmarks that CHIQ leads to state-of-the-art results across most settings, showing highly competitive performances with systems leveraging closed-source LLMs. Our study provides a first step towards leveraging open-source LLMs in conversational search, as a competitive alternative to the prevailing reliance on commercial LLMs. Data, models, and source code will be publicly available upon acceptance at https://github.com/fengranMark/CHIQ.

## introduction conversational search enables users to interact with the system in a multi-turn fashion to satisfy their complex information needs @xcite . one of the crucial steps is to compose adequate search queries for each contextdependent utterance. recent advancements in the task-solving capabilities of large language models (llms) @xcite @xcite have motivated researchers to integrate these models into existing conversational search systems. most recent studies @xcite leverage llms to directly generate search queries based on the context of the conversation history. although seemingly straightforward, this technique is shown to achieve higher effectiveness in query rewriting than fine-tuning a smaller language model, such as t5 @xcite . however, these performance gains are primarily achieved through the use of commercial, closed-source llms @xcite . this is primarily because closed-source llms can better perform complex reasoning tasks @xcite compared to open-source models. one of the main challenges in conversational search resides in the ambiguous nature of the conversation history. figure 1 illustrates an example where solving co-reference relation in u 4 and elaborating the response in r 3 can help generate an adequate search query. intuitively, performing these tasks requires basic nlp task-solving capabilities, which even small-scale open-source llms (e.g., 7b) possess @xcite . the key challenge is to unlock the full capabilities of open-source llms for conversational search. this requires carefully preparing the conversation history to enhance its quality, rather than directly using it to generate the search query. in this paper, we propose chiq, a method that aims to enhance the quality of contextual history for improving query rewriting. as illustrated in figure 1 , we leverage the nlp capabilities of llms (e.g. solving coreference relation or expanding the context) to make the conversational history less ambiguous, consequently enhancing the relevance of the generated search query. we investigate various methods for integrating refined conversational history into existing frameworks, including ad-hoc query rewriting, generating pseudo supervision signals for fine-tuning query rewriting models, and the fusion of both approaches. we conduct extensive experiments using the open-source llm, llama-2-7b @xcite , across five well-established conversational search benchmarks under both dense and sparse retrieval settings. the experimental results indicate that enhancing the conversational history using our method achieves state-of-the-art performance ## related work different from traditional ad-hoc retrieval, which assumes users submit a stand-alone query, conversational search provides a conversational interface so that users can elaborate more complex search requirements, and interactively perform search. the main challenge lies in accurately understanding the user's real search intent, which may be embedded within a longer, noisy, and more complex conversational context history. there are two wellestablished approaches in the literature for conversational search: conversational dense retrieval (cdr) @xcite @xcite and conversational query rewriting (cqr) @xcite . the cdr systems aim to fine-tune an end-toend conversational dense retriever that can directly model the entire conversational history to return relevant documents @xcite . conversely, the cqr systems focus on formulating an adequate search query based on the conversational history. this query can then serve as the input to an existing, well-established retrieverranker framework. we base our solution on cqr, leveraging its ability to integrate with existing adhoc search models, which has demonstrated significant practical value @xcite . earlier approaches to cqr attempted to select useful tokens from the conversation context (kumar and callan, 2020; @xcite or to train generative rewriter models with conversational sessions to mimic the humanrewritten query @xcite @xcite . to optimize query rewrit-ing, some studies have adopted reinforcement learning @xcite , or used the ranking signals with the rewriting model training @xcite @xcite . in addition, there have been endeavors to improve the conversion history quality through context denoising @xcite @xcite @xcite , and data augmentation @xcite @xcite . unlike them, we enhance query rewriting by leveraging the nlp capabilities of open-source llms to reduce the ambiguity of conversational history. there have been multiple endeavors to integrate llms to solve traditional ad-hoc search sub-tasks @xcite , such as query expansion @xcite , dense retrieval @xcite , and re-ranking @xcite . about conversational search, @xcite al. (2023), jang et al. (2023), and chen et al. (2024a) attempt to improve cdr with unsupervised fine-tuning. mao et al. (2023a), mo et al. (2024d), and ye et al. (2023) explore how llms can understand users' contextualized search intents via cqr. unlike direct rewriting the query using llms, we investigate various approaches for integrating refined conversational history into cqr frameworks. ## conclusion in this paper, we propose chiq, an approach that leverages the basic nlp capabilities of llms to enhance the quality of contextual history for improving the query rewriting performance in terms of conversational search. despite its simplicity, our approach achieves superior performance across various datasets and settings, using open-source llms compared to closed-source alternatives. this study shows that instead of simply ask an llm to generate a search query, it is critical to design strategies to generate different facets of enhancement in view of finding the target information.