Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers (2024)

Abstract: Query rewriting is a crucial technique for passage retrieval in open-domain conversational question answering (CQA). It decontexualizes conversational queries into self-contained questions suitable for off-the-shelf retrievers. Existing methods attempt to incorporate retriever’s preference during the training of rewriting models. However, these approaches typically rely on extensive annotations such as in-domain rewrites and/or relevant passage labels, limiting the models’ generalization and adaptation capabilities. In this paper, we introduce AdaQR (Adaptive Query Rewriting), a framework for training query rewriting models with limited rewrite annotations from seed datasets and completely no passage label. Our approach begins by fine-tuning compact large language models using only 10% of rewrite annotations from the seed dataset training split. The models are then utilized to self-sample rewrite candidates for each query instance, further eliminating the expense for human labeling or larger language model prompting often adopted in curating preference data. A novel approach is then proposed to assess retriever’s preference for these candidates with the probability of answers conditioned on the conversational query by marginalizing the Top- K passages. This serves as the reward for optimizing the rewriter further using Direct Preference Optimization (DPO), a process free of rewrite and retrieval annotations. Experimental results on four open-domain CQA datasets demonstrate that AdaQR not only enhances the in-domain capabilities of the rewriter with limited annotation requirement, but also adapts effectively to out-of-domain datasets.

## introduction passage retrieval in open-domain conversational question answering (cqa) have gained significant prominence in recent years @xcite . ## main results tables 1 and 2 show the main result of adaqr across 4 benchmarks and 2 retrievers, alongside the comparisons with baseline approaches. preference optimization brings improvement over sft. adaqr (ours) shows consistent and evident improvement over its sft-only counterpart across all the combinations of datasets and retrievers, indicating the effectiveness of preference optimization in further enhancing rewriters' abilities. adaqr improves in-and out-of-domain performance. for in-domain scenarios, on qrecc, our conversation answer is as effective as passage labels for preference optimization. both our approach and its variant with gold-label, are implemented in the same paradigm but with different types of reward (marginal probability of answer vs. ranking of gold passages). the two approaches have comparable performances, and even sometimes ours outperforms gold-label, demonstrating the effective role of the marginal probabilitybased reward for modeling the retrievers' preference, while such reward is more accessible. these makes adaqr a quite cost-effective method for adapting rewriters to various cqa tasks. adaqr has applicability to both sparse and dense retrievers. positive effect of our approach can be seen for both sparse (bm25) and dense (e5, ance) retrievers, which verifies the general applicability of our approach to various retrievers. on the other hand, note that the ance retriever has better performance than the bm25 for topi-ocqa, as exemplified by sft's higher retrieval metrics under ance over under bm25. we also observe that for topiocqa, the improvement of ours over sft under ance is greater than under bm25. therefore, it is reasonable to speculate that the benefit brought by our approach would get more pronounced with better retrievers. similar patterns also manifest in the cases of using gemma and llama2 as the base models (see @xcite topiocqa ( 2514 we further analyze two passage organization types (concatenation and marginalization) in appendix f. ## conclusion we introduce adaqr for enhancing query rewriters with minimal to zero in-domain labels, through the preference optimization towards retrievers' preferences. a novel feature of adaqr is in measuring retrievers' preferences with marginal probabilities of answers based on conversations and retrieved passages, enabling improvement or adaptation of query rewriters without labeled data. experiments show that adaqr brings significant improvement to query rewriters' in-domain performance and adapts them well to out-of-domain conversational question answering tasks. adaqr shows promise in establishing effective query rewriters for arbitrary conversational question answering tasks with minimal effort. innohk scheme.