Learning When to Retrieve, What to Rewrite, and How to Respond in Conversational QA (2024)

Abstract: Augmenting Large Language Models (LLMs) with information retrieval capabilities (i.e., Retrieval-Augmented Generation (RAG)) has proven beneficial for knowledge-intensive tasks. However, understanding users’ contextual search intent when generating responses is an understudied topic for conversational question answering (QA). This conversational extension leads to additional concerns when compared to single-turn QA as it is more challenging for systems to comprehend conversational context and manage retrieved passages over multiple turns. In this work, we propose a method for enabling LLMs to decide when to retrieve in RAG settings given a conversational context. When retrieval is deemed necessary, the LLM then rewrites the conversation for passage retrieval and judges the relevance of returned passages before response generation. Operationally, we build on the single-turn SELF-RAG framework (Asai et al., 2023) and propose SELF-multi-RAG for conversational settings. SELF-multi-RAG demonstrates improved capabilities over single-turn variants with respect to retrieving relevant passages (by using summarized conversational context) and assessing the quality of generated responses. Experiments on three conversational QA datasets validate the enhanced response generation capabilities of SELF-multi-RAG with improvements of ~13% measured by human annotation.

## introduction recent advances in llm technology has made the conversational search paradigm @xcite increasingly viable for mainstream use as next generation search technology. unlike traditional search engines which primarily process keyword queries, users can treat conversational search systems as a knowledgeable expert and directly barrett left in april 1968 due to deteriorating mental health yes no pink floyd were founded in 1965 by syd barrett (guitar, lead vocals), nick mason (drums), roger waters (bass guitar, vocals) and richard wright @xcite . under barrett's leadership, they released two hit singles, "arnold layne" and "see emily play", and the successful debut album the piper at the gates of @xcite . the guitarist and vocalist david gilmour joined in december 1967; barrett left in april 1968 due to deteriorating mental health. retrieval? pink floyd were founded in 1965 and the founding members were: syd @xcite who among them left the band first? retrieval? figure 1 : understanding conversational context. in multi-turn conversations user questions often refer to responses in previous turns based on passages already retrieved, as shown in the example above. to answer the follow-up question, it is not necessary to retrieve new passages and the llm should refer back to the previously retrieved passage, which contains the response. thus, understanding if retrieval is necessary q: who was martina hingis? a: martina hingis is a swiss former professional tennis player. q: how many career games did she play? a: she had played over 1,300 career singles matches and over 700 career doubles matches. q: when did she retire? a: she retired in november 2007 after being hampered by a hip injury for several months. q: was martina hingis injured often? a: martina hingis had a right ankle surgery in october of 2001, and a left ankle surgery in may of 2002. after that she struggled with injuries. q: did she come back from retirement? gpt4 summary: martina hingis is a swiss former professional tennis player who had to retire due to a hip injury in 2007. she has also struggled with injuries previously, undergoing surgeries on both her ankles. did martina hingis make a return from retirement? human rewrite: did martina hingis come back from her 2007 retirement? t5 rewrite: did martina hingis come back to play tennis again after retirement? figure 2 : summarizing conversational context. while using the entire conversational context as query to a retrieval model might introduce noise, using traditional rewriting methods might miss on important aspects of the conversations. conversation summaries provide an adaptable approach as a retrieval query. for high-quality response generation is an important research question, especially in the context of conversational qa. in multi-turn question answering, comprehending users' contextual intent and generating responses pose significant challenges due to complexities introduced by the extended context window containing previous user interactions @xcite @xcite . the system, when deciding whether to retrieve or estimating the usefulness of its own response, must process a longer context of the conversation history, understand the user intent of current turn, ensure prevention of information repetition, maintain user engagement, etc. an example is provided in figure 1 , which shows that the decision to retrieve or not might depend on the conversational context rather than last turn only. furthermore, when retrieval is expensive or noisy, it is beneficial to utilize already retrieved documents in the conversation, given they are relevant and contain the necessary answers. additionally, detrimental context as a result of noisy retrieval can degrade response generation quality @xcite . lastly, when conversation history is longer, traditional conversational query rewriting methods @xcite @xcite that typically emphasize co-reference resolution, might be insufficient to contain all the information required for an effective retrieval @xcite . as shown in figure 2 , both the gold and a t5-based query rewriting miss potentially important signals (e.g., hip injury) for retrieving correct passages. in that case, representing the conversation history in a summarized form can lead to more effective retrieval. in this work, we propose self-multi-rag, an ap-proach for efficient retrieval during a multi-turn conversation for improved response generation. self-multi-rag provides refined contextual signals for retrieval and enhances multi-turn answer generation by potentially reusing already retrieved passages in previous turns. in particular, self-multi-rag enables: (i) better understanding of the conversational context to decide whether retrieval is needed and to generate useful responses accordingly in a conversational setting and (ii) summarizing the conversational context such that it can be used as a query to retrieve relevant documents, when needed, with high retrieval effectiveness. our specific contribution are as follows: ## self-multi-rag in this work, we propose self-multi-rag, which extends self-rag @xcite to generate responses in a conversational setting. our proposed methodology trains a llm to comprehend longer conversational contexts to learn when retrieval is needed and also critiquing the quality of the passages and its own generation given the previous turns by generating special tokens. importantly, when the llm decides retrieval is needed to generate a response, we train it to summarize the conversational context for more effective passage retrieval, which consequently leads to better response generation. self-multi-rag has two important functions: understanding conversational context. in a multi-turn context, evaluating the relevance of a retrieved passage involves considering both the current question and the previous conversation history, unlike a single-turn question. furthermore, retrieved passages from previous turns can also be provided in the conversational context or a user may simply provide a passage and ask questions based on that. when the cost of retrieval is high, a model should not decide to retrieve if the question in the current turn can be answered from passages retrieved in previous turns (as shown in figure 1 ). in that case, the model should not only comprehend the conversational context but also the previously retrieved passages to be able to determine the conversation history previously retrieved passages retrieval? retrieval component corpus yes retrieved passages query retriever generate responses no utility? final response relevance? generate intermediate responses groundedness? summarise conversation reflection tokens obtained from critic additional components for multi-turn domain necessity of retrieval. utilizing already retrieved passages (given they are relevant) not only mitigates the harmful effects of noisy retrieval @xcite , but also saves costs by reducing the number of context tokens @xcite . ## experimental setup we evaluate self-multi-rag on three benchmarks: qrecc @xcite , ultrachat @xcite and mt-eval @xcite . qrecc contains conversational questions answers to which can be found within a collection of 10m web pages. answers to questions in the same conversation may be distributed across several web pages. qrecc provides gold passage annotations which indicates the passage from where a question in a conversation can be answered from. ultrachat is traditionally used as supervised fine tuning (sft) data for llms. it contains diverse and informative instructional conversations and covers a wide range of topics and instructions. while conversations in qrecc are knowledge-grounded and ideally rag should be beneficial for every turn of the conversation, it is not so for ultrachat. mt-eval is similar to ultrachat as it also contains diverse instructional conversations. it forms our out-of-domain evaluation benchmark as we use samples freom qrecc and ultrachat for training self-multi-rag. additional details on the benchmarks are provided in appendix b. ## conclusion in this work, we propose self-multi-rag, a framework to train llms to learn when to retrieve and generate response for better conversational qa. we perform extensive evaluation on three conversational qa benchmarks and demonstrated improved performance over previous approaches. this is achieved by overcoming the previous limitations to accurately critic when to retrieve or whether the retrieved documents are relevant or the usefulness of generated response given a multi-turn dialogue. self-multi-rag is better at comprehending the longer contexts of multi-turn conversations resulting in better critic and consequently generator performances. finally, we observe that summaries of conversational history generated by self-multi-rag, increase retrieval effectiveness when used as query to retrieve passages and consequently leads to improved response generation. as future work, we would like to (i) consider longer and multithreaded conversations, (ii) including diversity as a metric while considering passages for response generation.