Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modeling (2023)

Abstract: In this paper, we propose an unsupervised query enhanced approach for knowledge-intensive conversations, namely QKConv. There are three modules in QKConv: a query generator, an off-the-shelf knowledge selector, and a response generator. QKConv is optimized through joint training, which produces the response by exploring multiple candidate queries and leveraging corresponding selected knowledge. The joint training solely relies on the dialogue context and target response, getting exempt from extra query annotations or knowledge provenances. To evaluate the effectiveness of the proposed QKConv, we conduct experiments on three representative knowledge-intensive conversation datasets: conversational question-answering, task-oriented dialogue, and knowledge-grounded conversation. Experimental results reveal that QKConv performs better than all unsupervised methods across three datasets and achieves competitive performance compared to supervised methods.

## introduction in addition to open-domain chitchat, there exist various knowledge-intensive conversations, such as conversational question-answering, task-oriented dialogue, and knowledge-grounded conversation. although large-scale language models can implicitly store common knowledge within parameters @xcite , they are known to suffer from producing plausible statements with factual errors (a.k.a. knowledge hallucination) @xcite . therefore, there is a trend to rely on external resources, such as wikipedia databases or search engine results, to facilitate knowledge-intensive conversations @xcite . in knowledge-intensive conversations, the most straightforward way to retrieve external knowledge is to take the dialogue context as the query and use an off-the-shelf retriever to return the knowledge entry. however, it encounters some difficulties in retrieving appropriate knowledge @xcite . as the focus or topic changes along with the conversation flow, the outdated information in the dialogue context brings extra noise to the retriever, resulting in obsolete or irrelevant knowledge retrieved. moreover, the dialogue context has a native misalignment with the short and interrogative query preferred in existing retrievers. some methods choose to finetune a task-specific retriever to enhance the performance of knowledge selection @xcite @xcite . however, this strategy is usually computationally expensive (e.g., finetuning a dense retriever requires constant recomputation for massive knowledge entries) or even infeasible for complex retrieval systems (e.g., retraining a search engine is impractical). some other methods choose to generate a self-contained query based on the dialogue context @xcite @xcite . this strategy relies on careful query annotations to guarantee the completeness of essential information extraction and the adaptation to the knowledge selector. in this paper, we introduce a novel unsupervised query enhanced approach for knowledge-intensive conversations, namely qkconv. as shown in figure 1 , qkconv consists of three modules: a query generator, an off-the-shelf knowledge selector, and a response generator. specifically, qkconv is optimized through joint training, which produces the response by exploring multiple candidate queries and leveraging corresponding selected knowledge. we also integrate two types of query guidance to regulate query generation and facilitate joint training: context-sensitive guidance (e.g., the last context utterance) and response-sensitive guidance (e.g., the target response). the benefits brought by qkconv's design are three-fold. firstly, the training of qkconv solely relies on the dialogue context and target response, ## methodology this paper introduces a query enhanced approach of qkconv, which incorporates query generation to boost knowledge-intensive conversations and optimizes the dialogue system via unsupervised joint training. as shown in figure 1 , qkconv consists of three modules: query generator to generate multiple queries based on the dialogue context; an off-the-shelf knowledge selector to find relevant knowledge given queries; response generator to produce the final response. in the following, we will elaborate the design of these modules and discuss the process of joint training in detail. ## discussion in this section, to further dissect the proposed qkconv, more experiments are conducted on the qrecc dataset. unless specified, the pre-trained model of t5-large is employed in the following experiments. ## related work knowledge-intensive conversation to attend knowledge in conversations, some prior studies concentrate on how to ground the given knowledge @xcite or elicit parametric knowledge from large language models @xcite . recently, access to an external knowledge corpus has attracted a spate of interest, in line with our paper, and has come up with several datasets. for instance, some datasets provide a fixed small knowledge base for each sample @xcite @xcite . in more natural circumstances, using a uniform large-scale knowledge base for all samples, such as wikipedia dumps, web crawl data, or even search engines, has become a trend @xcite @xcite . however, it should be noted that knowledge selection challenges increase with the size of the knowledge base, and selection performance bounds the performance of response generation. therefore, the performance of knowledge selection is crucial for knowledge-intensive dialogue. two primary directions to address knowledge selection are finetuning knowledge selectors and generating a context-independent query. ## conclusion this paper introduces a query enhanced approach of qkconv for knowledge-intensive conversations, which is optimized via unsupervised joint training without any reliance on query annotations or knowledge provenances. the experiments are carried out on three knowledge-intensive conversation datasets: conversational question answering qrecc, taskoriented dialogue smd, and knowledge-grounded conversation wow. the proposed qkconv outperforms all unsupervised methods across three datasets. compared to supervised methods, qk-conv even establishes new state-of-the-art results on qrecc and wow. further analysis demonstrates that with joint training, the query generation adapts well to the knowledge selector, and the response generation has utilization robustness towards various knowledge.