DIALKI : Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization (2021)

Abstract: Identifying relevant knowledge to be used in conversational systems that are grounded in long documents is critical to effective response generation. We introduce a knowledge identification model that leverages the document structure to provide dialogue-contextualized passage encodings and better locate knowledge relevant to the conversation. An auxiliary loss captures the history of dialogue-document connections. We demonstrate the effectiveness of our model on two document-grounded conversational datasets and provide analyses showing generalization to unseen documents and long dialogue contexts.

## introduction many conversational agent scenarios require knowledge-grounded response generation, where knowledge is represented in written documents. most prior work has explored architectures for knowledge grounding in an end-to-end framework, optimizing a loss function targeting response generation @xcite @xcite . however, the knowledge needed at any one turn in the dialogue is typically localized in the document, and some studies have shown that directly optimizing for knowledge extraction helps resolve complex user queries @xcite and increases user engagement in social chat @xcite . for long documents, explicit knowledge identification can also be useful for model interpretability and human-in-the-loop assistant scenarios. following @xcite , we define knowledge identification as the task of locating knowledge in a long document that is relevant to the current user query given the conversation context (figure 1 ). knowledge identification is similar to open question answering @xcite ## conclusion in summary, we introduce dialki to address knowledge identification in conversational systems with long grounding documents, taking advantage of document structure to contextualize document passages together with the dialogue history. di-alki uses a multi-task objective that identifies knowledge for the next turn and used knowledge for previous turns, which captures interconnections between the dialogue and the document. additional posterior regularization in learning further improves results. the model gives state of the art performance for this task on doc2dial and wizard of wikipedia, respectively. we show that improvements in knowledge selection transfer to response generation with a baseline generator. the current study is limited by the static nature of the available data. further work is needed to assess performance in an interactive setting. it would also be of interest to consider scenarios where document information is irrelevant or users change their minds about what information they want. the research was supported by the onr n00014-18-1-2826, darpa n66001-19-2-4031, nsf (iis 1616112), nsf career award, and allen distinguished investigator award. we also thank members of uw-nlp who provided feedback and insights to this work. we would also like to thank hao cheng for providing insights for improving model robustness and thank michael lee for early discussions.