TAKE : Topic-shift Aware Knowledge s E lection for Dialogue Generation (2022)

Abstract: Knowledge-grounded dialogue generation consists of two subtasks: knowledge selection and response generation. The knowledge selector generally constructs a query based on the dialogue context and selects the most appropriate knowledge to help response generation. Recent work finds that realizing who (the user or the agent) holds the initiative and utilizing the role-initiative information to instruct the query construction can help select knowledge. It depends on whether the knowledge connection between two adjacent rounds is smooth to assign the role. However, whereby the user takes the initiative only when there is a strong semantic transition between two rounds, probably leading to initiative misjudgment. Therefore, it is necessary to seek a more sensitive reason beyond the initiative role for knowledge selection. To address the above problem, we propose a Topic-shift Aware Knowledge sElector(TAKE). Specifically, we first annotate the topic shift and topic inheritance labels in multi-round dialogues with distant supervision. Then, we alleviate the noise problem in pseudo labels through curriculum learning and knowledge distillation. Extensive experiments on WoW show that TAKE performs better than strong baselines.

## introduction due to the dull response generation problem in the general open-domain dialogue generation technology, an increasing number of researchers focus on knowledge-grounded dialogue generation (kgdg) @xcite @xcite . by connecting the external knowledge base with the generation model as supplement information, the generated response becomes more engaging and informative. 4 ) exhibit a sub-topic shift: from dog's nickname to dog's color; the utterances (4) to ( 6 ) exhibit an obvious topic shift: from dog to snake; the utterances (6) to ( 8 ) exhibit a topic inheritance. knowledge selection plays a vital role in kgdg task @xcite . since one can choose any reasonable knowledge to carry on the conversation, one-to-many relations exist between dialogue context and knowledge @xcite . thus, selecting the most appropriate knowledge in the vast knowledge pool becomes a significant challenge. and most of the existing methods pay close attention to the design of the knowledge selector. some methods attempt to improve the accuracy of knowledge selection by discovering more features in the dialogue context @xcite or introducing extra posterior knowledge @xcite @xcite . these methods often directly take the whole dialogue context as the input of the knowledge selector, ignoring that different parts of the context play different roles. considering this, @xcite propose to decouple the knowledge selector according to the different part of the input and only keep part of the history information by introducing mixed-initiative (userinitiative and agent-initiative) characteristics. in their method, the user takes the initiative when the knowledge connection between two adjacent rounds is unsmooth. such a judgement only works when there is a strong semantic transition between two rounds. however, the conversational direction can be changed by the user when the user shifts the topic from one to another relevant one. for example, the utterance flow (2) → (3) → (4) in to address the above problem, we bring the topic into multi-turn knowledge selection. through our observation, we find that the topic shift and topic inheritance affect knowledge selection deeply. for the topic shift, it is generally caused by the active user's frequent questioning and the model should select knowledge according to the current user utterance. for the topic inheritance, it is mainly caused by relatively passive users who agree with what the agent says, and the model needs to find some relative topics to continue the conversation according to previously selected knowledge. we obtain the topic shift label via distant supervision @xcite , where we regard the retrieving entity as the topic word. considering that there may exist noises in the pseudo labels, we further alleviate their negative effects through curriculum learning and knowledge distillation (see section 2.6 for details). our contributions in this paper are as follows: ## conclusion in this paper, we propose a topic-shift aware knowledge selector(take) model, which better locates the relevant parts from dialogue history to improve the performance of knowledge selection. besides, we obtain topic shift labels inspired by the idea of distant supervision and adopt curriculum learning and knowledge distillation methods to alleviate the negative influence of noises. experiments on the wow show that our model outperforms the baselines, and the ablation study indicates that all components of our methods work. in the future, we will research on multi-sentence knowledge integration further and combine the labeling work with knowledge graph.