Explicit Use of Topicality in Dialogue Response Generation (2022)

Abstract: The current chat dialogue systems implicitly consider the topic given the context, but not explicitly. As a result, these systems often generate inconsistent responses with the topic of the moment. In this study, we propose a dialogue system that responds appropriately following the topic by selecting the entity with the highest “topicality.” In topicality estimation, the model is trained through self-supervised learning that regards entities that appear in both context and response as the topic entities. In response generation, the model is trained to generate topic-relevant responses based on the estimated topicality. Experimental results show that our proposed system can follow the topic more than the existing dialogue system that considers only the context.

## introduction in recent years, end-to-end chat dialogue systems have been developed remarkably, making it possible to generate rich and flexible responses. however, such current chat dialogue systems only implicitly consider the topic given the context as it is, but do not explicitly consider it @xcite . as a result, these systems often generate inconsistent responses with the topic @xcite . in this study, we propose a dialogue system that selects the next dialogue topic and responds using the selected topic explicitly. our system selects the next topic entity based on topicality @xcite . here, we define the entity as a noun or compound nouns, and topicality as the degree of speaker awareness directed toward each entity in the dialogue context. in addition, we call the entity with the highest topicality in the context topic entity. in the response generation part, our system generates responses based on the estimated topic entity as well as the context. we propose the two-stage model, which learns topicality estimation and response generation in two stages, and the end-to-end model, which learns in the end-to-end method. due to the lack of dialogue corpus with the topic annotated, we use a self-supervised learning method to train our proposed models. specifically, we extract triples @xmath0, response, labeled topic entity @xmath1. for labeling, we regard topic entity candidates (= entities in the context) in the response as topic entities and assign labels to them. this procedure assumes that the entity in the response can be considered the topic entity. furthermore, zero anaphora resolution is applied to restore those omitted words when assigning labels because word omission is pervasive in actual dialogue (especially in japanese). the automatic and human evaluation results show that our proposed system can follow the topic more than the existing dialogue system that considers only the context. ## dataset construction we construct the dataset for self-supervised training of the proposed model. the dataset is constructed based on the assumption that entities that appear in both context and response are the topic entities. considering the pervasiveness of word omission in actual dialogue, the omitted words are restored by zero anaphora resolution. ## model in this section, we describe the two-stage model, in which topicality estimation and response generation are learned in two stages (section 4.1), and the end-to-end model, in which they are learned in the end-to-end method (section 4.2). ## conclusion we proposed dialogue systems that explicitly consider topicality to generate responses following the topic. both automatic and human evaluation results confirmed that the proposed two-stage model could generate more topic-following responses than the dialogue system that only considers context. in addition, by restoring the word omission in the response by zero anaphora resolution, topicality estimation was further improved, and it was also confirmed that the generated responses can better capture the topic. on the other hand, the naturalness score of the generated responses tends to be low overall in the human evaluation, and there is still room for improvement. we will work to improve the quality of the response generation part by using some pretraining models as future work.