Dialogue Response Generation Using Completion of Omitted Predicate Arguments Based on Zero Anaphora Resolution (2023)

Abstract: Human conversation attempts to build common ground consisting of shared beliefs, knowledge, and perceptions that form the premise for understanding utterances. Recent deep learning-based dialogue systems use human dialogue data to train a mapping from a dialogue history to responses, but common ground not directly expressed in words makes it difficult to generate coherent responses by learning statistical patterns alone. We propose Dialogue Completion using Zero Anaphora Resolution (DCZAR), a framework that explicitly completes omitted information in the dialogue history and generates responses from the completed dialogue history. In this study, we conducted automatic and human evaluations by applying several pretraining methods and datasets in Japanese in various combinations. Experimental results show that the DCZAR framework contributes to the generation of more coherent and engaging responses.

## introduction dialogue systems for natural language conversation, dialogue, and discourse with humans have attracted widespread attention in industry and academia. especially in recent years, the development of deep learning techniques and large dialogue corpus have made remarkable progress in dialogue response generation @xcite @xcite . however, the performance of the dialogue systems is still unsatisfactory, and many problems remain to be resolved. one problem is that dialogue systems cannot accurately interpret the intent of human utterances because the construction of common ground, which is important in human-to-human dialogue, has not yet been established @xcite . common ground in dialogue refers to shared beliefs, knowledge, and perceptions that form the premise for understanding utterances. for example, much information is omitted in the dialogue in why has the construction of common ground not been realized in human-to-system dialogues? sequence-to-sequence (seq2seq) models @xcite have been widely used in recent dialogue systems @xcite @xcite @xcite . seq2seq models use large amounts of dialogue data to train a mapping from a dialogue history to responses. however, there are many omissions in dialogue data, and it is difficult for models to generate responses that convey human intentions simply by training statistical patterns. to address this problem, several methods that use a knowledge base (kb) have been proposed. these models bridge the gap between humans and models by introducing external knowledge and providing the models with common-sense knowledge @xcite @xcite . human common-sense knowledge is one piece of information that can be omitted, but the cost of building a kb is significant and not easily transferable to different domains or models. in this study, we considered a method to provide models with omitted information without using external knowledge. dialogue systems can precisely interpret the intent of human utterances only when the roles of involved persons and things are understood, but omissions frequently occur in japanese dialogue to avoid repetition and references to selfevident objects @xcite . thus, the coherence of responses can be improved by inferring and explicitly incorporating the roles of persons and things. inspired by the idea of zero anaphora resolution (zar), we propose dialogue completion using zero anaphora resolution (dczar), a framework that explicitly completes omitted information in a dialogue history and generates responses from the completed history. the dczar framework consists of three models: a predicate-argument structure analysis (pas) model, a dialogue completion (dc) model, and a response generation (rg) model. the pas model analyzes the omitted arguments (zero pronouns) in the dialogue, and the dc model determines which arguments to complete and where to complete them and explicitly completes the omissions in the dialogue history. the rg model, trained by the complementary dialogue history and response pairs, generates a response. the pas and rg models are constructed by fine-tuning the common pretrained model with a dataset corresponding to each task, while the dc model uses a pretrained model without fine-tuning. we used the japanese wikipedia dataset and japanese postings ("tweets") to twitter to build our pretrained models. since tweets are like dialogues in that they contain many abbreviations and short sentences, the model pretrained with tweets is expected to improve the performance of zar and dialogue response generation. in this study, we performed automatic and human evaluations of three models built by pretraining models constructed by combining different methods and datasets. experimental results show that the dczar framework can be used to generate more coherent and engaging responses. analysis of the responses shows that the model generated responses that were highly relevant to the dialogue history in dialogues with many characters. the three main contributions of this work are as follows: ## approach: dczar framework we propose the dczar framework, which, as mentioned in section 1, consists of three models: pas, dc, and rg. figure 1 shows an overview of the proposed dczar framework. ## conclusion we proposed the dczar framework, which explicitly completes omitted information in the dialogue history and generates responses from the completed dialogue history. experimental results showed that the dczar framework can generate more coherent and engaging responses.