Generate, Evaluate, and Select: A Dialogue System with a Response Evaluator for Diversity-Aware Response Generation (2022)

Abstract: We aim to overcome the lack of diversity in responses of current dialogue systems and to develop a dialogue system that is engaging as a conversational partner. We propose a generator-evaluator model that evaluates multiple responses generated by a response generator and selects the best response by an evaluator. By generating multiple responses, we obtain diverse responses. We conduct human evaluations to compare the output of the proposed system with that of a baseline system. The results of the human evaluations showed that the proposed system’s responses were often judged to be better than the baseline system’s, and indicated the effectiveness of the proposed method.

## introduction dialogue systems based on deep neural networks (dnns) have been widely studied. although these dialogue systems can generate fluent responses, they often generate dull responses such as "yes, that's right" and lack engagingness as a conversation partner @xcite . to develop an engaging dialogue system, it is necessary to generate a variety of responses not to bore users. however, dialogue systems that are capable of generating diverse responses are difficult to automatically evaluate. a commonly used evaluation metric is bleu @xcite used in machine translation, which measures the degree of n-gram agreement with the reference response. however, due to the diversity of responses, i.e., the one-to-many nature of dialogue @xcite , which means the existence of multiple appropriate responses to an utterance, methods that compare the response to reference responses are not appropriate. therefore, there is a need for evaluation methods that do not use reference responses, and one of them is supervised evaluation. it trains dnns using human evaluations of responses generated by humans and models @xcite . dnn-based evaluations correlate to some extent with human evaluations. we aim to develop a dialogue system that is more engaging as a conversational partner by combining independently studied response generation and response evaluation models into a single dialogue system. specifically, we propose a generator-evaluator model in which multiple responses are generated by the generation model, evaluated by the evaluation model, and the response with the highest evaluation score is selected. by generating multiple responses, we can obtain diverse responses. this can be enabled by the response evaluator that does not require reference responses. our methods of generating multiple responses include a method with multiple decoding schemes and a method that uses a model that can generate responses with a specified dialogue act (da). generating responses by specifying various das leads to a variety of responses. to evaluate the proposed method, we conducted human evaluation by crowdsourcing to compare the outputs of the proposed system and a baseline system. the evaluation results show that the proposed system outputs better responses, and indicate the effectiveness of the proposed method. we target japanese dialogue systems and construct datasets of japanese dialogues. ## related work methods for evaluating responses by dialogue systems can be divided into human and automatic evaluations. automatic evaluation can be further classified into evaluation with or without reference responses. as an automatic evaluation metric, bleu @xcite is mainly used. it evaluates responses in terms of n-gram agreement with the reference sentence. however, it has been shown that there is no correlation at all between bleu and human evaluations @xcite . one reason for this is the one-to-many nature of dialogue @xcite , which means that there are multiple appropriate responses to an utterance. considering this nature, a method that measures the degree of n-gram agreement with the reference response is inappropriate for evaluating responses. therefore, automatic evaluation methods without any reference responses have been studied @xcite . they trained bert @xcite on a dataset of human evaluations to perform response evaluation that correlates with the human evaluations. da represents the role of an utterance in a dialogue. there are some datasets annotated with das such as swda @xcite and mrda @xcite . however, such datasets exist only for english, and we construct a da dataset in japanese. @xcite ; @xcite constructed a model that classifies a da for an utterance. @xcite proposed a model to generate responses with a specified da. this was achieved through adversarial learning. in this study, we use a more straightforward method to control responses. ## experiments we do the evaluation by crowdsourcing. workers are shown the outputs of the two systems and asked which of the system they would prefer to continue the conversation with. we ask a question to three workers and take a majority vote as the result. the test corpus consists of 2,000 sentences from the twitter-single dataset described in section 4.1 which are not used for training. ## analysis 6.1 out-of-domain evaluator in the experiments in section 5, each evaluator of de and da was trained using the human evalu ## conclusion we developed a dialogue system that can generate engaging responses by incorporating a response evaluator within the dialogue system. we proposed a generator-evaluator model, which consists of multiple response generation through multiple decoding schemes or specified das, responses evaluations, and the best response selection. human evaluation showed that responses generated by the generator-evaluator model are more engaging than those by the baseline systems. however, it is still necessary to improve the quality of responses generated with specified das in the future.