D i SC o L : Toward Engaging Dialogue Systems through Conversational Line Guided Response Generation (2021)

Abstract: Having engaging and informative conversations with users is the utmost goal for open-domain conversational systems. Recent advances in transformer-based language models and their applications to dialogue systems have succeeded to generate fluent and human-like responses. However, they still lack control over the generation process towards producing contentful responses and achieving engaging conversations. To achieve this goal, we present DiSCoL ( Di alogue S ystems through Co versational L ine guided response generation). DiSCoL is an open-domain dialogue system that leverages conversational lines (briefly convlines ) as controllable and informative content-planning elements to guide the generation model produce engaging and informative responses. Two primary modules in DiSCoL’s pipeline are conditional generators trained for 1) predicting relevant and informative convlines for dialogue contexts and 2) generating high-quality responses conditioned on the predicted convlines. Users can also change the returned convlines to control the direction of the conversations towards topics that are more interesting for them. Through automatic and human evaluations, we demonstrate the efficiency of the convlines in producing engaging conversations.

## introduction over the past decade, users have actively engaged with dialogue systems to fulfill a wide range of requirements. task-oriented dialogue systems have assisted users in accomplishing specific tasks such as finding apartments @xcite and restaurants @xcite or even booking movie tickets @xcite . while, open-domain dialogue systems have been extensively leveraged for psychotherapy counseling, entertainment, and even teaching foreign languages to users @xcite @xcite figure 1: a dialogue context and its three responses generated based on dialogpt and our proposed dis-col system using originally inferred and manipulated convlines, respectively. discol leverages convlines (depicted in colored boxes) to guide the generation model to encapsulate those informative contents. our demo enables the user to edit or remove the inferred convlines (shown in blue for edits and red for removal) to guide the conversation towards its desired directions. @xcite . in this work, we focus on the second group. in the context of open-domain dialogue systems, neural-network-based generative models have outperformed retrieval-based systems by generating diverse and novel responses. more recently, largescale language models with transformer-based architectures, such as gpt-2 @xcite and bart @xcite , have advanced the state of the art in natural language generation and dialogue systems. such models can be further enhanced by fine-tuning them on task-specific data, as it is the case of dialogpt (dialogue generative pre-trained transformer) @xcite , a neural conversational response generation model, trained on 147m conversation-like exchanges extracted from reddit. although responses generated by such models are fluent and locally coherent, they usually suffer from content poverty (e.g., generating non-informative content), which can negatively impact user engagement. furthermore, these models do not allow the users to exert control on the generation process and guide the conversation toto alleviate this issue, here we propose discol, an open-domain dialogue system, which leverages convlines as primary elements to add control for generating informative and content-rich responses. convlines are abstract representations of utterances in the dialogues that can be used as content planning elements to form high-level content of an utterance and guide the generator to incorporate these informative units in the generation (see colored boxes in figure 1 ). content planning has been shown to be beneficial in the story generation task. these abstract representations known as storylines or story plots have been successful to guide the language models produce more coherent and fluent stories @xcite @xcite @xcite . discol is composed of four main neuralnetwork-based modules (see figure 3 ). the first two modules are designed to extract entities and topics of the dialogue context. the third module is a fine-tuned conditional generator that learns to take the dialogue context and previously extracted information and predict convlines that would be leveraged in the response generator module. similar to convline generator, response generator is a conditional auto-regressive language model that generates response conditioned on the dialogue context and its convlines, entities, and topics ex-tracted from previous modules. the middle block of figure 1 exhibits the generated response for the inferred convlines shown in green boxes. in the interactive setting of our devised demo from which a snapshot is shown in figure 2 , we provide the users with the facility to manipulate the predicted convlines to direct the conversation toward its topics of interest. the last block in figure 1 depicts the removed and edited convlines (red and blue boxes) that led the generator to generate a slightly different response by taking into account the applied adjustments. we validate discol on the topical chat dataset @xcite using both human and automatic evaluations. our results demonstrate the superiority of discol over dialogpt in terms of generating higher quality responses, thus indicating the usefulness of convlines as dialogue control mechanisms for generating more engaging responses. we release the source code and trained models to facilitate the future dialogue research. foot_0 ## system architecture the architecture of our proposed discol demo system and its modules are depicted in figure 3 . a user converses with the system by writing an utterance as an input. this utterance passes through all the modules and in each module some new information such as its extracted entities, topics, and convlines are augmented. the last module, response generator, incorporates all this information to generate a response as the output of the system. in this section, we explain each module in detail. ## system implementation we test our system on topical-chat dataset @xcite that includes knowledge-grounded human-human conversations covering a set of 8 different topics. this dataset has been collected by employing amazon mechanical turk (amt) workers who have been provided with specific entities and some external knowledge (wikipedia lead sections, washington post articles, or some reddit fun facts) to chat about. therefore, each utterance in the conversation is either based on provided knowledge sources or the user's personal knowledge. overall, 261 popular entities spanning 8 various topics (fashion, sports, books, politics, general entertainment, music, science & technology and movies) have been selected for the dataset collection. we add general topic for utterances (e.g. greetings) that do not include any specific contents such as "hi, how are you today?". ## experimental results we evaluate the performance of discol system against dialogpt, which is one of the strongest recent baselines that has shown its efficiency in generating consistent and relevant responses. ## conclusion we have introduced discol, an open domain dialogue system that leverages convline as an intermediate step toward generating more informative and controllable responses in dialogues. the convlines are predicted and subsequently leveraged in the response generation process. additionally, dis-col allows users to manipulate convlines towards their favorite conversational direction. our findings show that in contrast to other transformer-based dialogue systems that do not incorporate content planning, discol takes the advantage of such a principled structure to generate better and more engaging conversations with users. in the future, we imagine an open path of possible research in the controllable conversations which would guide the dialogue toward having pleasant features such as empathy and bias-free or even personalized convlines to generate dialogues with such aspects. it is also expecting to train dialogue models to converse by following specific styles such as generating formal conversations by predicting more formal convlines.