An Empirical Study of Multitask Learning to Improve Open Domain Dialogue Systems (2023)

Abstract: Autoregressive models used to generate responses in open-domain dialogue systems often struggle to take long-term context into account and to maintain consistency over a dialogue. Previous research in open-domain dialogue generation has shown that the use of auxiliary tasks can introduce inductive biases that encourage the model to improve these qualities. However, most previous research has focused on encoder-only or encoder/decoder models, while the use of auxiliary tasks in encoder-only autoregressive models is under-explored. This paper describes an investigation where four different auxiliary tasks are added to small and medium-sized GPT-2 models fine-tuned on the PersonaChat and DailyDialog datasets. The results show that the introduction of the new auxiliary tasks leads to small but consistent improvement in evaluations of the investigated models.

## introduction in recent years, open-domain dialogue systems have experienced increased research due to the availability of large corpora of dialogue and advances in deep learning techniques @xcite . unlike task-oriented dialogue systems designed for specific domains or tasks, such as flight booking, hotel reservation, customer service, and technical support @xcite @xcite , open-domain dialogue systems aim to have longterm connections with users by satisfying their emotional, social, and communication needs. therefore, such a system must comprehend the dialogue context and user demands in order to select the appropriate skill at the appropriate time and generate consistent @xcite and grounded @xcite interpersonal responses. open-domain dialogue systems can include single-turn or multi-turn dialogues, where the context and topic of the conversation may change throughout the interaction @xcite . multi-turn open-domain dialogue systems need to maintain the context of the conversation, generate appropriate responses concerning the context and predefined characteristics (persona), and handle various forms of input. a persona is a set of characteristics or attributes that describe a virtual agent's background, personality, and behavior. these attributes include the agent's name, age, gender, profession, interests, and other aspects @xcite , while a conversation context refers to the background information or previous interactions relevant to the current conversation, with word-level and utterance-level dependencies @xcite . recent developments in transformer-based architectures @xcite @xcite for large-scale pre-training, such as openai's gpt-2 @xcite , have shown exceptional results. later, the models are fine-tuned via more technical steps and at different scales on a large-scale dialogue corpus @xcite @xcite . pre-trained models on conversational datasets typically process dialogue context (a list of utterances) as a sequence of tokens per utterance to generate responses. although these approaches show effective results compared to previous approaches, they still need to catch the latent information in more complex structures rather than just tokens @xcite . a conversational domain is distinguished by the presence of another component called utterances 1 that plays an imperative role in conveying higher-level information in addition to tokens and their local relationships. recent research has put forth the use of auxiliary tasks as a form of regularization during the fine-tuning of models as a means to address the aforementioned issue. a majority of these additional training objectives are implemented solely on the encoder-only and encoder-decoder architectures. however, while the use of auxiliary tasks has led to improvement in encoder-only and encoder/decoder models, recent work has not explored the application of auxiliary tasks in decoderonly models. in this research, we propose incorporating auxiliary tasks on top of an autoregressive decoder-only model to examine and enhance the quality of generated responses concerning the latent information present within utterances. additionally, we demonstrated the impact of various auxiliary tasks on distinct elements of dialogue across two benchmark datasets. by examining the effect of different auxiliary tasks on various components of dialogue, we aimed to provide a deeper understanding of how these tasks can influence the performance and outcomes of conversational systems. additionally associated code to this research can be found in our github repository. foot_1 ## related works the motivation for this research is drawn from recent investigations into the utilization of auxiliary tasks to enhance the generated responses in opendomain dialogue systems by considering the context. to this end, we present and analyze these recent studies in this section. previous studies in this field can be broadly classified into three general categories. the first category pertains to the widespread use of encoder-decoder models in dialogue response generation, which have been observed to produce generic and uninteresting responses (e.g., "i'm good", "i don't know"). @xcite proposed an encoder-decoder architecture with two auxiliary tasks at token and utterance levels that can effectively exploit conversation context to generate responses, including order recovery and masked context recovery. analogously, @xcite examined a range of unsupervised pre-training objectives for acquiring dialogue context representations via encoder-decoder models by incorporating four auxiliary tasks, including next-utterance retrieval, next-utterance generation, masked-utterance retrieval, and inconsistency identification. dialogbert is a unique design that employs a hierarchical transformer architecture to comprehensively capture the context of dialogue @xcite . using two training objectives, similar to bert @xcite , allows the model to understand a conversation's nuances effectively. in the first objective, masked context regression, the model is trained to predict the missing context from a dialogue, and in the second objective, distributed utterance order prediction, the model is trained to predict the order of spoken utterances in a conversation so that it understands the flow and logic. lastly, decoder-only models, like dialogpt @xcite , make use of only the final component of the encoder-decoder structure. di-alogpt in particular, extends the gpt-2 @xcite architecture by being specifically developed and trained on a large corpus of dialogue data to generate responses in a conversational context. however, despite its ability to perform well in single-turn conversation, its lack of capability to capture latent information behind utterances in a multi-turn conversation, results in an inadequate understanding of the context. the utilization of auxiliary tasks in decoder-only models is a wellestablished practice. for instance, the gpt-2 based model transfertransfo @xcite , which adopts a multi-task objective, showed improvement over the basic gpt-2. these auxiliary tasks primarily take the form of sequence classification tasks. ## dataset and experiments the following section provides detail on the dataset and experimental settings used in our experiments. lm @xmath0. <@xmath1. <@xmath2. <@xmath3. <@xmath4, how are you doing today ? <@xmath5. <@xmath6, four sisters . just watching game of thrones . <@xmath7. <@xmath8. <@xmath9. <@xmath10. <@xmath11. <@xmath12. <@xmath13,<@xmath14. <@xmath15, four sisters . just watching game of thrones . <@xmath16. <@xmath17. <@xmath18. <@xmath19. <@xmath20. <@xmath21, how @xmath22. <@xmath23, four sisters . just watching footbal of thrones . <@xmath24. <@xmath25. <@xmath26. <@xmath27. <@xmath28. <@xmath29, how are we doing today ? <@xmath30. <@xmath31, four sisters . just observing game of thrones . <@xmath32. <@xmath33 ## results furthermore, compared to other auxiliary tasks, the use of utterance masking in the binary form leads to improvements in metrics such as bleu, rouge-l, moverscore, and extrema. the results also suggest that using auxiliary tasks in larger models improves performance. the comparison between the persona-chat dataset highlights the significance of using auxiliary tasks simultaneously in both the persona and context components, which results in even better performance. tables 3 and 4 , located in appendices a and b respectively, present sample generated responses for the two benchmarks, encompassing both the baseline and the optimal auxiliary model. ## conclusion in conclusion, our research has focused on improving the quality of generated responses using gpt-2 by proposing two auxiliary tasks. the first task, referred to as utterance permutation, aims to enhance the model's ability to comprehend the interconnections between words in a sentence and produce grammatically accurate responses. the second task, utterance masking, is designed to improve the coherence and consistency of the generated responses by challenging the model to predict masked words based on the surrounding context. our experiments indicate that combining these two auxiliary tasks substantially improves the quality of generated responses. this includes improved grammar, coherence, and consistency, which are crucial aspects of generating high-quality nlp re-sponses. furthermore, these results demonstrate the potential of incorporating auxiliary tasks with language models to enhance their performance and improve the quality of generated responses in nlp applications. a a comparison of vanilla gpt-2 and gpt-2 with auxiliary tasks in conversational contexts for the dailydialogue dataset model conversation ground response generated response vanillagpt-2 [sm] -by the way miss, where is the toilet? -toilets are in the rear, i am afraid all the toilets are fully occupied at the moment. -what? oh, what we live! thank you very much for your help, miss. you are welcome. -thanks. -i'll see you in beijing in a few days. -goodbye, bye. upd [sm][context] -by the way miss, where is the toilet? -toilets are in the rear, i am afraid all the toilets are fully occupied at the moment. -what? oh, what we live! thank you very much for your help, miss. you are welcome.